Convex domains and the Fourier transform on spaces of constant curvature

Toshiyuki Kobayashi*

School of Mathematics Institute for Advanced Study Princeton, NJ 08540, USA

and

Department of Mathematical Sciences

University of Tokyo

Komaba, Meguro, Tokyo, 153, Japan toshi@tansei.cc.u-tokyo.ac.jp

Typeset by AMS-TEX

*Supported by the NSF grant DMS-9100383.

Preface

These notes represent a series of twelve expository lectures delivered by the author at the CIMPA (Centre international de mathematiques pures et appliquees) Spring School on “Invariant Differential Operators on Lie Groups and Homogeneous Spaces” at WuHan University in P. R. China, from April 21 to May 11, 1991.

The main subject here is the study of the analytic set 𝒩(Ω) (null variety) associated to a bounded domain Ω in ℝ^{n} and in the hyperbolic space SO_{0}(n, 1)/SO(n). In particular, we are interested in the injectivity of Ω ↦ 𝒩(Ω), in the description of 𝒩(Ω) in an asymptotic sense and in the Pompeiu problem where 𝒩(Ω) contains a hypersphere. We try to present these topics in an elementary fashion.

Here is a brief outline of the contents:

In §1 we discuss three different topics: the null variety 𝒩(Ω) = {ζ ∈ ℂ^{n}:

∫_{Ω} e^{\sqrt{−1}(x,ζ)}dx = 0\}, integral geometry and a free boundary problem. In a special case, called the Pompeiu problem or Schiffer’s conjecture, it is well known that these topics are closely related with one another (Theorem (1.4.1)). We also set forth various related problems and give a short survey of results known so far. However, many of the basic questions have not found a final answer.

In §2 we deal with the asymptotic behavior of the zeros of some holomorphic functions of one variable. It is a classical result that the zeros of some special functions (e.g. the Bessel function J_{ν}(ζ), the associated Legendre function P_{ζ− \frac{1}{2}}^{λ} (z) and so on) are regularly distributed, and that most of them are real if the parameter involved is ‘real’. We generalize this from the viewpoint of integral transforms. That is, after a definition of appropriate function spaces and their norms, we give a description of the asymptotic behavior of the zeros of their Fourier transforms and find error estimates in terms of their norms. Since the author does now know convenient references for these results, we have given essentially complete treatments of these materials.

In §3 we discuss an asymptotic behavior of the Fourier transform of a convex domain in ℝ^{n}, using a method based on the Radon transform and going back to F. John. Combining this with the results of §2, we give an asymptotic behavior of the null variety 𝒩(Ω) (precisely, the intersection with an n + 1 dimensional real manifold). This result yields partial solutions to Problems A and B posed in §1.

§4 deals with four further results concerning the null variety 𝒩(Ω) in the case n = 2. That is, a positive solution to the injectivity problem ‘if one knows 𝒩(Ω), can Ω be recovered?’ for strictly convex domains (§4.1), a characterization of convexity of Ω in terms of 𝒩(Ω) (§4.2), and partial positive solutions to Schiffer’s conjecture in the case where Ω is a thin convex domain (§4.3) and ∂Ω is algebraic (§4.4). §5 treats the Pompeiu problem under a perturbation of a ball in ℝ^{n} by using spherical harmonics. §4.2 and §5 are new results.

In §6 we present a rapid outline of some basic concepts of semisimple Lie groups, (Riemannian) symmetric spaces, spherical homogeneous spaces, invariant differential operators and Fourier transforms. We develop more materials than we need, hoping that the reader who is not very familiar with them could get some ‘feeling’ about how and why they work.

Introducing briefly (horospherical) convexity, a pair of ‘Gauss maps’ and a pair of ‘supporting functions’ with their basic properties in a Riemannian symmetric space of the non-compact type of rank one in §7, we generalize some of the Euclidean results in §3 to the hyperbolic space SO_{0}(n, 1)/SO(n) in §8. We collect some classical results on Bessel functions in §9 as an appendix.

I should like to thank Jean-Michel Lemaire, Guy Roos, Pierre Torasso who directed the school, Yu Jia-Rong, Deng Hong and the people of the China-French center of Mathematics and Information and of the Department of mathematics at WuHan University for their hospitality, and the other participants for stimulating feedback during the lectures. I have benefitted from various discussions with and comments by Izrail Gelfand, Sigurdur Helgason, Bent Ørsted, Toshio Oshima and Henrik Schlichtkrull. Hideko Sekiguchi provided a list of errors in earlier versions, which was very helpful. Charles Conley and Atsuko Yamamoto have examined parts of the manuscript, and provided helpful advice throughout its writing. Carlos Berenstein made detailed comments and suggestions on the entire manuscript. I take this opportunity to thank all of them now; I apologize for those errors which undoubtedly remain. The final manuscript was written while a member of the School of Mathematics of the Institute for Advanced Study, for whose hospitality I am grateful. In particular, I am grateful to Ms. Dottie Phares for her very skillful typing of the manuscript.

Table of Contents

§1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 §2 The zeros of a certain class of entire functions . . . . . . . . . . . . . . . 19 §3 Convex domains Ω in ℝ^{n} and asymptotics of the null variety 𝒩(Ω) . . . . . . 49 §4 Some more results on 𝒩(Ω) in the case Ω in ℝ^{2} . . . . . . . . . . . . . . 59 §5 Pompeiu problem and a perturbation of a ball in ℝ^{n} . . . . . . . . . . . . 63 §6 A quick review of semisimple Lie groups, symmetric spaces, invariant

differential operators and intgral transforms . . . . . . . . . . . . . 69 §7 Submanifolds and convexity in a rank one symmetric space . . . . . . . . . 88 §8 Asymptotics of the null variety 𝒩(Ω) in the hyperbolic space . . . . . . . . 98 §9 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 A scheme showing the dependence of the various sections is indicated as follows:

1  2  6
↙  ↘  ↙  ↓
5  3  ↘  7
↓  ↓
4  8