ANALYSIS ON THE MINIMAL REPRESENTATION OF O(p, q)

– III. ULTRAHYPERBOLIC EQUATIONS ON ℝ^{p−1,q−1}

Toshiyuki KOBAYASHI and Bent ØRSTED

RIMS Kyoto and SDU-Odense University

Abstract. For the group O(p, q) we give a new construction of its minimal unitary representation via Euclidean Fourier analysis. This is an extension of the q = 2 case, where the representation is the mass zero, spin zero representation realized in a Hilbert space of solutions to the wave equation. The group O(p, q) acts as the Möbius group of conformal transformations on ℝ^{p−1,q−1}, and preserves a space of solutions of the ultrahyperbolic Laplace equation on ℝ^{p−1,q−1}. We construct in an intrinsic and natural way a Hilbert space of solutions so that O(p, q) becomes a continuous irreducible unitary representation in this Hilbert space. We also prove that this representation is unitarily equivalent to the representation on L^{2}(C), where C is the conical subvariety of the nilradical of a maximal parabolic subalgebra obtained by intersecting with the minimal nilpotent orbit in the Lie algebra of O(p, q).

Contents

§1. Introduction §2. Ultrahyperbolic equation on ℝ^{p−1,q−1} and conformal group §3. Square integrable functions on the cone §4. Green function and inner product §5. Bessel function and an integral formula of spherical functions §6. Explicit inner product on solutions □_{ℝ^{p−1,q−1}}f = 0

§1. Introduction

\underline{1.1.} In this paper we study the symmetries of the ultrahyperbolic Laplace operator on a real finite-dimensional vector space equipped with a non-degenerate symmetric bilinear form. We shall work in coordinates so that the operator becomes

□_{ℝ^{p−1,q−1}} ≡ □_{z} := \frac{∂^{2}}{∂z_{1}^{2}} + ⋯ + \frac{∂^{2}}{∂z_{p−1}^{2}} − \frac{∂^{2}}{∂z_{p}^{2}} − ⋯ − \frac{∂^{2}}{∂z_{p+q−2}^{2}} ,

on ℝ^{n} = ℝ^{p−1,q−1}. In the case of Minkowski space (q = 2) we are studying the wave equation, which is well-known to have a conformally invariant space of solutions, see [14]. This corresponds to the fact that the equation □_{ℝ^{p−1,q−1}}f = 0 in this case describes a particle of zero mass. Incidentally, it may also be interpreted as the bound states of the Hydrogen atom, namely each energy level corresponds to a K-type - for (p, q) = (4, 2). This gives the angular momentum values by further

Typeset by AMS-TEX

restriction to O(3). In general the indefinite orthogonal group G = O(p, q) acts as the Möbius group of meromorphic conformal transformations on ℝ^{p−1,q−1}, leaving a space of solutions to □_{ℝ^{p−1,q−1}}f = 0 invariant.

\underline{1.2.} The main object of the present paper is to construct in an intrinsic and natural way a Hilbert space of solutions of □_{ℝ^{p−1,q−1}} so that the action of O(p, q) becomes a continuous unitary irreducible representation in this Hilbert space for (p, q) such that p, q ≥ 2 and p + q > 4 is even. From an algebraic view point of representation theory, our representations are:

i) minimal representations if p + q ≥ 8 (i.e. the annihilator is the Joseph ideal). ii) not spherical if p ≠ q (i.e. no non-zero K-fixed vector). iii) not highest weight modules of SO_{0}(p, q) if p, q ≥ 3. In a long history of representation theory of semisimple Lie groups, it is only

quite recent that our representations for p, q ≥ 3 have been paid attention, especially as minimal unitary representations; they were first discovered by Kostant [13] for (p, q) = (4, 4) and generalized by Binegar-Zierau [3] as subrepresentations of degenerate principal series representations. There is also another algebraic approach to the same representations by using the theta correspondence for the trivial representation of SL(2, ℝ) by Huang-Zhu. Our previous papers [11] and [12] treated the same representation by geometric methods and with other points of view. We think that such various approaches reflect a rich structure of the minimal representations.

It is perhaps of independent interest that the (in some sense maximal group of) symmetries O(p, q) of the space of solutions of □_{ℝ^{p−1,q−1}}f = 0 lead to such a natural Hilbert space. Our inner product ( , )_{W} defined by an integration over a non-characteristic hyperplane (see (1.5.1)) is a generalization of the one coming from energy considerations in the case of wave equations, and even the translation invariance of the inner product contains some new information about solutions.

It is also of independent interest from the representation theory of semisimple Lie groups that our representations are unitarily equivalent to the representations on L^{2}(C), where C is the null cone of the quadratic form on ℝ^{p−1,q−1}. This result is proved via the Fourier transform in Theorem 4.9. Such L^{2}-realizations of “unipotent representations” is expected from the philosophy of the Kostant-Kirillov orbit method, but has not been proved except for some special cases of highest weight modules or spherical representations.

We have avoided most of the references to the theory of semisimple Lie groups and representation theory, and instead given direct constructions of the key objects, such as for example the minimal K-type; this is given as an explicit hypergeometric function, and we also calculate its Fourier transform in terms of a Bessel function. By application of explicit differential operators forming the Lie algebra of G we can generate the whole Hilbert space of solutions beginning from the minimal K-type.

\underline{1.3.} For q = 2 (or p = 2) we are dealing with highest weight representations (when restricting to the identity component SO_{0}(p, 2)), and these have been studied by many authors, in particular in the physics literature. For a nice introduction to this representation and its construction via geometric quantization (and more) see [8]. In this case the K-types may be identified with energy levels of the bound states of the Hydrogen atom, and the smallest one with the bound state of lowest energy.

We can summarize the situation, covering both the classical Kepler problem and its quantization in case q = 2, as in the diagram below. Here the left-hand side represents the classical descriptions of respectively the Kepler problem and geodesic flow on the sphere; by “symplectic transform” we are alluding to the change of variables between these two Hamiltonian systems as presented in [8]. The righthand side involves the quantizations of these two systems, where the wave-equation is considered as the quantization of geodesic flow, also to be thought of as geometric optics. The quantum analogue of the “symplectic transform” involves the Fourier transform. Finally we invoke conformal geometry and combine it with the Fourier transform, which in a different (and new as far as constructing Hilbert spaces and unitary actions) way appears in passing from the wave equation to the Fourier realization of solutions - this is the last arrow on the right-hand side.

Classical ⇒ Quantum

Kepler problem ⇒

Hydrogen atom

⇓ symplectic transform  ⇓ Fourier transform

geometric optics ⇒ (A) wave equation

conformal geometry ⇓ Fourier transform

(B) other realizations and

explicit inner products of minimal representations

The main focus of this paper is on the boxes (A) and (B). In particular, we give an explicit inner product in the model (A) (Theorems 1.4 and 1.5) and construct via Fourier transform a new realization of the minimal representation (Theorem 1.6) for general p, q.

\underline{1.4.} From now, suppose that n := p + q − 2 is an even integer greater than 2, and p, q ≥ 2. Let us briefly state some of our main results in a more explicit way.

First, we find a formula of the Green function E_{0} of the ultrahyperbolic Laplace operator □_{ℝ^{p−1,q−1}}, in Proposition 4.2, namely, E_{0} is given by a constant multiple of the imaginary part of the regularized Schwartz distribution:

e^{\frac{\sqrt{−1}π(q−1)}{2}} (x_{1}^{2} + ⋯ + x_{p−1}^{2} − x_{p}^{2} − ⋯ − x_{p+q−1}^{2} + \sqrt{−1}0)^{1− \frac{n}{2}} .

See the recent paper of Hörmander [9] for further details on distributions associated with this ultrahyperbolic equation. Then we construct solutions of □_{ℝ^{p−1,q−1}}f = 0 by the integral transformation:

S : C_{0}^{∞}(ℝ^{n}) → C^{∞}(ℝ^{n}), φ ↦ E_{0} ∗ φ (see (4.3.1)).

The image S(C_{0}^{∞}(ℝ^{n})) turns out to be “large” in Ker □_{ℝ^{p−1,q−1}} (see §4.7, Remark (2)). On this image, we define a Hermitian form ( , )_{N} by

(1.4.1)  (f_{1}, f_{2})_{N} := ∫_{ℝ^{n}} ∫_{ℝ^{n}} E_{0}(y − x)φ_{1}(x)\overline{φ_{2}(y)} dxdy,

where f_{i} = E_{0} ∗ φ_{i} (i = 1, 2). Here is a part of Theorem 4.7, which is the first of our main results:

Theorem 1.4. ( , )_{N} is positive-definite on the image of S. Furthermore, O(p, q) acts as an irreducible unitary representation on its Hilbert completion H.

We shall write (ϖ_{ℝ^{p−1,q−1}}^{min}, H) for this unitary representation. We also prove that this representation is isomorphic to the minimal representation of O(p, q), constructed previously by Kostant, Binegar-Zierau ([13], [3]). Thus, Theorem 1.4 may be regarded as a realization of the minimal representation (with an explicit inner product) in the solution space of the ultrahyperbolic equation.

\underline{1.5.} The above definition of the inner product ( , )_{N} (see (1.4.1)) uses the integral expression of solutions of □_{ℝ^{p−1,q−1}}f = 0. Can we write the inner product without knowing the preimage? Yes, the second of our main results is to give an intrinsic inner product on the same solution space by using the Cauchy data. For simplicity, we take z_{1} = 0 as a non-characteristic hyperplane. Then, we decompose a solution

f = f_{+} + f_{−}

such that f_{±}(z_{1}, …, z_{n}) is holomorphic with respect to the first variable z_{1} in the complex domain of {z_{1} ∈ ℂ : ± Im z_{1} > 0} of the z_{1}-variable. This is an expression of f as a hyperfunction, and such a pair (f_{+}, f_{−}) can be obtained by the convolution in the z_{1}-variable (see (6.2.3)):

f_{±}(z) = \frac{1}{2π\sqrt{−1}} · \frac{±1}{z_{1} ∓ \sqrt{−1}0} ∗ f(z_{1}, …, z_{n}),

where the integration makes sense for f with suitable decay at infinity. Then we define a Hermitian form

(1.5.1)  (f, f)_{W} := \frac{1}{\sqrt{−1}} ∫_{ℝ^{n−1}} (f_{+} \overline{\frac{∂f_{+}}{∂z_{1}}} − f_{−} \overline{\frac{∂f_{−}}{∂z_{1}}} ) |_{z_{1}=0} dz_{2} ⋯ dz_{n}.

Then we shall prove that ( , )_{W} is independent of the specific choice of a noncharacteristic hyperplane, as follows from the (non-trivial) isometric invariance. Much more strongly, ( , )_{W} is conformally invariant. A precise formulation for this is given in Theorem 6.2, which includes:

Theorem 1.5. 4π( , )_{W} = ( , )_{N}. In particular, ( , )_{W} is positive definite and O(p, q)-invariant.

Hence, in place of Theorem 1.4, we can obtain the same irreducible unitary representation of O(p, q) on the Hilbert completion of a space of solutions with respect to the inner product ( , )_{W}.

An interesting property of this inner product is its large invariance group. Even in the case of the usual wave equation (q = 2 case) our approach to the Hilbert space of solutions and the corresponding representation offers some new points of view. In this case, if we take the non-characteristic hyperplane as fixed time coordinate (namely, if z_{1} stands for the time), then the translational invariance amounts to a remarkable “conservation law”. Instead, we can take the non-characteristic hyperplane by fixing one of the space coordinates, and an analogous integration over the hypersurface (containing the time coordinate) still gives the same inner product ! As a final remark in §6.7, we note the connection to the theory of conserved quantities for the wave equation (q = 2 case), such as the energy and others obtained by the action of the conformal group.

\underline{1.6.} The Gelfand-Kirillov dimension of our representation (ϖ_{ℝ^{p−1,q−1}}^{min}, H) is p + q − 3. So, we may expect that the representation could be realized on a (p + q − 3)- dimensional manifold. For this purpose, we define the null cone of the metric as

C := {ζ ∈ ℝ^{n} : ζ_{1}^{2} + ⋯ + ζ_{p−1}^{2} − ζ_{p}^{2} − ⋯ − ζ_{n}^{2} = 0}.

The third of our main results is another realization of the unitary representation (ϖ_{ℝ^{p−1,q−1}}^{min}, H) in a function space on a (p + q − 3)-dimensional manifold C. The Fourier transform F maps solutions of □_{ℝ^{p−1,q−1}}f = 0 to distributions supported on the null cone C. Surprisingly, the inner product of our Hilbert space turns out to be simply the L^{2}-norm on C with respect to a canonical measure dμ (see (3.3.3)) ! Here is a part of Theorem 4.9: We regard L^{2}(C) as a subspace of distributions by a natural injective map T : L^{2}(C) → S′(ℝ^{n}).

Theorem 1.6. (2π)^{− \frac{n}{2}} T^{−1} ∘ F is a surjective unitary operator from H to L^{2}(C).

Theorem 1.6 defines an irreducible unitary representation of G = O(p, q) on L^{2}(C), denoted by π, which is unitarily equivalent to (ϖ_{ℝ^{p−1,q−1}}^{min}, H). Since the maximal parabolic subgroup \overline{P^{max}} of G (see §2.7) acts on ℝ^{p−1,q−1} as affine transformations, the restriction π|\overline{P^{max}} has a very simple form, namely, the one obtained by the classical Mackey theory (see (3.3.5)). In this sense, Theorem 1.6 may be also regarded as an extension theorem of an irreducible unitary representation from the maximal parabolic subgroup \overline{P^{max}} to the whole group G.

\underline{1.7.} The fourth of our main results is about the representation (π, L^{2}(C)) as (𝔤, K)-modules on the Fourier transform side, especially to find an explicit vector in the minimal K-type.

In the realization on L^{2}(C), the action π(g) is not simple to describe except for g ∈ \overline{P^{max}}. Instead, we consider the differential action dπ of the Lie algebra 𝔤_{0} on smooth vectors of L^{2}(C), which turns out to be given by differential operators at most of second order (see §3.2). This makes the analogy with the metaplectic representation (where G is replaced by the symplectic group) a good one . Here we are recalling the fact, that the even part of the metaplectic representation may be realized as an L^{2}-space of functions on the cone generated by rank one projections in ℝ^{n}.

Moreover, by using a reduction formula of an Appell hypergeometric function, we find explicitly the Fourier transform of a Jacobi function multiplied by some conformal factor which equals to a scalar multiple of

ψ_{0,e}(ζ) := |ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|)dμ ∈ S′(ℝ^{n}).

Here K_{ν}(ζ) is a modified Bessel function of the second kind. This vector ψ_{0,e}(ζ) corresponds to the bound state of lowest energy for q = 2 case . For general p, q, the K-span of ψ_{0,e}(ζ) generates the minimal K-type in the realization on L^{2}(C).

We define a subspace U of S′(ℝ^{n}) to be the linear span of its iterative differentials

dπ(X_{1}) ⋯ dπ(X_{k})ψ_{0,e}(ζ) (X_{1}, …, X_{k} ∈ 𝔤_{0} ⊗_{ℝ} ℂ).

What comes out of §5 may be formulated in this way (combining with Theorem 4.9, see §3.2 for notation): Suppose p + q ∈ 2ℤ, p + q > 4 and p ≥ q ≥ 2.

Theorem 1.7. 1) |ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|) is a K-finite vector in L^{2}(C).

2) U is an infinitesimally unitary (𝔤, K)-module via ϖ_{\frac{n−2}{2} ,ϵ}.

3) U is dense in the Hilbert space T (L^{2}(C)).

4) The completion of (2) defines an irreducible unitary representation of G on T (L^{2}(C)), and then also on L^{2}(C).

In the paper [4] one finds a similar construction of Hilbert spaces and unitary representations for Koecher-Tits groups associated with semisimple Jordan algebras under the assumption that the representations are spherical, and there also occur Bessel functions as spherical vectors. In our situation the representations are not spherical if p ≠ q. Our approach is completely different from [4], and contrary to what is stated in [4] (p. 206) we show that for G = O(p, q) it is possible to extend the Mackey representation of the maximal parabolic subgroup to the whole group. Furthermore, even for p = q case, our approach for Theorem 1.7 has an advantage that we give the exact constants normalizing the unitary correspondence between the minimal K-type in other realizations and the Bessel function in our realization on L^{2}(C) (see Theorem 5.5).

\underline{1.8.} The paper is organized as follows: We begin by recalling some results from conformal geometry and facts about the conformal group, in particular in [11]. In section 3 we give the basic setup for a realization on the null cone via Fourier transform. Then we construct the intertwining operator from the minimal representation to the model treated here and calculate the new expression for the inner product (see Theorem 1.4). We show in Proposition 4.2 that the Green function of □_{ℝ^{p−1,q−1}} has a Fourier transform equal to the invariant measure on the null-cone, allowing one more expression for the inner product (see Theorem 1.6); also we obtain from this an intertwining operator from test functions to solutions. Indeed, in section 4 Proposition 4.6 we prove that the Green function is up to a constant exactly the kernel in the Knapp-Stein intertwining integral operator between degenerate principal series representations at the parameters we study; this makes the proof of unitarity of the minimal representation elementary and formulated independent of the semi-simple theory usually employed here. Note that all normalizing constants are computed explicitly. Lemma 2.6 states the irreducibility and unitarizability, which we use; we give in [12] independent proofs of these facts.

In section 5 we construct the lowest K-type as a modified Bessel function, whose concrete properties are important for K-type information about L^{2}(C). The idea here is to use a classical formula on the Hankel transform due to Baily in 1930s, and then apply reduction formulae of an Appell hypergeometric function of two variables.

Section 6 contains formulae for the inner product ( , )_{W} in terms of integration over a Cauchy hypersurface. Summarizing, we give five different realizations of the inner product together with the normalizations of these relative to each other. Namely, in addition to ( , )_{N} and ( , )_{W} we also define three more: ( , )_{M} (coming from a pseudodifferential operator on M = S^{p−1} × S^{q−1}), ( , )_{A} (coming from a normalized Knapp-Stein intertwining operator), and finally ( , )_{C}, which is just L^{2}(C). This is seen in the key diagram (see Section 4.11)

C_{0}^{∞}(ℝ^{n}) →^{S}Ψ∗_{\frac{n−2}{2}} (Δ̃_{M}) →^{F} S′(ℝ^{n}) ↩^{T} L^{2}(C)
∩
Ker □_{ℝ^{p−1,q−1}}

where the spaces correspond to four different ways of generating solutions to our ultrahyperbolic equation. S will be an integral transform against the Green kernel (essentially, a Knapp-Stein intertwining operator with a specific parameter), and F the Fourier transform, mapping solutions to distributions supported on the null cone C. Correspondingly to the various ways of generating solutions, we write down explicitly the unitary inner product and its Hilbert space. We have tried to avoid the use of any semi-simple theory and stay within classical analysis on spheres and Euclidean spaces; still our treatment may also be of interest to people working with the classification of the unitary dual of semi-simple Lie groups, since we are providing new models of some unipotent representations. Tools like the standard Knapp-Stein intertwining operators become very natural to use here, also from the more elementary viewpoint, and the close connection between these and Green functions for ultrahyperbolic differential operators seems not to have been noticed before.

The first author expresses his sincere gratitude to SDU - Odense University for the warm hospitality.

§2. Ultrahyperbolic equation on ℝ^{p−1,q−1} and conformal group

\underline{2.1.} As explained in the Introduction, we shall give a flat picture, the so-called Npicture, of the minimal representation, which is connected to classical facts about conformal geometry in ℝ^{n}. We shall give a unitary inner product in this realization (see Theorem 6.2) and also in its Fourier transform (Theorem 4.9), together with an explicit form of minimal K-type in this realization (see Theorem 5.5).

We shall assume p+q ∈ 2ℕ, p ≥ 2, q ≥ 2 and (p, q) ≠ (2, 2). The parity condition p + q ∈ 2ℕ is not necessary when we consider a representation of the parabolic subgroup \overline{P^{max}} or of the Lie algebra 𝔤. Indeed, it will be interesting to relax this parity condition in order to obtain an infinitesimally unitary representation, which does not integrate to a global unitary representation of G.

Throughout this paper, we let

n = p + q − 2

This section is written in an elementary way, intended also for non-specialists of semisimple Lie groups. §2.2 and §2.6 review the needed results in [11].

\underline{2.2.} We recall some basic fact of the distinguished representation of a conformal group (see [11], § 2). Let M be an n-dimensional manifold with pseudo-Riemannian metric g_{M}. We denote by Δ_{M} the Laplace operator on M, and by K_{M} the scalar curvature of M. The Yamabe operator is defined to be

Δ̃_{M} := Δ_{M} − \frac{n − 2}{4(n − 1)}K_{M}.

Suppose (M, g_{M}) and (N, g_{N}) are pseudo-Riemannian manifolds. A local diffeomorphism Φ: M → N is called a conformal map if there exists a positive-valued function Ω on M such that Φ^{∗}g_{N} = Ω^{2}g_{M}. For λ ∈ ℂ, we introduce a twisted pull-back

(2.2.1)  Φ_{λ}^{∗}: C^{∞}(N) → C^{∞}(M), f ↦ Ω^{λ} · f ∘ Φ.

Then the conformal quasi-invariance of the Yamabe operator is expressed by:

(2.2.2)  Φ∗_{\frac{n+2}{2}} Δ̃_{N} = Δ̃_{M}Φ∗_{\frac{n−2}{2}} .

Let G be a Lie group acting conformally on M. If we write the action as x ↦ L_{h}x (h ∈ G, x ∈ M), we have a positive function Ω(h, x) ∈ C^{∞}(G × M) such that

L_{h}^{∗}g_{M} = Ω(h, ·)^{2} g_{M} (h ∈ G).

We form a representation ϖ_{λ} of G, with parameter λ ∈ ℂ, on C^{∞}(M) as follows:

(2.2.3)  ϖ_{λ}(h^{−1})f(x) = Ω(h, x)^{λ}f(L_{h}x), (h ∈ G, f ∈ C^{∞}(M), x ∈ M).

Note that the right-hand side is given by the twisted pull-back (L_{h})_{λ}^{∗} according to the notation (2.2.1). Then, Formula (2.2.2) implies that Δ̃_{M}: C^{∞}(M) → C^{∞}(M) is a G-intertwining operator from ϖ_{\frac{n−2}{2}} to ϖ_{\frac{n+2}{2}} . Thus, we have constructed a distinguished representation of the conformal group:

Lemma 2.2 (see [11], Theorem 2.5). Ker Δ̃_{M} is a representation space of the conformal group G of a pseudo-Riemannian manifold (M, g_{M}), through ϖ_{\frac{n−2}{2}} .

If (N, g_{N}) is also a pseudo-Riemannian manifold on which the same group G acts conformally, then one can also define a representation ϖ_{λ,N} on C^{∞}(N). Then the twisted pull-back Φ_{λ}^{∗} is a G-intertwining operator.

\underline{2.3.} Here is a setup on which we construct the minimal representation of O(p, q) by applying Lemma 2.2. Let p, q ≥ 2. We note n = p+q−2. We write {e_{0}, …, e_{n+1}} for a standard basis of ℝ^{p+q} and the corresponding coordinate as

(v_{0}, …, v_{n+1}) = (x, y) = (v_{0}, z′, z′′, v_{n+1}),

where x ∈ ℝ^{p}, y ∈ ℝ^{q}, z′ ∈ ℝ^{p−1}, z′′ ∈ ℝ^{q−1}. The notation (x, y) will be used for S^{p−1} × S^{q−1}, while (z′, z′′) for ℝ^{n} = ℝ^{(p−1)+(q−1)}. The standard norm on ℝ^{l} will be written as | · | (l = p − 1, p, q − 1, q).

We denote by ℝ^{p,q} the pseudo-Riemannian manifold ℝ^{p+q} equipped with the flat pseudo-Riemannian metric:

(2.3.1)  g_{ℝ^{p,q}} = dv_{0}^{2} + ⋯ + dv_{p−1}^{2} − dv_{p}^{2} − ⋯ − dv_{n+1}^{2}.

We put two functions on ℝ^{p+q} by

(2.3.2)  ν: ℝ^{p+q} → ℝ,  (x, y) ↦ |x|,
(2.3.3)  μ: ℝ^{p+q} → ℝ, (v_{0}, …, v_{n+1}) ↦ \frac{1}{2}(v_{0} + v_{n+1}).

and define three submanifolds of ℝ^{p,q} by

Ξ := {(x, y) ∈ ℝ^{p,q} : |x| = |y| ≠ 0},
M := {v ∈ ℝ^{p,q} : ν(v) = 1} ∩ Ξ  = S^{p−1} × S^{q−1},
N := {v ∈ ℝ^{p,q} : μ(v) = 1} ∩ Ξ  ←^{∼}_{ι} ℝ^{n}.

where the bijection ι: ℝ^{n} → N is given by

(2.3.4)  ι: ℝ^{n} → N, (z′, z′′) ↦ (1 − \frac{|z′|^{2} − |z′′|^{2}}{4} , z′, z′′, 1 + \frac{|z′|^{2} − |z′′|^{2}}{4} ).

We say a hypersurface L of Ξ is transversal to rays if the projection

(2.3.5)  Φ: Ξ → M, v ↦ \frac{v}{ν(v)}

induces a local diffeomorphism Φ|_{L}: L → M. Then, one can define a pseudo-Riemannian metric g_{L} of signature (p − 1, q − 1) on L by the restriction of g_{ℝ^{p,q}}. In particular, M itself is transversal to rays, and the induced metric g_{S^{p−1}×S^{q−1}} equals g_{S^{p−1}} ⊕ (−g_{S^{q−1}}), where g_{S^{n−1}} denotes the standard Riemannian metric on the unit sphere S^{n−1}. Likewise, the induced pseudo-Riemannian metric on ℝ^{n} through ι: ℝ^{n} ↪ ℝ^{p,q} coincides with the standard flat pseudo-Riemannian metric g_{ℝ^{p−1,q−1}} on ℝ^{n}.

\underline{2.4.} Let I_{p,q} := diag(1, …, 1, −1, …, −1) ∈ GL(p + q, ℝ). The indefinite orthogonal group

G = O(p, q) := {g ∈ GL(p + q, ℝ) : ^{t}gI_{p,q}g = I_{p,q}},

acts isometrically on ℝ^{p,q} by the natural representation, denoted by v ↦ g · v. This action stabilizes the light cone Ξ. We note that the multiplicative group ℝ_{+}^{×} := {r ∈ ℝ : r > 0} also acts on Ξ as dilation, which commutes with the linear action of G. Then, using dilation, one can define an action of G on M, and also a meromorphic action on ℝ^{p−1,q−1} as follows:

(2.4.1) L_{h}: M → M,  v ↦ \frac{h · v}{ν(h · v)}  (h ∈ G),
(2.4.2) L_{h}: ℝ^{p−1,q−1} → ℝ^{p−1,q−1}, z ↦ ι^{−1} (\frac{h · ι(z)}{μ(h · ι(z))}) (h ∈ G).

Then, both of these actions are conformal:

(2.4.3)  (L_{h})^{∗}g_{M} = ν(h · v)^{−2}g_{M}
(2.4.4)  (L_{h})^{∗}g_{ℝ^{p−1,q−1}} = μ(h · ι(z))^{−2}g_{ℝ^{p−1,q−1}}.

We note that (2.4.2) and (2.4.4) are well-defined if μ(h · ι(z)) ≠ 0. In fact, G acts only meromorphically on ℝ^{p−1,q−1}. An illustrative example for this feature is the linear fractional transformation of SL(2, ℂ) on ℙ^{1}ℂ = ℂ ∪ {∞}, which is a meromorphic action on ℂ. This example essentially coincides with (2.4.2) for (p, q) = (3, 1), since SL(2, ℂ) is locally isomorphic to O(3, 1) and ℂ ≃ ℝ^{2}.

\underline{2.5.} The (meromorphic) conformal groups for the submanifolds M and N of Ξ are the same, namely, G = O(p, q), while their isometry groups are different subgroups of G, as we shall see in Observation 2.5. In order to describe them, we define subgroups K, M^{max}, N^{max}, A^{max} and \overline{N^{max}} of G as follows:

First, we set

m_{0} := −I_{p+q},
K := G ∩ O(p + q)  = O(p) × O(q),
M_{+}^{max} := {g ∈ G : g · e_{0} = e_{0}, g · e_{n+1} = e_{n+1}} ≃ O(p − 1, q − 1),
M^{max} := M_{+}^{max} ∪ m_{0}M_{+}^{max}  ≃O(p − 1, q − 1) × ℤ_{2}.

The Lie algebra of G is denoted by 𝔤_{0} = 𝔬(p, q), which is given by matrices:

𝔤_{0} ≃ {X ∈ M(p + q, ℝ) : XI_{p,q} + I_{p,q}^{t}X = O}.

Next, we keep n = p + q − 2 in mind and put

(2.5.1)  ε_{j} = \{ \begin{matrix}1 & (1 ≤ j ≤ p − 1), \\ −1 & (p ≤ j ≤ n), \end{matrix}

and define elements of 𝔤_{0} as follows:

(2.5.2)(a) \overline{N}_{j} := E_{j,0} + E_{j,n+1} − ε_{j}E_{0,j} + ε_{j}E_{n+1,j} (1 ≤ j ≤ n),
(2.5.2)(b) N_{j} := E_{j,0} − E_{j,n+1} − ε_{j}E_{0,j} − ε_{j}E_{n+1,j} (1 ≤ j ≤ n),
(2.5.2)(c)  E := E_{0,n+1} + E_{n+1,0},

where E_{ij} denotes the matrix unit. Now, we define abelian subgroups of G by

\overline{N^{max}} := exp(∑_{j=1}^{n} ℝ\overline{N}_{j}), N^{max} := exp(∑_{j=1}^{n} ℝN_{j}), A^{max} := exp(ℝE).

For example, M_{+}^{max} is the Lorentz group and M_{+}^{max}N^{max} is the Poincaré group if (p, q) = (2, 4). It is convenient to identify ℝ^{n} with \overline{N^{max}} by putting

(2.5.3)  \overline{n}_{a} := exp(∑_{j=1}^{n} a_{j}\overline{N_{j}}) ∈ \overline{N^{max}}  for a = (a_{1}, …, a_{n}) ∈ ℝ^{n}.

The geometric point here will be the following:

Observation 2.5. 1) On S^{p−1} × S^{q−1}, G acts conformally, while K isometrically.

2) On ℝ^{p−1,q−1}, G acts meromorphically and conformally, while the motion group M_{+}^{max}\overline{N^{max}} isometrically.

\underline{2.6.} Next, let us consider the pseudo-Riemannian manifold M = S^{p−1} × S^{q−1}. It follows from (2.3.3) and (2.4.3) that we can define a representation ϖ_{λ,M} of G on C^{∞}(M) by

(ϖ_{λ,M}(h^{−1})f)(v) := ν(h · v)^{−λ}f(L_{h}v),

The Yamabe operator on M is of the form:

Δ̃_{M} = Δ_{S^{p−1}} − Δ_{S^{q−1}} − (\frac{p − 2}{2})^{2} + (\frac{q − 2}{2})^{2} = Δ̃_{S^{p−1}} − Δ̃_{S^{q−1}}.

Applying Lemma 2.2, we obtain a representation of the conformal group G = O(p, q), denoted by (ϖ^{p,q}, V^{p,q}), as a subrepresentation of ϖ_{\frac{p+q−4}{2} ,M}:

V^{p,q} := Ker Δ̃_{M} = {f ∈ C^{∞}(M) : Δ̃_{M}f = 0},

(ϖ^{p,q}(h^{−1})f)(v) := ν(h · v)^{− \frac{p+q−4}{2}} f(L_{h}v), for h ∈ G, v ∈ M, f ∈ V^{p,q}.

The restriction of ϖ^{p,q} from the conformal group to the isometry group gives a useful knowledge on the representation ϖ^{p,q}. For this, we recall the classical theory of spherical harmonics, which is a generalization of Fourier series for S^{1}. For p ≥ 2 and k ∈ ℕ, we define the space of spherical harmonics of degree k by

(2.6.1) H^{k}(ℝ^{p}) = {f ∈ C^{∞}(S^{p−1}) : Δ_{S^{p−1}}f = −k(k + p − 2)f},

= \{f ∈ C^{∞}(S^{p−1}) : Δ̃_{S^{p−1}}f = (\frac{1}{4} − (k + \frac{p − 2}{2})^{2}) f\}.

Then O(p) acts irreducibly on H^{k}(ℝ^{p}) and the algebraic direct sum ⨁_{k=0}^{∞} H^{k}(ℝ^{p}) is dense in C^{∞}(S^{p−1}). We note that H^{k}(ℝ^{2}) ≠ {0} only if k = 0 or 1.

Now, we review a basic property of this representation (ϖ^{p,q}, V^{p,q}) on M = S^{p−1} × S^{q−1}:

Lemma 2.6 (see [3]; [11], §3). Assume p, q ≥ 2, p + q ∈ 2ℕ and (p, q) ≠ (2, 2).

1) (ϖ^{p,q}, V^{p,q}) is an infinite dimensional irreducible representation of G.

2) (K-type formula) V^{p,q} contains the algebraic direct sum

(2.6.2)  ⨁_{\begin{matrix}a,b∈ℕ \\ a+ \frac{p}{2} =b+ \frac{q}{2} \end{matrix}} H^{a}(ℝ^{p}) ⊗ H^{b}(ℝ^{q})

as a dense subspace with respect to the Fréchet topology on C^{∞}(M).

3) G preserves the norm on V^{p,q} defined by

∥F ∥_{M}^{2} := ∥(\frac{1}{4} − Δ̃_{S^{p−1}})^{\frac{1}{4}} F ∥_{L^{2}(M)}^{2} = ∑_{a≥max(0, \frac{p−q}{2} )}(a + \frac{q − 2}{2})∥F_{a,b}∥_{L^{2}(M)}^{2}.

if F = ∑_{a} F_{a,b} ∈ V^{p,q} with F_{a,b} ∈ H^{a}(ℝ^{p}) ⊗ H^{b}(ℝ^{q}) and b = a + \frac{p−q}{2} . Here, (\frac{1}{4} −Δ̃_{S^{p−1}})^{\frac{1}{4}} is a pseudo-differential operator on M, which is equal to (\frac{1}{4} −Δ̃_{S^{q−1}})^{\frac{1}{4}} on Ker Δ̃_{M}.

We write ( , )_{M} for the corresponding inner product. We denote by \overline{V^{p,q}} the Hilbert completion of V^{p,q}, on which G acts as an irreducible unitary representation of G. We shall use the same notation ϖ^{p,q} to denote this unitary representation.

If p ≥ q then V^{p,q} contains the K-type of the form 1⊠H^{\frac{p−q}{2}} (ℝ^{q}). This K-type is called a minimal K-type in the sense of Vogan, namely, its highest weight (with respect to a fixed positive root system of 𝔨_{0}) attains the minimum distance from the sum of negative roots of 𝔨_{0} among all highest weights of K-types occurring in ϖ^{p,q}. Likewise for p < q.

Remark. 1) If p + q ≥ 8, ϖ^{p,q} is called the minimal representation in the representation theory of semisimple Lie groups, in the sense that the annihilator is the Joseph ideal.

2) The formula (2.6.2) is regarded as a branching law from the conformal group G to the isometry subgroup K of the pseudo-Riemannian manifold M = S^{p−1} × S^{q−1} (see Observation 2.5). In [12], we generalized this branching law with respect to a non-compact reductive subgroup and proved the Parseval-Plancherel formula, in the framework of discretely decomposable restrictions [10].

\underline{2.7.} Let us consider the flat pseudo-Riemannian manifold ℝ^{p−1,q−1}. The Yamabe operator on ℝ^{p−1,q−1} is of the form:

□_{ℝ^{p−1,q−1}} ≡ □_{z} := \frac{∂^{2}}{∂z_{1}^{2}} + ⋯ + \frac{∂^{2}}{∂z_{p−1}^{2}} − \frac{∂^{2}}{∂z_{p}^{2}} − ⋯ − \frac{∂^{2}}{∂z_{p+q−2}^{2}} ,

because the scalar curvature on ℝ^{p−1,q−1} vanishes. Since G = O(p, q) acts on ℝ^{p−1,q−1} as a (meromorphic) conformal transform by (2.4.4), we obtain a ‘representation’ with parameter λ ∈ ℂ as in (2.2.3):

(2.7.1) ϖ_{λ,ϵ,ℝ^{n}}(g^{−1})f(z) = |μ(gι(z))|^{−λ}χ_{ϵ}(sgn(μ(gι(z))))f(L_{g}z), (g ∈ G).

Here, for ϵ = ±1, we put

χ_{ϵ}: ℝ^{×} → {±1}

by χ_{1} ≡ 1 and χ_{−1} = sgn. We may write ϖ_{λ,ϵ,ℝ^{p−1,q−1}} for ϖ_{λ,ϵ,ℝ^{n}} if we emphasize a view point of conformal geometry on the flat space ℝ^{p−1,q−1}.

We note that C^{∞}(M) is not stable by ϖ_{λ,ϵ,ℝ^{p−1,q−1}}(g^{−1}) because L_{g} is meromorphic. To make (2.7.1) a representation, we need to consider suitable class of functions controlled at infinity. One method for this is to use a conformal compactification

ℝ^{p−1,q−1} ↪ (S^{p−1} × S^{q−1})/ ∼ ℤ_{2},

and to take a twisted pull-back Ψ_{λ}^{∗} from C^{∞}(M) by a conformal map Ψ. This method is easy, and we shall explain it soon in §2.8 and §2.9. The other is to find an inner product for specific parameter λ so that G acts as a continuous unitary representation on the Hilbert space. This is particularly non-trivial for a subrepresentation, and we shall consider it for Ker □_{ℝ^{p−1,q−1}} in §6.

Before taking a suitable class of functions, we first write a more explicit form of (2.7.1). First, we note that the maximal parabolic subgroup

\overline{P^{max}} := A^{max}M^{max}\overline{N^{max}} = (ℝ_{+}^{×} × O(p − 1, q − 1) × ℤ_{2}) ⋉ ℝ^{n}

acts transitively on the manifold ι(ℝ^{n}) as affine transformations. Furthermore, M^{max}\overline{N^{max}} acts on ι(ℝ^{n}) as isometries (see Observation 2.5). Correspondingly, the representation ϖ_{λ,ϵ} ≡ ϖ_{λ,ϵ,ℝ^{n}} given in (2.7.1) has a simple form when restricted to the subgroup \overline{P^{max}}:

(2.7.2)(a)  (ϖ_{λ,ϵ}(m)f)(z) = f(m^{−1}z)  (m ∈ M_{+}^{max}),
(ϖ_{λ,ϵ}(m_{0})f)(z) = ϵf(z),
(2.7.2)(b)  (ϖ_{λ,ϵ}(e^{tE})f)(z) = e^{λt}f(e^{t}z)  (t ∈ ℝ),
(2.7.2)(c)  (ϖ_{λ,ϵ}(\overline{n}_{a})f)(z) = f(z − 2a)  (a ∈ ℝ^{n}).

Second, we write an explicit formula of the differential action of (2.7.1). We define a linear map

ω: 𝔤_{0} → C^{∞}(ℝ^{n})

by the Lie derivative of the conformal factor Ω(h, z) := μ(h · ι(z))^{−1} (see (2.4.4)). For Y = (Y_{i,j})_{0≤i,j≤n+1} ∈ 𝔤_{0} and z ∈ ℝ^{n}, we have

(2.7.3)  ω(Y)_{z} := \frac{d}{dt}|_{t=0}Ω(e^{tY} , z) = −Y_{0,n+1} − \frac{1}{2} ∑_{j=1}^{n}(Y_{0,j} + Y_{n+1,j})z_{j}.

We write the Euler vector field on ℝ^{n} as

(2.7.4)  E_{z} = ∑_{j=1}^{n} z_{j} \frac{∂}{∂z_{j}} .

Then the differential dϖ_{λ}: 𝔤_{0} → End(C^{∞}(ℝ^{n})) is given by

(2.7.5) dϖ_{λ}(Y) = −λω(Y) − ω(Y)E_{z}

− ∑_{i=1}^{n} \{(Y_{i,0} + Y_{i,n+1}) + \frac{|z′|^{2} − |z′′|^{2}}{4} (−Y_{i,0} + Y_{i,n+1}) + ∑_{j=1}^{n} Y_{i,j}z_{j}\} \frac{∂}{∂z_{j}}

for Y = (Y_{i,j})_{0≤i,j≤n+1} ∈ 𝔤_{0} and z ∈ ℝ^{n}. In particular, we have

(2.7.6) dϖ_{λ}(N_{j}) = −λε_{j}z_{j} − ε_{j}z_{j}E_{z} + \frac{1}{2}(|z′|^{2} − |z′′|^{2})\frac{∂}{∂z_{j}} ,  (1 ≤ j ≤ n).

\underline{2.8.} We recall M = S^{p−1} × S^{q−1}. This subsection relates the representation ϖ_{λ,M} and ϖ_{λ,ℝ^{p−1,q−1}} by the stereographic projection Ψ^{−1} : M → ℝ^{p−1,q−1} defined below.

We set a positive valued function τ: ℝ^{n} → ℝ by

τ(z) ≡ τ(z′, z′′) := ν ∘ ι(z)

= ((1 − \frac{|z′|^{2} − |z′′|^{2}}{4} )^{2} + |z′|^{2})^{\frac{1}{2}}

= ((1 + \frac{|z′|^{2} − |z′′|^{2}}{4} )^{2} + |z′′|^{2})^{\frac{1}{2}}

= (1 + (\frac{|z′| + |z′′|}{2})^{2})^{\frac{1}{2}} (1 + (\frac{|z′| − |z′′|}{2})^{2})^{\frac{1}{2}} . (2.8.1)

We define an injective diffeomorphism as a composition of ι: ℝ^{p−1,q−1} ↪ Ξ (see (2.3.4)) and Φ: Ξ → M (see (2.3.5)):

(2.8.2)  Ψ : ℝ^{p−1,q−1} → M, z ↦ τ(z)^{−1}ι(z).

The image of Ψ is

(2.8.3) M_{+} := {u = (u_{0}, u′, u′′, u_{n+1}) ∈ M = S^{p−1} × S^{q−1} : u_{0} + u_{n+1} > 0}.

Then, Ψ is a conformal map (see [11], Lemma 3.3, for example) such that

(2.8.4)  Ψ^{∗}g_{M} = τ(z)^{−2}g_{ℝ^{p−1,q−1}}.

The inverse of Ψ: ℝ^{p−1,q−1} → M_{+} is given by

(2.8.5)  Ψ^{−1}(u_{0}, u′, u′′, u_{n+1}) = (\frac{u_{0} + u_{n+1}}{2} )^{−1}(u′, u′′) = μ(u)^{−1}(u′, u′′)

Ψ^{−1} is nothing but a stereographic projection if q = 1. We note that Ψ induces a conformal compactification of the flat space ℝ^{p−1,q−1}:

ℝ^{p−1,q−1} ↪ (S^{p−1} × S^{q−1})/ ∼ ℤ_{2}.

Here ∼ ℤ_{2} denotes the equivalence relation in the direct product space S^{p−1} ×S^{q−1} defined by u ∼ −u.

As in (2.2.1), we define the twisted pull-back by

(2.8.6)  Ψ_{λ}^{∗} : C^{∞}(M) → C^{∞}(ℝ^{n}), F ↦ τ(z)^{−λ}F (Ψ(z)).

Let

C^{∞}(M)_{ϵ} := {f ∈ C^{∞}(M) : f(−u) = ϵf(u), for any u ∈ M}.

Then Ψ_{λ}^{∗}|_{C^{∞}(M)_{ϵ}} is injective. The inverse map is given by

(2.8.7) (Ψ_{λ,ϵ}^{∗})^{−1} : Ψ_{λ}^{∗}(C^{∞}(M)_{ϵ}) → C^{∞}(M)_{ϵ},

f ↦ \{ \begin{matrix}|\frac{u_{0}+u_{n+1}}{2} |^{−λ}f(Ψ^{−1}(u)) & (u ∈ M_{+}) \\ ϵ|\frac{u_{0}+u_{n+1}}{2} |^{−λ}f(Ψ^{−1}(−u)) & (u ∈ M_{−}). \end{matrix}

We note that (Ψ_{λ,ϵ}^{∗})^{−1}f makes sense for f ∈ C_{0}^{∞}(ℝ^{n}), since we have

C_{0}^{∞}(ℝ^{n}) ⊂ Ψ_{λ}^{∗}(C^{∞}(M)_{ϵ}).

Now, the representation ϖ_{λ,ϵ,ℝ^{n}} is well-defined on the following representation space: Ψ_{λ}^{∗}(C^{∞}(M)), a subspace of C^{∞}(ℝ^{n}) through ϖ_{λ,M}.

Then, by (2.2.2) (see [11], Proposition 2.6), we have:

Lemma 2.8. Ψ∗_{\frac{n−2}{2}} (V^{p,q}) ⊂ Ker □_{ℝ^{p−1,q−1}}, where V^{p,q} = Ker Δ̃_{M}.

\underline{2.9.} In the terminology of representation theory of semisimple Lie groups, Ψ_{λ}^{∗} is a G-intertwining operator from the K-picture (ϖ_{λ,M}, C^{∞}(M)_{ϵ}) to the N-picture (ϖ_{λ,ϵ,ℝ^{n}}, Ψ_{λ}^{∗}(C^{∞}(M))). To see this in an elementary way, we argue as follows: For ν ∈ ℂ, we denote by the space

(2.9.1)  S^{ν}(Ξ) := {h ∈ C^{∞}(Ξ) : h(tξ) = t^{ν}h(ξ), for any ξ ∈ Ξ, t > 0},

of smooth functions on Ξ of homogeneous degree ν. Then G acts on S^{ν}(Ξ) by left translations. Furthermore, for ϵ = ±1, we put

(2.9.2)  S^{ν,ϵ}(Ξ) := {h ∈ S^{ν}(Ξ) : h(−ξ) = ϵh(ξ), for any ξ ∈ Ξ}.

Then we have a direct sum decomposition

S^{ν}(Ξ) = S^{ν,1}(Ξ) + S^{ν,−1}(Ξ),

on which G acts by left translations, respectively. Then S^{ν,ϵ}(Ξ) corresponds to the degenerate principal series representation (see [11] for notation):

(2.9.3)  C^{∞}- Ind_{P^{max}}^{G}(ϵ ⊗ ℂ_{λ}) ≃ S^{−λ− \frac{n}{2} ,ϵ}(Ξ),

where P^{max} = M^{max}A^{max}N^{max}.

Lemma 2.9. 1) The restriction S^{−λ,ϵ}(Ξ) → C^{∞}(M)_{ϵ}, h ↦ h|_{M} induces the isomorphism of G-modules between S^{−λ,ϵ}(Ξ) and (ϖ_{λ}, C^{∞}(M)_{ϵ}) for any λ ∈ ℂ.

2) The restriction S^{−λ,ϵ}(Ξ) → C^{∞}(ℝ^{n}), h ↦ h|_{ℝ^{n}} induces the isomorphism of Gmodules between S^{−λ,ϵ}(Ξ) and (ϖ_{λ,ϵ,ℝ^{n}}, Ψ_{λ}^{∗}(C^{∞}(M)_{ϵ})) for any λ ∈ ℂ.

Proof. See [11], Lemma 3.7.1 for (1). (2) follows from the commutative diagram:

S^{−λ,ϵ}(Ξ)
(2.9.4)  r_{1} ↙  ↘ r_{2}
C^{∞}(M)_{ϵ}  ⟶^{Ψ_{λ}^{∗}}  C^{∞}(ℝ^{n}),

because r_{1} is bijective and r_{2} is injective. □

\underline{2.10.} A natural bilinear form ⟨ , ⟩ : S^{−λ− \frac{n}{2}} (Ξ) × S^{λ− \frac{n}{2}} (Ξ) → ℂ is defined by

(2.10.1)  ⟨h_{1}, h_{2}⟩ := ∫_{M} h_{1}(b)h_{2}(b) db

= 2 ∫_{ℝ^{n}} h_{1}(ι(z))h_{2}(ι(z)) dz (see (2.3.4)). (2.10.2)

Here, db is the Riemannian measure on M = S^{p−1} × S^{q−1}. The second equation follows from (h_{1}h_{2})(ι(z)) = τ(z)^{−n}(h_{1}h_{2})(Ψ(z)) and the Jacobian for Ψ: ℝ^{n} → M is given by τ(z)^{−n} (see (2.8.4)). Then ⟨ , ⟩ is K-invariant and \overline{N^{max}}-invariant from (2.10.1) and (2.10.2), and thus G-invariant since G is generated by K and \overline{N^{max}}.

§3. Square integrable functions on the cone

\underline{3.1.} In this section we shall study the irreducible unitary representation of the motion group M_{+}^{max}\overline{N^{max}} ≃ O(p − 1, q − 1) ⋉ ℝ^{p+q−2} and the maximal parabolic subgroup \overline{P^{max}} = M^{max}A^{max}\overline{N^{max}} on the space of solutions to our ultrahyperbolic equation □_{ℝ^{p−1,q−1}}f = 0. This is a standard induced representation by the Mackey machine, and will be later extended to the minimal representation of G = O(p, q) (see Theorem 4.9 (3)).

\underline{3.2.} In the flat picture ℝ^{p−1.q−1}, our minimal representation V^{p,q} of O(p, q) can be realized in a subspace of Ker □_{ℝ^{p−1,q−1}} (see Lemma 2.8). We shall study the representation space by means of the Fourier transform.

We normalize the Fourier transform on S(ℝ^{n}) by

(Ff)(ζ) = ∫_{ℝ^{n}} f(z)e^{\sqrt{−1}(z_{1}ζ_{1}+···+z_{n}ζ_{n})}dz_{1} ⋯ dz_{n},

and extends it to S′(ℝ^{n}), the space of the Schwartz distributions.

By composing the following two injective maps

C^{∞}(M)_{ϵ} ⟶^{Ψ∗ ; \frac{n−2}{2}} C^{∞}(ℝ^{n}) ∩ S′(ℝ^{n}) ⟶^{F} S′(ℝ^{n}),

we define a representation of G and 𝔤 on the image FΨ∗_{\frac{n−2}{2}} (C^{∞}(M)), denoted by ϖ̂_{λ,ϵ} ≡ ϖ_{λ,ϵ,ℝ^{n}}, so that F ∘ Ψ∗_{\frac{n−2}{2}} is a bijective G-intertwining operator from the representation space (ϖ_{λ,M}, C^{∞}(M)_{ϵ}) to (ϖ̂_{λ,ϵ}, FΨ∗_{\frac{n−2}{2}} (C^{∞}(M)_{ϵ})). Then, it follows from (2.7.2) that the representation ϖ̂_{λ,ϵ} has a simple form when restricted to the subgroup \overline{P^{max}} = M^{max}A^{max}\overline{N^{max}}:

(3.2.1)(a) (ϖ̂_{λ,ϵ}(m)h)(ζ) = h(^{t}mζ)  (m ∈ M_{+}^{max}),
(ϖ̂_{λ,ϵ}(m_{0})h)(ζ) = ϵh(ζ),
(3.2.1)(b) (ϖ̂_{λ,ϵ}(e^{tE})h)(ζ) = e^{(λ−n)t}h(e^{−t}ζ)  (t ∈ ℝ),
(3.2.1)(c) (ϖ̂_{λ,ϵ}(\overline{n}_{a})h)(ζ) = e^{2\sqrt{−1}(a_{1}ζ_{1}+···+a_{n}ζ_{n})}h(ζ)  (a ∈ ℝ^{n}).

We remark that in the above formula, we regarded h as a function. The action of A^{max} on the space of distributions is slightly different by the contribution of the measure dζ:

(3.2.1)(b′)  (ϖ̂_{λ,ϵ}(e^{tE})ϕ)(ζ) = e^{λt}ϕ(e^{−t}ζ)  (t ∈ ℝ),

if we write ϕ(ζ) = h(ζ)dζ ∈ S′(ℝ^{n}).

The differential representation dϖ̂_{λ,ϵ} of 𝔤_{0} on FΨ∗_{\frac{n−2}{2}} (C^{∞}(M)) is given by the following lemma:

Lemma 3.2. We recall that E_{ζ} is the Euler operator (see (2.7.4)). With notation in (2.5.2), we have

dϖ̂_{λ,ϵ}(\overline{N}_{j}) = 2\sqrt{−1}ζ_{j}  (1 ≤ j ≤ n),
dϖ̂_{λ,ϵ}(N_{j}) = \sqrt{−1} (λε_{j} \frac{∂}{∂ζ_{j}} − E_{ζ}ε_{j} \frac{∂}{∂ζ_{j}} + \frac{1}{2}ζ_{j}□_{ζ})  (1 ≤ j ≤ n),
dϖ̂_{λ,ϵ}(E) = λ − n − E_{ζ}.

Proof. Lemma follows from the correspondence under the Fourier transform \frac{∂}{∂z_{j}} ↔ −\sqrt{−1}ζ_{j}, z_{j} ↔ −\sqrt{−1}\frac{∂}{∂ζ_{j}}, and therefore from E_{z} ↔ −n − E_{ζ}, P (z) ↔ −□_{ζ}, where P (z) := z_{1}^{2} + ⋯ + z_{p−1}^{2} − z_{p}^{2} − ⋯ − z_{p+q−2}^{2}. □

Remark. 1) We note that dϖ̂_{λ,ϵ} is independent of the signature ϵ = ±1.

2) In Theorem 4.9, we shall find that L^{2}(C), the Hilbert space of square integrable functions on the cone C in ℝ^{n}, is a G-invariant subspace of the Schwartz distributions S′(ℝ^{n}). Then, the action of the Lie algebra 𝔤 can be written in terms of differential operators along the cone C at most of second order.

\underline{3.3.} We define a quadratic form Q on ℝ^{n} (≃ (ℝ^{n})^{∗}) as the dual of P (z) on ℝ^{n} by

(3.3.1) Q(ζ) := ζ_{1}^{2} + ⋯ + ζ_{p−1}^{2} − ζ_{p}^{2} − ⋯ − ζ_{p+q−2}^{2}  for ζ ∈ ℝ^{n} = ℝ^{p+q−2}

and define a closed cone by

(3.3.2)  C := {ζ ∈ ℝ^{n} : Q(ζ) = 0}.

It follows from Lemma 2.8 that the support of the distribution FΨ∗_{\frac{n−2}{2}} F is contained in the cone C, for any F ∈ V^{p,q}. Surprisingly, FΨ∗_{\frac{n−2}{2}} F becomes square integrable on C (see Theorem 4.9). As a preparation for the proof, we study a natural action on L^{2}(C) of a parabolic subgroup \overline{P^{max}} in this subsection.

We take a differential (n − 1)-form dμ on C such that

dQ ∧ dμ = dζ_{1} ∧ ⋯ ∧ dζ_{n}.

Then the restriction dμ to the cone C defines a canonical measure (we use the same notation dμ). Using polar coordinates on C: ζ = (sω, sω′) with s > 0, ω ∈ S^{p−2}, ω′ ∈ S^{q−2}, we write down the canonical measure dμ on C explicitly by

(3.3.3)  ∫_{C} ϕdμ = \frac{1}{2} ∫_{0}^{∞} ∫_{S^{p−2}} ∫_{S^{q−2}} ϕ(sω, sω′)s^{n−3} ds dω dω′

for a test function ϕ. If n > 2, that is if p + q > 4, then the measure dμ defines a Schwartz distribution on ℝ^{n}, which is equal to the Delta function δ(Q) supported on the cone C (see [7]).

For a measurable function ϕ on C, we define a norm of ϕ by

(3.3.4)  ∥ϕ∥^{2} := ∫_{C} |ϕ|^{2}dμ

and denote by L^{2}(C) ≡ L^{2}(C, dμ) the Hilbert space of square integrable functions.

By the Mackey theory, we can define a natural representation π of the maximal parabolic subgroup \overline{P^{max}} = A^{max}M^{max}\overline{N^{max}} on L^{2}(C) if p + q ∈ 2ℤ, by

(3.3.5)(a) (π(e^{tE})ψ)(ζ) := e^{− \frac{n−2}{2} t}ψ(e^{−t}ζ),  (t ∈ ℝ)
(3.3.5)(b) (π(m)ψ)(ζ) := ψ(^{t}mζ)  (m ∈ M_{+}^{max}),
(π(m_{0})ψ)(ζ) := (−1)^{\frac{p−q}{2}} ψ(ζ)
(3.3.5)(c) (π(\overline{n}_{a})ψ)(ζ) := e^{2\sqrt{−1}(a_{1}ζ_{1}+···+a_{n}ζ_{n})}ψ(ζ)  (a ∈ ℝ^{n}).

Proposition 3.3. 1) The representation (π, L^{2}(C)) of \overline{P^{max}} is unitary.

2) The representation π is still irreducible when restricted to the motion group M_{+}^{max}\overline{N^{max}} ≃ O(p − 1, q − 1) ⋉ ℝ^{p+q−2}. In particular, it is irreducible as a \overline{P^{max}}- module.

Proof. (1) is straightforward from the definitions (3.3.4) and (3.3.5). Let us prove (2). It follows from (3.3.5)(c) that any \overline{N^{max}}-invariant closed subspace of L^{2}(C) is of the form L^{2}(C′) where C′ is a measurable subset of C. As M_{+}^{max} acts transitively on C, L^{2}(C′) is M_{+}^{max}-invariant only if the measure of C′ is either null or conull. Thus, L^{2}(C′) equals either {0} or L^{2}(C). Therefore, the unitary representation L^{2}(C) is irreducible as an M_{+}^{max}\overline{N^{max}}-module. □

\underline{3.4.} It is not clear a priori if (π, L^{2}(C)) extends from \overline{P^{max}} to G. We shall prove in Theorem 4.9 that if p, q ≥ 2 and n(= p + q − 2) > 2 then π extends to G as an irreducible unitary representation through an injective map T: L^{2}(C) ↪ S′(ℝ^{n}), defined as follows:

By using the Cauchy-Schwarz inequality, we see the following map

T (ψ): S(ℝ^{n}) → ℂ, φ ↦ ∫_{C} φψ dμ

is well-defined and continuous if n > 2, for each ψ ∈ L^{2}(C). Thus we have a natural map

(3.4.1)  T: L^{2}(C) ⟶ S′(ℝ^{n}), ψ ↦ ψdμ.

Clearly, T is injective. We shall regard T (L^{2}(C)) as a Hilbert space such that T is a unitary operator.

Lemma 3.4. 1) T is a \overline{P^{max}}-intertwining operator from (π, L^{2}(C)) to (ϖ_{\frac{n−2}{2} ,ϵ}|\overline{P^{max}}, S′(ℝ^{n})).

2) (ϖ_{\frac{n−2}{2} ,ϵ}|\overline{P^{max}}, T (L^{2}(C))) is an irreducible unitary representation of \overline{P^{max}}. It is still irreducible as an M^{max}\overline{N^{max}}-module.

Proof. (1) follows directly from the definitions (3.2.1) and (3.3.5). (2) follows from (1) and Proposition 3.3 (2). □

§4. Green function and inner product

\underline{4.1.} In this section, we shall construct solutions of the ultrahyperbolic equation □_{ℝ^{p−1,q−1}}f = 0 by the integral transform given by convolution with the Green kernel. Then, we shall show that the Green kernel coincides with a special value of the Knapp-Stein intertwining operator for a degenerate principal series. This observations gives another expression of the inner product of the minimal representation of O(p, q) by using the Green kernel (see Theorem 4.7), and also leads to a realization of the minimal representation on L^{2}(C), the Hilbert space of square integrable functions on a cone C as will be discussed in §6.

We put

P (x) = x_{1}^{2} + ⋯ + x_{p−1}^{2} − x_{p}^{2} − ⋯ − x_{n}^{2}

for x ∈ ℝ^{n} = ℝ^{p+q−2}.

\underline{4.2.} A distribution E satisfying □_{ℝ^{p−1,q−1}}E = δ (Dirac’s delta function) is called a fundamental solution of the ultra-hyperbolic Laplace operator □_{ℝ^{p−1,q−1}}. Recall from [7], page 354, if n is even and n > 2 then

E = \frac{−Γ(\frac{n}{2} − 1)e^{\frac{\sqrt{−1}π(q−1)}{2}}}{4π^{\frac{n}{2}}} (P (x) + \sqrt{−1}0)^{1− \frac{n}{2}}

is a fundamental solution of □_{ℝ^{p−1,q−1}}, where (P (x) + \sqrt{−1}0)^{λ} stands for the limit of the distribution (P (x) + \sqrt{−1}R(x))^{λ} as a positive definite quadratic form R(x) tends to 0. In view of the integral formula in [7], Chapter III, §2.6

F(P + \sqrt{−1}0)^{1− \frac{n}{2}} = \frac{4π^{\frac{n}{2}} e^{−^{\frac{\sqrt{−1}π(q−1)}{2}}}}{Γ(\frac{n}{2} − 1)} (Q − \sqrt{−1}0)^{−1},

we have readily the following formula for the Green function E_{0} of □_{ℝ^{p−1,q−1}}:

Proposition 4.2. We define a distribution E_{0} on ℝ^{n} by

(4.2.1)  E_{0} = \frac{1}{2π\sqrt{−1}}(−E + \overline{E}).

Then □_{ℝ^{p−1,q−1}}E_{0} = 0 and its Fourier transform is given by

(4.2.2)  FE_{0} = \frac{1}{2π\sqrt{−1}}((Q − \sqrt{−1}0)^{−1} − (Q + \sqrt{−1}0)^{−1}) = δ(Q).

In the Minkowski case, i.e. q = 2, such a formula has been known ([1]), since the so-called two-point functions in the quantum field theory for a zero mass field exactly corresponds to (Q + \sqrt{−1}0)^{−1} for negative frequency. In this case C naturally splits in two components, a forward and a backward light cone, and functions supported on the forward cone have Fourier transforms that extend to holomorphic functions on the corresponding tube domain, thus yielding a unitary highest weight representation of the connected group. The reproducing kernel of this representation is the Fourier transform of the measure on the forward cone, in analogy with what happens in Proposition 4.2.

\underline{4.3.} In order to give the integral expression of solutions □_{ℝ^{p−1,q−1}}f = 0, we define a convolution map by the Green kernel:

(4.3.1)  S: C_{0}^{∞}(ℝ^{n}) → C^{∞}(ℝ^{n}), φ ↦ E_{0} ∗ φ.

Fix ν ∈ ℂ. We consider the representation ϖ_{ν,ϵ} ≡ ϖ_{ν,ϵ,ℝ^{n}} of G on a subspace of C^{∞}(ℝ^{n}) (see §2). The restriction to \overline{P^{max}} stabilizes C_{0}^{∞}(ℝ^{n}), as follows from (2.7.2).

Lemma 4.3. 1) Image S ⊂ Ker □_{ℝ^{p−1,q−1}}.

2) S is an intertwining operator of \overline{P^{max}}-modules between on one side ϖ_{ν+2,ϵ}|\overline{P^{max}} and on the other side ϖ_{ν,ϵ}|\overline{P^{max}} for any ν ∈ ℂ and ϵ = ±1.

Proof. (1) As □_{ℝ^{p−1,q−1}}E_{0} = 0, we have Image S ⊂ Ker □_{ℝ^{p−1,q−1}}. The proof of (2) is direct from (2.7.2). We illustrate it by the action of e^{tE} ∈ A^{max}:

S(ϖ_{ν+2,ϵ}(e^{tE})φ)(y) = ∫_{ℝ^{n}} E_{0}(y − z)e^{(ν+2)t}φ(e^{t}z) dz
= ∫_{ℝ^{n}} E_{0}(e^{t}y − e^{t}z)e^{(ν+2)t+(n−2)t}φ(e^{t}z) dz
= e^{νt}(Sφ)(e^{t}y)
= ϖ_{ν,ϵ}(e^{tE})(Sφ)(y).

This shows that S intertwines the action of A^{max}. The case for the action of M^{max}\overline{N^{max}} is similar and easier. □

We shall see that S extends to a G-intertwining operator for ν = \frac{n}{2} − 1 in Proposition 4.6.

\underline{4.4.} Recall the notation in §2.9. Let

[u, v] := u_{0}v_{0} + ⋯ + u_{p−1}v_{p−1} − u_{p}v_{p} − ⋯ − u_{n+1}v_{n+1}

for u, v ∈ ℝ^{n+2} = ℝ^{p+q}. The Knapp-Stein intertwining operator

A_{λ,ϵ}: S^{−λ− \frac{n}{2} ,ϵ}(Ξ) → S^{λ− \frac{n}{2} ,ϵ}(Ξ)

is given by the integral operator with kernel function

(4.4.1)  ψ_{λ− \frac{n}{2} ,ϵ}(u_{0} − u_{n+1}) = ψ_{λ− \frac{n}{2} ,ϵ}([u, ξ_{0}])

on Ξ. Here, we put ξ_{0} := ^{t}(1, 0, …, 0, 1) ∈ Ξ and ψ_{ν,ϵ} is a distribution (or a hyperfunction) of one variable is defined by

ψ_{ν,ϵ}(y) := \frac{1}{Γ(\frac{2ν+3−ϵ}{4})}|y|^{ν}χ_{ϵ}(sgn y).

Via the bijection r_{1} in the commutative diagram (2.9.4),

A_{λ,ϵ} : C^{∞}(M)_{ϵ} → C^{∞}(M)_{ϵ}

is written as

(A_{λ,ϵ}^{M}f)(u) ≡ (A_{λ,ϵ}f)(u) := ∫_{M} ψ_{λ− \frac{n}{2} ,ϵ}([u, v]) f(v) dv (u ∈ M)

in the compact picture M ≃ S^{p−1} × S^{q−1}.

The Gamma factor in the definition of ψ_{ν,ϵ}(y) exactly cancels the poles of the distribution |y|^{ν}χ_{ϵ}(sgn y) of one variable y with meromorphic parameter ν. This means that the distribution ψ_{ν,ϵ}([u, v]) of multi-variables makes sense for any ν ∈ ℂ when restricted to the open set {(u, v) ∈ M × M : u ≠ ±v}, where [u, dv]+[v, du] ≠ 0. Then ψ_{ν,ϵ}([u, v]) continues meromorphically as a distribution on M × M with possible poles only at λ = 0, −1, …, whose residues are distributions supported on {(u, v) ∈ M × M : u = ±v}.

In view of the normalization of our parameter (see Lemma 2.9), A_{λ,ϵ} is a Gintertwining operator from ϖ_{λ+ \frac{n}{2}} to ϖ_{−λ+ \frac{n}{2}} for λ ≠ 0, −1, −2, . . .. What we need is the case λ = 1 and we recall from [11], §3.9 (basically since we know the composition series of the induced representations and the eigenvalue of A_{λ,ϵ} on each K-type):

Lemma 4.4. Let p ≡ q mod 2, p, q ≥ 2 and (p, q) ≠ (2, 2). We put

(4.4.2)(a)  ϵ := (−1)^{\frac{p−q}{2}} = \{ \begin{matrix}1 & p − q ≡ 0, 4 mod 8, \\ −1 & p − q ≡ 2, 6 mod 8. \end{matrix}
(4.4.2)(b)  δ := (−1)^{[ \frac{q−p}{4} ]} = \{ \begin{matrix}1 & p − q ≡ 0, 2 mod 8, \\ −1 & p − q ≡ 4, 6 mod 8. \end{matrix}

1) The image (ϖ_{\frac{n}{2} −1}, A_{1,ϵ}(C^{∞}(M)_{ϵ})) is a dense subrepresentation of (ϖ^{p,q}, V^{p,q}), where we recall V^{p,q} = Ker Δ̃_{M}.

Let us define a Hermitian form ( , )_{A} on the same image A_{1,ϵ}(C^{∞}(M)_{ϵ}) by

(4.4.3)  (F_{1}, F_{2})_{A} := δ⟨A_{1,ϵ}φ_{1}, \overline{φ_{2}}⟩ = δ⟨φ_{1}, \overline{A_{1,ϵ}φ_{2}}⟩,

for F_{i} = A_{1,ϵ}φ_{i}, φ_{i} ∈ C^{∞}(M)_{ϵ} (i = 1, 2), where ⟨ , ⟩ is the bilinear form given as the integral over M (see (2.10.1)).

2) The Hermitian form ( , )_{A} is well-defined (namely, independent of the choice of φ_{i}) and G-invariant under the action of ϖ_{\frac{n}{2} −1}.

3) In comparison with the inner product ( , )_{M} given in Lemma 2.6, we have

(4.4.4)  (F_{1}, F_{2})_{A} = c_{1}(F_{1}, f_{2})_{M}  for any F_{1}, F_{2} ∈ A_{1,ϵ}(C^{∞}(M)_{ϵ}),

where we put

c_{1} := \frac{Γ(\frac{n−1−ϵ}{4})}{2^{\frac{n}{2}} π^{\frac{n+1}{2}}}.

In particular, the Hermitian form ( , )_{A} is positive definite and the completion of a pre-Hilbert space (A_{1,ϵ}(C^{∞}(M)_{ϵ}), ( , )_{A}) coincides with the Hilbert space \overline{V^{p,q}} given in §2.6.

\underline{4.5.} In the flat picture ℝ^{p−1,q−1}, we have

−ξ_{0} + ξ_{n+1} = |z′|^{2} − |z′′|^{2} = P (z)

for ξ = ι(z) by (2.3.4). Then, via the injection r_{2} in the commutative diagram (2.9.4), the Knapp-Stein intertwining operator is given by the convolution:

(4.5.1)  A_{λ,ϵ}^{ℝ^{n}}φ ≡ A_{λ,ϵ}φ := 2ψ_{λ− \frac{n}{2} ,ϵ}(P (z)) ∗ φ

when restricted to C_{0}^{∞}(ℝ^{n}) ⊂ Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ}). Then we have

(4.5.2)  A_{1,ϵ}^{ℝ^{n}} ∘ Ψ∗_{\frac{n+2}{2}} = Ψ∗_{\frac{n−2}{2}} ∘ A_{1,ϵ}^{M}.

Lemma 4.5. Retain the setting of Lemma 4.4. We put

(4.5.3)  h(y) := e^{\frac{\sqrt{−1}π(q−1)}{2}} (y + \sqrt{−1}0)^{1− \frac{n}{2}} .

Then we have

(4.5.4)  ψ_{1− \frac{n}{2} ,ϵ}(y) = δ \frac{Γ(\frac{−1+n+ϵ}{4})}{2π\sqrt{−1}}(h(y) − \overline{h(y)}).

Proof. Solving (y ± \sqrt{−1}0)^{ν} = y_{+}^{ν} + e^{±\sqrt{−1}πν}y_{−}^{ν}, we have

ψ_{ν,ϵ}(y) = \frac{(e^{−\sqrt{−1}νπ} − ϵ)(y + \sqrt{−1}0)^{ν} − (e^{\sqrt{−1}νπ} − ϵ)(y − \sqrt{−1}0)^{ν}}{−2\sqrt{−1} sin πνΓ(\frac{2ν+3−ϵ}{4})} .

Then, Lemma 4.5 follows from a residue calculation:

lim_{ν→1− \frac{n}{2}} \frac{e^{−\sqrt{−1}νπ} − ϵ}{sin πν Γ(\frac{2ν+3−ϵ}{4})} = lim_{a→0} \frac{e^{−\sqrt{−1}aπ} − (−1)^{q}}{sin πa Γ(\frac{5−n−ϵ}{4} + \frac{a}{2})} = \frac{−δe^{\frac{\sqrt{−1}π(q−1)}{2}} Γ(\frac{−1+n+ϵ}{4})}{π}.

\underline{4.6.} Our key observation is that the special value of the Knapp-Stein operator is given by the Green function up to a scalar constant. In particular, the image of C_{0}^{∞}(ℝ^{n}) under the integral transform S satisfies not only the ultra-hyperbolic equation □_{ℝ^{p−1,q−1}}f = 0 but also a certain decay condition at infinity that matches a conformal compactification of ℝ^{p−1,q−1} (see §2.8).

Proposition 4.6. Retain the notation as in Proposition 4.2 and Lemma 4.4. We recall that δ, ϵ = ±1 are determined by p − q mod 8 as in (4.4.2) and define the constant c_{2} by

c_{2} := \frac{4δπ^{\frac{n}{2}} Γ(\frac{−1+n+ϵ}{4})}{Γ(\frac{n}{2} − 1)}.

Then we have:

(4.6.1)  ψ_{1− \frac{n}{2} ,ϵ}(P (z)) = c_{2}E_{0},
(4.6.2)  A_{1,ϵ}^{ℝ^{n}}|_{C_{0}^{∞}(ℝ^{n})} = 2c_{2}S.

In particular, S(C_{0}^{∞}(ℝ^{n})) ⊂ Ψ∗_{\frac{n−2}{2}} (V^{p,q}).

Proof. The first formula follows from Lemma 4.5 and from the definition of E_{0} (see (4.2.1)). The second formula then follows from the definition (4.5.1). Then, in view of (4.5.2) and (4.6.2), S extends to a G-intertwining operator between

(ϖ_{\frac{n+2}{2}} , Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) and (ϖ_{\frac{n−2}{2}} , Ψ∗_{\frac{n−2}{2}} (C^{∞}(M)_{ϵ})). Then we have

S(C_{0}^{∞}(ℝ^{n})) ⊂ A_{1,ϵ}^{ℝ^{n}}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) = Ψ∗_{\frac{n−2}{2}} (A_{1,ϵ}^{M}(C^{∞}(M)_{ϵ})) ⊂ Ψ∗_{\frac{n−2}{2}} (V^{p,q}).

because C_{0}^{∞}(ℝ^{n}) ⊂ Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ}). □

\underline{4.7.} We are ready to introduce an inner product on Ker □_{ℝ^{p−1,q−1}} by using the Green function E_{0} (see (4.2.1)), so that our minimal representation is realized here as a unitary representation.

We define a Hermitian form on S(C_{0}^{∞}(ℝ^{n})) ⊂ Ker □_{ℝ^{p−1,q−1}} by

(4.7.1)  (f_{1}, f_{2})_{N} := ∫_{ℝ^{n}} ∫_{ℝ^{n}} E_{0}(y − x)φ_{1}(x)\overline{φ_{2}(y)} dx dy

for f_{i} = Sφ_{i} = E_{0} ∗ φ_{i} (1 ≤ i ≤ 2). The right-hand side of (4.7.1) does not depend on the choice of φ_{i} because of the formula

(f_{1}, f_{2})_{N} = (f_{1}, φ_{2})_{L^{2}(ℝ^{n})} = (φ_{1}, f_{2})_{L^{2}(ℝ^{n})}.

We recall from Proposition 4.6 and Lemma 2.8 the inclusive relations: (4.7.2)

S(C_{0}^{∞}(ℝ^{n})) ⊂ A_{1,ϵ}^{ℝ^{n}}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) ⊂ Ψ∗_{\frac{n−2}{2}} (V^{p,q}) ⊂ Ker □_{ℝ^{p−1,q−1}} ⊂ C^{∞}(ℝ^{n}).

Theorem 4.7. Let p ≡ q mod 2, p, q ≥ 2 and (p, q) ≠ (2, 2). Recall n = p + q − 2 and ϵ = (−1)^{\frac{p−q}{2}} .

1) The Hermitian form ( , )_{N} is positive definite on S(C_{0}^{∞}(ℝ^{n})).

2) The Hermitian form ( , )_{N} is invariant under ω_{\frac{n−2}{2} ,ϵ,ℝ^{n}}(G).

Let H be the completion of the pre-Hilbert space (S(C_{0}^{∞}(ℝ^{n})), ( , )_{N}).

3) The Fréchet representation ω_{\frac{n−2}{2} ,ϵ,ℝ^{n}} of G on Ψ∗_{\frac{n−2}{2}} (C^{∞}(M)_{ϵ}) induces a repre-

sentation of G on the Hilbert space H, which we shall denote by (ϖ_{ℝ^{p−1,q−1}}^{min}, H).

4) The unitary representation (ϖ_{ℝ^{p−1,q−1}}^{min}, H) is irreducible.

5) The map (see (2.8.7) for the definition)

(Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1} :S(C_{0}^{∞}(ℝ^{n}))→V^{p,q}
∩  ∩
C^{∞}(ℝ^{n})  C^{∞}(M)

extends uniquely to a unitary isomorphism between (ϖ_{ℝ^{p−1,q−1}}^{min}, H) and the minimal representation (ϖ^{p,q}, \overline{V^{p,q}}) up to a scalar constant. More precisely, for any f_{1}, f_{2} ∈ H, we have

(4.7.3)  (f_{1}, f_{2})_{N} = 2^{2−n}(F_{1}, F_{2})_{M},

where we put

F_{i} := (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1}f_{i}, (i = 1, 2).

Remark. 1) We shall give a different proof of the first statement of Theorem 4.7 in Theorem 4.9 by using the Fourier transform of the Green kernel.

2) As we shall see in the proof of Theorem 4.7, S(C_{0}^{∞}(ℝ^{n})) is dense in Ψ∗_{\frac{n−2}{2}} (V^{p,q}) with respect to the above inner product.

3) We can realize naturally the Hilbert space H as a subspace of the Schwartz distributions S′(ℝ^{n}), namely, F^{−1}T (L^{2}(C)), as we shall prove in Theorem 4.9.

\underline{4.8.} In order to prove Theorem 4.7, we need:

Lemma 4.8. Retain the notation of Theorem 4.7.

1) For any f_{1}, f_{2} ∈ S(C_{0}^{∞}(ℝ^{n})), we have

(4.8.1)  (f_{1}, f_{2})_{N} = c_{3}(F_{1}, F_{2})_{A},

where

c_{3} := \frac{π^{\frac{n}{2}} Γ(\frac{−1+n+ϵ}{4})}{Γ(\frac{n}{2} − 1)}.

We extend the Hermitian form ( , )_{N} from S(C_{0}^{∞}(ℝ^{n})) to A_{1,ϵ}^{ℝ^{n}}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) by using the right side of (4.8.1) (recall the inclusive relation (4.7.2)).

2) The Hermitian form ( , )_{N} is positive definite on A_{1,ϵ}^{ℝ^{n}}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})), in which S(C_{0}^{∞}(ℝ^{n})) is dense. In particular, the Hilbert space H (see Theorem 4.7) coincides with the completion of the pre-Hilbert space A_{1,ϵ}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})).

We first finish the proof of Theorem 4.7, and then give a proof of Lemma 4.8.

Proof of Theorem 4.7. (1) is clear from Lemma 4.8 (1) and Lemma 4.4. Next, let us prove (4.7.3). We consider the G-intertwining operator

(Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1} : Ψ∗_{\frac{n−2}{2}} (A_{1,ϵ}^{M}(C^{∞}(M)_{ϵ})) → A_{1,ϵ}^{M}(C^{∞}(M)_{ϵ}) ⊂ V^{p,q},

or equivalently by (4.5.2),

(Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1} : A_{1,ϵ}^{ℝ^{n}} (Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) → V^{p,q}.

Combining (4.4.4) and (4.8.1), we have

(f_{1}, f_{2})_{N} = c_{3}(F_{1}, F_{2})_{A} = c_{1}c_{3}(F_{1}, F_{2})_{M} = 2^{2−n}(F_{1}, F_{2})_{M},

where the second equality follows from a classical formula of the Gamma function:

2^{2z−1}Γ(z) Γ(z + \frac{1}{2}) = \sqrt{π} Γ(2z).

Thus, we have proved (4.7.3), especially, (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1} is an isometry up to scalar.

Then (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1} extends to an isometric (up to scalar) G-intertwining operator

H → \overline{V^{p,q}}. This map is surjective because (ϖ^{p,q}, \overline{V^{p,q}}) is an irreducible unitary representation of G (see Lemma 2.6). All other statements are now clear. □

Proof of Lemma 4.8. 1) Suppose f_{i} = Sφ_{i}, φ_{i} ∈ C_{0}^{∞}(ℝ^{n}) (i = 1, 2). Recall the constant c_{2} in Proposition 4.6, we have

(f_{1}, f_{2})_{N} = ∫_{ℝ^{n}}(Sφ_{1})(x)\overline{φ_{2}(x)}dx

= \frac{1}{2c_{2}} ∫_{ℝ^{n}}(A_{1,ϵ}^{ℝ^{n}}φ_{1})(x) \overline{φ_{2}(x)} dx

= \frac{1}{4c_{2}} ∫_{M}(A_{1,ϵ}^{M}((Ψ_{\frac{n+2}{2} ,ϵ}^{∗})^{−1}φ_{1})(b) \overline{(Ψ_{\frac{n+2}{2} ,ϵ}^{∗})^{−1}φ_{2}(b)} db. (4.8.2)

We put

ψ_{i} := \frac{1}{c_{2}} (Ψ_{\frac{n+2}{2} ,ϵ}^{∗})^{−1}φ_{i} ∈ C^{∞}(M)_{ϵ}, (i = 1, 2).

Then F_{i} = A_{1,ϵ}^{M}ψ_{i} by (4.5.2). Therefore,

(4.8.2) = \frac{1}{4c_{2}} c_{2}^{2} ∫_{M}(A_{1,ϵ}^{M}ψ_{1})(b) \overline{ψ_{2}(b)} db = \frac{δc_{2}}{4} (F_{1}, F_{2})_{A}

which equals the right side of (4.8.1).

2) It is enough to show that S(C_{0}^{∞}(ℝ^{n})) is dense in a pre-Hilbert space A_{1,ϵ}(Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) because the inner product ( , )_{A} is positive definite from Lemma 4.4.

Suppose f = A_{1,ϵ}φ (φ ∈ Ψ∗_{\frac{n+2}{2}} (C^{∞}(M)_{ϵ})) is orthogonal to S(C_{0}^{∞}(ℝ^{n})) with respect to the inner product ( , )_{A}. This means that (f, A_{1,ϵ}ϕ)_{A} = 0 for any ϕ ∈ C_{0}^{∞}(ℝ^{n}). Then we have

∫_{M} (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1}f(b) \overline{(Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1}ϕ(b)} db = 0 for any ϕ ∈ C_{0}^{∞}(ℝ^{n}).

Since (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1}(C_{0}^{∞}(ℝ^{n})) is dense in L^{2}(M)_{ϵ} := \{f ∈ L^{2}(M) : f(−u) = ϵf(u)\},

we have (Ψ_{\frac{n−2}{2} ,ϵ}^{∗})^{−1}f = 0, and thus f = 0. □

\underline{4.9.} We recall Ψ∗_{\frac{n−2}{2}} (V^{p,q}) ⊂ Ψ∗_{\frac{n−2}{2}} (C^{∞}(M)_{ϵ}) ⊂ S′(ℝ^{n}), on which we can define the Fourier transform F. We consider the following maps (see §4.3, §3.2, §3.4):

C_{0}^{∞}(ℝ^{n}) →^{S} Ψ∗_{\frac{n−2}{2}} (V^{p,q}) →^{F} S′(ℝ^{n}) ↩^{T} L^{2}(C)

Here is a description of the Fourier transform of the minimal unitary representation (ϖ_{ℝ^{p−1,q−1}}^{min}, H) of G = O(p, q) which is obtained as the completion of a pre-Hilbert space (S(C_{0}^{∞}(ℝ^{n})), ( , )_{N}) (see Theorem 4.7).

Theorem 4.9. 1) FS(C_{0}^{∞}(ℝ^{n})) is contained in T (L^{2}(C)). Furthermore, it is dense in the Hilbert space T (L^{2}(C)).

2) T^{−1} ∘ F: S(C_{0}^{∞}(ℝ^{n})) → L^{2}(C) extends uniquely to a unitary map H → L^{2}(C) up to a scalar constant. This constant is given explicitly by

(4.9.1)  (2π)^{n}(f, f)_{N} = ∥T^{−1} ∘ Ff∥_{L^{2}(C)}^{2}.

3) The \overline{P^{max}}-module (π, L^{2}(C)) extends to an irreducible unitary representation of G, denoted by the same letter π, so that (2π)^{− \frac{n}{2}} T^{−1} ∘ F gives a unitary equivalence between (ϖ_{ℝ^{p−1,q−1}}^{min}, H) and (π, L^{2}(C)).

4) T^{−1} ∘F ∘Ψ∗_{\frac{n−2}{2}} : V^{p,q} → L^{2}(C) induces a unitary equivalence between irreducible

unitary representations (ϖ^{p,q}, \overline{V^{p,q}}) and (π, L^{2}(C)), up to a scalar constant given by:

(4.9.2)  (ϕ, ϕ)_{M} = 4π^{n}∥T^{−1} ∘ F ∘ Ψ∗_{\frac{n−2}{2}} ϕ∥_{L^{2}(C)}^{2}

Proof. If f = E_{0} ∗ φ (φ ∈ C_{0}^{∞}(ℝ^{n})), then it follows from the integration formula of the Green function (see Proposition 4.2) that its Fourier transform is given by

Ff = F(E_{0} ∗ φ) = (FE_{0})(Fφ) = (Fφ)δ(Q) = T ((Fφ)|_{C}).

Since φ ∈ C_{0}^{∞}(ℝ^{n}), we have (Fφ)|_{C} ∈ L^{2}(C). Hence, FS(C_{0}^{∞}(ℝ^{n})) is contained in T (L^{2}(C)). Then we have (4.9.1), as follows from

(4.9.3) ∥(Fφ)|_{C}∥_{L^{2}(C)}^{2} = ((FE_{0})(Fφ), Fφ) = (F(E_{0} ∗ φ), Fφ) = (2π)^{n}(f, f)_{N}.

We note that (4.9.3) gives a different proof that ( , )_{N} is a positive definite Hermitian form on S(C_{0}^{∞}(ℝ^{n})) (see Remark after Theorem 4.7).

It follows from Lemma 3.4 that T^{−1} ∘ F is an \overline{P^{max}}-intertwining operator from (ϖ_{\frac{n−2}{2} ,ϵ,ℝ^{n}}|\overline{P^{max}}, S(C_{0}^{∞}(ℝ^{n}))) to (π, L^{2}(C)). This map is isometric up to a scalar by (4.9.1). Then, it extends naturally to an \overline{P^{max}}-intertwining operator from (ϖ_{ℝ^{p−1,q−1}}^{min}|\overline{P^{max}}, H) to (π, L^{2}(C)), which is surjective because (π, L^{2}(C)) is irreducible (see Proposition 3.3). Hence we have proved (1), (2) and (3). The statement (4) follows from (2) and Theorem 4.7. Thus, we have proved Theorem 4.9. □

\underline{4.10.} By semisimple theory, it is known that a minimal representation is still irreducible when restricted to any maximal parabolic subgroup. In particular, ϖ^{p,q} is irreducible as an \overline{P^{max}}-module. In our case, this fact can be strengthened as follows:

Corollary 4.10. The restriction of the minimal representation ϖ^{p,q} (equivalently, ϖ_{ℝ^{p−1,q−1}}^{min}) to M_{+}^{max}\overline{N^{max}} ≃ O(p − 1, q − 1) × ℝ^{p+q−2} is still irreducible.

Proof. Theorem 4.7 and Theorem 4.9 show that ϖ^{p,q}, ϖ_{ℝ^{p−1,q−1}}^{min} and π are unitary equivalent to one another. Now, Corollary follows from Proposition 3.3. □

\underline{4.11.} For the convenience of the reader, we summarize the maps used in the proofs.

(4.11.1)C^{∞}(M) ⟶^{Ψ∗ ; \frac{n−2}{2}} C^{∞}(ℝ^{n}) ∩ S′(ℝ^{n}) ⟶^{F} S′(ℝ^{n})
∪  ∪  ∪
V^{p,q}  →^{∼} Ψ∗_{\frac{n−2}{2}} (V^{p,q})  →^{∼} FΨ∗_{\frac{n−2}{2}} (V^{p,q})
∩ dense  ∩ dense  ∩ dense
(ϖ^{p,q}, \overline{V^{p,q}}) →^{∼} (ϖ_{ℝ^{p−1,q−1}}^{min}, H) →^{∼} T (L^{2}(C)) ←^{∼}_{T} (π, L^{2}(C)).

In the last line, we have written also the notation for unitary representations.

§5. Bessel function and an integral formula of spherical functions

\underline{5.1.} In this section, we shall compute explicitly the lowest K-type of our minimal representation in the N-picture, i.e. find it as a solution to □_{ℝ^{p−1,q−1}}f = 0, and also its Fourier transform as a function on C; this turns out to be written in terms of a Bessel function (see Theorem 5.5). Note that except when p = q we are not dealing with a spherical representation of G (namely, there is no non-zero K-fixed vector in our representation). At the end of this section, we reformulate the equivalent realizations of the minimal representation found in the previous section, using now the minimal K-type to understand the different pictures.

\underline{5.2.} Without loss of generality, we may and do assume p ≥ q in this section. Instead of K-fixed vectors, our idea here is to focus on an O(p) × O(q − 1)-fixed vector. Then, it follows from Lemma 2.6 that such a vector, which we shall denote by F_{0}, is unique in our minimal representation (ϖ^{p,q}, V^{p,q}) up to a scalar multiple, and is contained in the minimal K-type of the form 1 ⊠ H^{\frac{p−q}{2}} (ℝ^{q}). (We note that this K-type is not one dimensional if p ≠ q).

We shall find an explicit formula of the Fourier transform of this vector F_{0} after a conformal change of coordinates. We start with the following classical lemma, for which we give a proof for the sake of the completeness. We take a coordinate (u_{p+1}, …, u_{p+q}) in ℝ^{q} and realize O(q − 1) in O(q) such that it stabilizes the last coordinate u_{p+q}.

Lemma 5.2. For any l ∈ ℕ, O(q − 1)-invariant spherical harmonics of degree l form a one dimensional vector space. More precisely, we have

H^{l}(ℝ^{q})^{O(q−1)} ≃ ℂ _{2}F_{1}(\frac{−l}{2} , \frac{q − 2 + l}{2} ; \frac{q − 1}{2}; u_{p+1}^{2} + ⋯ + u_{p+q−1}^{2}).

Proof. In terms of the polar coordinate of S^{q−1}:

(5.2.1)  Φ_{++} : S^{q−2} × (0, \frac{π}{2} ) → S^{q−1}, (y, θ) ↦ ((sin θ)y, cos θ),

the Laplace-Beltrami operator on S^{q−1} takes the form:

(5.2.2)  Δ_{S^{q−1}} = \frac{∂^{2}}{∂θ^{2}} + (q − 2) cot θ \frac{∂}{∂θ} + \frac{1}{sin^{2} θ} Δ_{S^{q−2}}.

If F ∈ H^{l}(ℝ^{q}) is O(q−1)-invariant, then F ∘Φ_{++}(θ, y) depends only on θ, for which we write h(θ). Then h(θ) is an even function satisfying:

(\frac{d^{2}}{dθ^{2}} + (q − 2) cot θ \frac{d}{dθ} + l(l + q − 2))h(θ) = 0.

Since h(θ) is regular at θ = 0, it is a scalar multiple of the Jacobi function:

φ_{\sqrt{−1}(l+ \frac{q−2}{2} )}^{\frac{q−3}{2} ,− \frac{1}{2}}(\sqrt{−1}θ) = _{2}F_{1}(\frac{−l}{2} , \frac{q − 2 + l}{2} ; \frac{q − 1}{2}; sin^{2} θ)

= _{2}F_{1}(−l, q − 2 + l; \frac{q − 1}{2}; \frac{1 − cos θ}{2} ).

Thus, we have proved the lemma. □

\underline{5.3.} In view of Lemma 2.6, the special case of Lemma 5.2 with l = \frac{p−q}{2} yields:

Proposition 5.3. Suppose p ≥ q ≥ 2, p + q ∈ 2ℕ and (p, q) ≠ (2, 2). Let (u_{1}, …, u_{p+q}) be the coordinate of M = S^{p−1} ×S^{q−1} in ℝ^{p+q}. We define a function F_{0} : M → ℂ by

(5.3.1) F_{0}(u_{1}, …, u_{p+q}) := _{2}F_{1}(\frac{q − p}{4}, \frac{p + q − 4}{4}; \frac{q − 1}{2}; u_{p+1}^{2} + ⋯ + u_{p+q−1}^{2}).

Then F_{0} is an O(p) × O(q − 1)-invariant analytic function on M satisfying the Yamabe equation Δ̃_{M}F_{0} = 0. Conversely, any such function is a scalar multiple of F_{0}.

Remark. If p + q is odd, then F_{0} in the right side of (5.3.1) still gives a solution Δ̃_{M}F = 0 on an open dense set of M such that u_{p+q} ≠ 0. Furthermore, F_{0} is a continuous function on M. However, it does not solve the Yamabe equation as a distribution on M.

\underline{5.4.} We recall τ(z′, z′′) is a conformal factor defined in (2.8.1). Let us define

f_{0} := Ψ∗_{\frac{n−2}{2}} F_{0}, namely,

(5.4.1) f_{0}(z′, z′′) := τ(z′, z′′)^{− \frac{p+q−4}{2}} _{2}F_{1}(\frac{q − p}{4}, \frac{p + q − 4}{4}; \frac{q − 1}{2}; \frac{|z′′|^{2}}{τ(z′, z′′)^{2}} ).

We note that |τ(z′, z′′)| ≥ |z′′| for any (z′, z′′) ∈ ℝ^{p−1,q−1}. The equality holds if and only if |z′|^{2} − |z′′|^{2} = −4.

The following Proposition is immediate from Lemma 2.8 and Proposition 5.3:

Proposition 5.4. With the same assumption on p, q in Proposition 5.3, we have:

1) f_{0} is a real analytic function on ℝ^{n} that solves □_{ℝ^{p−1,q−1}}f_{0} = 0.

2) f_{0} is O(p) × O(q − 1)-invariant.

We say F_{0} is the generating function of V^{p,q} = Ker Δ̃_{M}, and f_{0} is that of (ϖ_{ℝ^{p−1,q−1}}^{min}, H).

Remark. More strongly than Proposition 5.4, one can prove that f_{0} is a real analytic solution of □_{ℝ^{p−1,q−1}}f_{0} = 0 if p + q > 4 by using Proposition 5.6, where we do not assume that p + q is even. The real analyticity is not obvious from the expression (5.4.1) in the neighbourhood of the hypersurface of |z′|^{2} − |z′′|^{2} = −4.

\underline{5.5.} We recall the definitions of Bessel functions:

J_{ν}(z) = ∑_{m=0}^{∞} \frac{(−1)^{m}(\frac{1}{2}z)^{ν+2m}}{m! Γ(ν + m + 1)}  (Bessel function),

I_{ν}(z) = ∑_{m=0}^{∞} \frac{(\frac{1}{2}z)^{ν+2m}}{m! Γ(ν + m + 1)}  (modified Bessel function of the first kind),

K_{ν}(z) = \frac{π}{2} \frac{I_{−ν}(z) − I_{ν}(z)}{sin νπ}  (modified Bessel function of the second kind).

Then K_{ν} satisfies

(z^{2} \frac{d^{2}}{dz^{2}} + z \frac{d}{dz} − (z^{2} + ν^{2})) K_{ν}(z) = 0.

The asymptotic of the functions K_{ν} is well-known, for example z^{−ν}K_{ν}(z) decays exponentially as z → +∞.

Theorem 5.5. We put |ζ| := (ζ_{1}^{2} + ⋯ + ζ_{n}^{2})^{\frac{1}{2}} for ζ = (ζ_{1}, …, ζ_{n}) ∈ ℝ^{n}. Let F_{0} ∈ C^{∞}(M) be the generating function of V^{p,q} = Ker Δ̃_{M} (see Proposition 5.3). Then

(FΨ∗_{\frac{n−2}{2}} F_{0})(ζ) = (2π)^{\frac{p+q−2}{2}} 2^{− \frac{p−5}{2}} \frac{Γ(\frac{q−1}{2})}{Γ(\frac{p+q−4}{2})}|ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|)δ(Q).

Note that the |ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|) belongs to the Hilbert space L^{2}(C) if p + q > 4 by the asymptotic behaviour of the Bessel function K_{ν} and by the explicit form of δ(Q) in (3.3.3).

\underline{5.6.} Theorem 5.5 follows from the following Proposition:

Proposition 5.6. We write F^{−1} for the inverse Fourier transform. With notation in (5.4.1), we have

(5.6.1)  F^{−1} (|ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|)δ(Q)) (z) = \frac{Γ(\frac{p+q−4}{2})}{2^{\frac{q+3}{2}} π^{\frac{p+q−2}{2}} Γ(\frac{q−1}{2})}f_{0}(z).

Proof. Let ϕ(r) be a function of one variable, which will be taken later to be r^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2r). Then, it follows from (3.3.3) that for z = (z′, z′′) ∈ ℝ^{p−1,q−1},

2F^{−1} (ϕ(|ζ|)δ(Q)) (z)

= (2π)^{−(p+q−2)} ∫_{0}^{∞} ∫_{S^{p−2}} ∫_{S^{q−2}} ϕ(r)e^{−\sqrt{−1}((z′,rω)+(z′′,rη))} r^{p+q−5}dr dω dη

Using the formula ∫_{S^{m−1}} e^{\sqrt{−1}t(η,ω)}dω = (2π)^{\frac{m}{2}} t^{1− \frac{m}{2}} J_{\frac{m}{2} −1}(t), we have

=(2π)^{− \frac{p+q−2}{2}} ∫_{0}^{∞} ϕ(r) (r|z′|)^{\frac{3−p}{2}} J_{\frac{p−3}{2}} (r|z′|)(r|z′′|)^{\frac{3−q}{2}} J_{\frac{q−3}{2}} (r|z′′|)r^{p+q−5} dr
=(2π)^{− \frac{p+q−2}{2}} |z′|^{\frac{3−p}{2}} |z′′|^{\frac{3−q}{2}} ∫_{0}^{∞} ϕ(r)J_{\frac{p−3}{2}} (r|z′|)J_{\frac{q−3}{2}} (r|z′′|)r^{\frac{p+q−4}{2}} dr.

Now, put ϕ(r) := r^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2r). We use the following formula of the Hankel transform due to Bailey [2] (see also [6], §19.6 (8))

∫_{0}^{∞} t^{λ−1}J_{μ}(at)J_{ν}(bt)K_{ρ}(ct)dt

=\frac{2^{λ−2}a^{μ}b^{ν}Γ(\frac{1}{2}(λ + μ + ν − ρ))Γ(\frac{1}{2}(λ + μ + ν + ρ))}{c^{λ+μ+ν}Γ(μ + 1)Γ(ν + 1)}

× F_{4}(\frac{1}{2}(λ + μ + ν − ρ), \frac{1}{2}(λ + μ + ν + ρ); μ + 1, ν + 1; −\frac{a^{2}}{c^{2}} , −\frac{b^{2}}{c^{2}} ).

Here, F_{4} is the Appell hypergeometric function of two variables, defined by

F_{4}(a, b; c, d; x, y) = ∑_{i=0}^{∞} ∑_{j=0}^{∞} \frac{(a)_{i+j}(b)_{i+j}}{i!j!(c)_{i}(d)_{j}} x^{i}y^{j}.

Then we have:

F^{−1} (ϕ(|ζ|)δ(Q)) (z)
=(2π)^{− \frac{p+q−2}{2}} |z′|^{\frac{3−p}{2}} |z′′|^{\frac{3−q}{2}} ∫_{0}^{∞} K_{\frac{q−3}{2}} (2r)J_{\frac{p−3}{2}} (r|z′|)J_{\frac{q−3}{2}} (r|z′′|)r^{\frac{p−1}{2}} dr
=(2π)^{− \frac{p+q−2}{2}} 2^{\frac{p−3}{2}} \frac{Γ(\frac{p+q−4}{2})}{Γ(\frac{q−1}{2})}F_{4}(\frac{p − 1}{2}, \frac{p + q − 4}{2}; \frac{p − 1}{2}, \frac{q − 1}{2}; \frac{−|z′|^{2}}{4} ; \frac{−|z′′|^{2}}{4} ).

\underline{5.7.} Then, the proof of Proposition 5.6 will be finished by showing the following reduction formula:

Lemma 5.7. Let τ(z′, z′′) be the conformal factor defined in (2.8.1). We have

F_{4}(\frac{p − 1}{2}, \frac{p + q − 4}{2}; \frac{p − 1}{2}, \frac{q − 1}{2}; \frac{−|z′|^{2}}{4} ; \frac{−|z′′|^{2}}{4} )

(5.7.1)  =τ(z′, z′′)^{− \frac{p+q−4}{2}} _{2}F_{1}(\frac{q − p}{4}, \frac{p + q − 4}{4}; \frac{q − 1}{2}; \frac{|z′′|^{2}}{τ(z′, z′′)^{2}} ).

Proof of Lemma 5.7. We recall a reduction formula of Appell’s hypergeometric functions (see [5], §5.10, (8)):

F_{4}(α, β; 1 + α − β, β; \frac{−x}{(1 − x)(1 − y)}, \frac{−y}{(1 − x)(1 − y)}) (5.7.2)

=(1 − y)^{α}_{2}F_{1}(α, β; 1 + α − β; \frac{−x(1 − y)}{1 − x})

and a quadratic transformation for hypergeometric functions (see [5], §2.11 (32)):

(5.7.3) _{2}F_{1}(α, β, 1 + α − β; z) = (1 − z)^{−α}_{2}F_{1}(\frac{α}{2}, \frac{α + 1 − 2β}{2} ; 1 + α − β; \frac{−4z}{(1 − z)^{2}} ).

Combining (5.7.2) with (5.7.3) for α = \frac{p+q−4}{2} and β = \frac{p−1}{2}, and using the symmetry of a and b; (c, x) and (d, y) in F_{4}(a, b; c, d; x, y), we have

F_{4}(\frac{p − 1}{2}, \frac{p + q − 4}{2}; \frac{p − 1}{2}; \frac{q − 1}{2}; \frac{−x}{(1 − x)(1 − y)}, \frac{−y}{(1 − x)(1 − y)})

= (\frac{(1 − x)(1 − y)}{1 − xy})^{\frac{p+q−4}{2}} _{2}F_{1}(\frac{p + q − 4}{4}, \frac{q − p}{4}; \frac{q − 1}{2}; \frac{4y(1 − x)(1 − y)}{(1 − xy)^{2}})

If we put

\frac{|z′|^{2}}{4} = \frac{x}{(1 − x)(1 − y)}, \frac{|z′′|^{2}}{4} = \frac{y}{(1 − x)(1 − y)}

then a simple computation shows

τ(z′, z′′)^{2} = (\frac{1 − xy}{(1 − x)(1 − y)})^{2} , \frac{|z′′|^{2}}{τ(z′, z′′)^{2}} = \frac{4y(1 − x)(1 − y)}{(1 − xy)^{2}}.

Thus, Lemma 5.7 is proved. □

\underline{5.8.} Using the generating function F_{0} we may recover the whole representation by letting the Lie algebra of G act. Let us see how our previous results may be reformulated: It follows from the definition of ϖ_{\frac{n−2}{2} ,ϵ} (see §3.2) that the linear map

F ∘ Ψ∗_{\frac{n−2}{2}} : C^{∞}(M) → S′(ℝ^{n})

induces a natural intertwining map from (ϖ^{p,q}, V^{p,q}) to (ϖ_{\frac{n−2}{2} ,ϵ}, FΨ∗_{\frac{n−2}{2}} (V^{p,q})) as

G-modules and also as 𝔤-modules. Here again ϵ = (−1)^{\frac{p−q}{2}} , V^{p,q} = Ker Δ̃_{M} (⊂

C^{∞}(M)) and Ψ∗_{\frac{n−2}{2}} (V^{p,q}) ⊂ Ker □_{ℝ^{p−1,q−1}}.

For b ∈ ℝ^{n}, m ∈ O(p − 1, q − 1), we put a function on the cone C by

(5.8.1)  ψ_{b,m}(ζ) := e^{\sqrt{−1}⟨b,ζ⟩}|mζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|mζ|).

In particular, we have

ψ_{0,e}(ζ) = |ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|).

Here we give explicit functions which are dense in the minimal representations.

Theorem 5.8. Suppose p ≥ q ≥ 2, p + q ∈ 2ℕ, and (p, q) ≠ (2, 2).

1) ψ_{0,e}(ζ) is a K-finite vector of (π, L^{2}(C)). It belongs to the minimal K-type of π.

2) ℂ-span{ψ_{b,m} : b ∈ ℝ^{n}, m ∈ O(p − 1, q − 1)} is a dense subspace of the minimal representation (π, L^{2}(C)). 2′) ℂ-span{ψ_{b,m}δ(Q) : b ∈ ℝ^{n}, m ∈ O(p − 1, q − 1)} is a dense subspace of the minimal representation (ϖ_{\frac{n−2}{2} ,ϵ}, FΨ∗_{\frac{n−2}{2}} (\overline{V^{p,q}})).

Proof. This follows by combining Theorem 4.9 and Lemma 5.5, and using that by Mackey theory we have an irreducible representation of the parabolic group. □

\underline{5.9.} The advantage in the realization on L^{2}(C) is that the action for \overline{P^{max}} and the inner product are easily described. On the other hand, the action of K is not easy to be described, and especially, the K-finiteness in the statement (2) is non-trivial.

Let U(𝔤) the enveloping algebra of the complexified Lie algebra 𝔤. We define a subspace of S′(ℝ^{n}) by

(5.9.1)  U := dϖ_{\frac{n−2}{2} ,ϵ}(U(𝔤))(ψ_{0,e}δ(Q)).

That is, U is the linear span of a Bessel function |ζ|^{\frac{3−q}{2}} K_{\frac{q−3}{2}} (2|ζ|)dμ on the cone C and its iterative differentials corresponding to the action of the Lie algebra 𝔤.

We have seen that FΨ∗_{\frac{n−2}{2}} (V^{p,q}) ⊂ T (L^{2}(C)) in Theorem 4.9. We may restate in this way:

Theorem 5.9. 1) U is an infinitesimally unitary (𝔤, K)-module via ϖ_{\frac{n−2}{2} ,ϵ}.

2) U is dense in the Hilbert space T (L^{2}(C)).

3) The completion of (1) defines an irreducible unitary representation of G on T (L^{2}(C)), and then also on L^{2}(C). This gives an extension of π from \overline{P^{max}} to G.

This has already been done by Theorem 4.9 and the irreducibility of the minimal representation.

One of non-trivial parts of the above assertion is to show

U ∩ T (L^{2}(C)) ≠ {0}

which was proved in Theorem 5.5.

§6. Explicit inner product on solutions □_{ℝ^{p−1,q−1}}f = 0

\underline{6.1.} The aim of this section is to provide an explicit inner product on a certain subspace (see (6.2.1)) of solutions of the ultrahyperbolic equation □_{ℝ^{p−1,q−1}}f = 0, such that its Hilbert completion gives the unitarization of the minimal representation of O(p, q).

Roughly speaking, the inner product will be given in terms of the integration over a hyperplane after convoluting a distribution along the normal direction.

We assume n > 2. We fix i ∈ {1, 2, …, n} once and for all. The hyperplane on which we integrate will be {z = (z_{1}, …, z_{n}) ∈ ℝ^{n} : z_{i} = 0}, for which we simply write ℝ^{n−1}.

\underline{6.2.} Let C^{(i)} := \{ζ ∈ C : ζ_{i} ≠ 0\}, an open dense subset of the null cone C (see (3.3.2)). We note that C_{0}^{∞}(C \ {0}) is dense in L^{2}(C). We define a subspace of solutions of Ker □_{ℝ^{p−1,q−1}} by

(6.2.1)  W := F^{−1} ∘ T (C_{0}^{∞}(C^{(i)})).

Here, we recall T : L^{2}(C) ↪ S′(ℝ^{n}) is the embedding via the measure dμ on the cone C. By the Paley-Wiener theorem for compactly supported distributions, W consists of real analytic solutions of □_{ℝ^{p−1,q−1}}f = 0.

Using an interpretation of the Dirac delta function in terms of hyperfunctions:

δ(z_{i}) = \frac{1}{2π\sqrt{−1}} (\frac{1}{z_{i} − \sqrt{−1}0} − \frac{1}{z_{i} + \sqrt{−1}0}) ,

we decompose f ∈ W as

(6.2.2)  f(z) = f_{+}^{(i)}(z) + f_{−}^{(i)}(z),

where f_{±}^{(i)}(z) is defined by the convolution in the z_{i}-variable:

(6.2.3)  f_{±}^{(i)}(z) := \frac{1}{2π\sqrt{−1}} · \frac{±1}{z_{i} ∓ \sqrt{−1}0} ∗ f(z).

We shall see later that the decomposition (6.2.2) makes sense not only for f ∈ W but also for any f ∈ F^{−1} ∘ T (L^{2}(C)) (see Lemma 6.5). We set

(6.2.4) (f, f)_{W} := \frac{1}{\sqrt{−1}} ∫_{ℝ^{n−1}} (f_{+}^{(i)} \overline{\frac{∂f_{+}^{(i)}}{∂z_{i}}} − f_{−}^{(i)} \overline{\frac{∂f_{−}^{(i)}}{∂z_{i}}} ) |_{z_{i}=0} dz_{1} ⋯ d̂z_{i} ⋯ dz_{n}

Theorem 6.2. Fix any i ∈ {1, 2, …, n}.

1) The formula (6.2.4) defines a positive definite Hermitian form on W, a subspace of solutions of the ultrahyperbolic operator □_{ℝ^{p−1,q−1}} = \frac{∂^{2}}{∂z_{1}^{2}} + ⋯ + \frac{∂^{2}}{∂z_{p−1}^{2}} − \frac{∂^{2}}{∂z_{p}^{2}} −

⋯ − \frac{∂^{2}}{∂z_{p+q−2}^{2}} on ℝ^{p−1,q−1}.

2) The inner product (6.2.4) is independent of the choice of i ∈ {1, 2, …, n}.

3) The action of G = O(p, q) preserves the inner product (6.2.4), so that the Hilbert completion \overline{W} of W defines a unitary representation of G = O(p, q).

4) The resulting unitary representation is unitarily equivalent to the minimal representation (ϖ^{p,q}, \overline{V^{p,q}}). The G-intertwining operator Ψ∗_{\frac{n−2}{2}} : \overline{V^{p,q}} → \overline{W} gives a unitary equivalence up to a scalar constant.

5) cF^{−1} ∘ T : L^{2}(C) → \overline{W} is a unitary G-intertwining operator, if we put c = 2^{\frac{n+2}{2}} π^{\frac{n+1}{2}} .

This Theorem gives a new formulation of the Hilbert space of the minimal representation purely in terms of intrinsic objects in the flat space ℝ^{p−1,q−1} where the differential equation is the classical ultrahyperbolic one. It generalizes the q = 2 case where an inner product was known in terms of integration of Cauchy data - here one could interpret the inner product in terms of the energy generator. The interesting property about the inner product is its large invariance group; even translational invariance amounts to a remarkable “conservation law”, and we may also note that the integration over a coordinate hyperplane can be replaced by integration over any non-characteristic hyperplane (since such a hyperplane is conjugate to either z_{1} = 0 or z_{n} = 0 by O(p − 1, q − 1) ⋉ ℝ^{p−1,q−1}), or even the image of such a hyperplane under conformal inversion.

The strategy of the proof of Theorem 6.2 is as follows: We recall from Theorem 4.7 that the Hilbert space H is the completion of the space S(C_{0}^{∞}(ℝ^{n})) with respect to another inner product ( , )_{N} (S is an integral transform by the Green kernel). Since C_{0}^{∞}(C \ {0}) is dense in L^{2}(C) and since T^{−1} ∘ F : H → L^{2}(C) is an isomorphism of Hilbert spaces (up to a scalar) by Theorem 4.9 (1), W is a dense subspace of the Hilbert space (H, ( , )_{N}). In light of this, the key ingredient of the proof of Theorem 6.2 is to give a formula of ( , )_{W} by means of ( , )_{L^{2}(C)}. We shall prove:

(6.2.5)  2(2π)^{n+1}(f, f)_{W} = ∥T^{−1} ∘ Ff∥_{L^{2}(C)}^{2} for any f ∈ W.

Once we prove (6.2.5), it follows from (4.9.1) that

4π( , )_{W} = ( , )_{N}

on the subspace W. In particular, we have

H = \overline{W} ,

and all other statements of Theorem 6.2 on our inner product ( , )_{W} are clear from the corresponding results on the inner product ( , )_{N} proved in Theorem 4.7 and Theorem 4.9 (e.g. Theorem 6.2 (4) corresponds to Theorem 4.7 (5); Theorem 6.2 (5) to Theorem 4.9 (3)).

The rest of this section is devoted to the proof of the formula (6.2.5).

\underline{6.3.} We define an open subset of the cone C by

C_{±}^{(i)} := {ζ ∈ C : ±ζ_{i} > 0}.

Then

C^{(i)} := C_{+}^{(i)} ∪ C_{−}^{(i)}

is an open dense subset of the cone C, and we have a direct sum decomposition of the Hilbert space:

L^{2}(C) = L^{2}(C_{+}^{(i)}) ⊕ L^{2}(C_{−}^{(i)}).

We define the Heaviside function Y_{±}^{(i)}(ζ) of the variable ζ_{i} by

Y_{±}^{(i)}(ζ_{1}, …, ζ_{n}) = \{ \begin{matrix}1 & if ±ζ_{i} > 0, \\ 0 & if ±ζ_{i} ≤ 0, \end{matrix} for ζ = (ζ_{1}, …, ζ_{n}) ∈ ℝ^{n}.

For ϕ ∈ C_{0}^{∞}(C^{(i)}), we put

ϕ_{±}^{(i)} := Y_{±}^{(i)}ϕ.

Then Supp ϕ_{±}^{(i)} ⊂ C_{±}^{(i)}, and we have

(6.3.1)  ϕ = ϕ_{+}^{(i)} + ϕ_{−}^{(i)},
(6.3.2)  ∥ϕ∥_{L^{2}(C)}^{2} = ∥ϕ_{+}^{(i)}∥_{L^{2}(C)}^{2} + ∥ϕ_{−}^{(i)}∥_{L^{2}(C)}^{2}.

\underline{6.4.} Let us take the n−1 variables ζ_{1}, …, ζ̂_{i}, …, ζ_{n} as a coordinate on C_{±}^{(i)}. Then we have

(6.4.1)  ζ_{i} = ±\sqrt{Q^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n})}

on C_{±}^{(i)}, respectively, if we put

(6.4.2) Q^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) := −ε_{i} (ζ_{1}^{2} + ζ_{2}^{2} + ⋯ ˆ±ζ_{i}^{2} ± ⋯ − ζ_{n−1}^{2} − ζ_{n}^{2}) ,

where ε_{i} = ±1 is the signature of ζ_{i}^{2} in the quadratic form Q(ζ) as in (2.5.1). We note that Q^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) ≥ 0 on the cone C, and the map

(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) ↦ (ζ_{1}, …, ±\sqrt{Q^{(i)}}, …, ζ_{n})

gives a bijection from {(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) ∈ ℝ^{n} : Q^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) ≠ 0} onto C^{(i)}. By substituting (6.4.1) into ϕ_{±}^{(i)}, we put

φ_{±}^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n}) := ϕ_{±}^{(i)}(ζ_{1}, …, ±\sqrt{Q^{(i)}}, …, ζ_{n}).

Since the measure dμ on the cone C is of the form \frac{1}{2\sqrt{Q^{(i)}}} dζ_{1} ⋯ d̂ζ_{i} ⋯ dζ_{n}, we have

(6.4.3) ∥ϕ_{±}^{(i)}∥_{L^{2}(C)}^{2} = ∫_{ℝ^{n−1}} \frac{|φ_{±}^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n})|^{2}}{2\sqrt{Q^{(i)}}} dζ_{1} ⋯ d̂ζ_{i} ⋯ dζ_{n}
(6.4.4) F^{−1}T ϕ_{±}^{(i)}(z) = \frac{1}{(2π)^{n}} ∫_{ℝ^{n−1}} e^{−\sqrt{−1}(z_{1}ζ_{1}+···+ẑ_{i}ζ_{i}+···+z_{n}ζ_{n})}

× e^{∓\sqrt{−1}z_{i}\sqrt{Q^{(i)}}} \frac{φ_{±}^{(i)}(ζ_{1}, …, ζ̂_{i}, …, ζ_{n})}{2\sqrt{Q^{(i)}}}dζ_{1} ⋯ d̂ζ_{i} ⋯ dζ_{n}.

From (6.4.4) we have the following:

Lemma 6.4. We write F_{ℝ^{k}} for the Fourier transform in ℝ^{k} (k = n − 1, n). Then,

F_{ℝ^{n}}^{−1}T ϕ_{±}^{(i)}|_{z_{i}=0} = \frac{1}{4π} F_{ℝ^{n−1}}^{−1} (\frac{φ_{±}^{(i)}}{\sqrt{Q^{(i)}}} ) (z_{1}, …, ẑ_{i}, …, z_{n}),

\frac{∂}{∂z_{i}} |_{z_{i}=0}F_{ℝ^{n}}^{−1}T ϕ_{±}^{(i)} = \frac{∓\sqrt{−1}}{4π}F_{ℝ^{n−1}}^{−1} (φ_{±}^{(i)}) (z_{1}, …, ẑ_{i}, …, z_{n}).

\underline{6.5.} We recall the Fourier transform of the Riesz potential

∫_{−∞}^{∞} e^{\sqrt{−1}xξ}ξ_{+}^{λ} dx = \sqrt{−1}e^{\frac{\sqrt{−1}λπ}{2}} Γ(λ + 1)(ξ + \sqrt{−1}0)^{−λ−1}

for a meromorphic parameter λ. Letting λ = 0, we have

F^{−1}(Y_{±}^{(i)}) = \frac{∓\sqrt{−1}}{2π}(z_{i} ∓ \sqrt{−1}0)^{−1}δ(z_{1}, …, ẑ_{i}, …, z_{n}),

where δ(z_{1}, …, ẑ_{i}, …, z_{n}) is the Dirac delta function of n − 1 variables. Then we have

F^{−1}(Y_{±}^{(i)} · T ϕ) = (F^{−1}Y_{±}^{(i)}) ∗ (F^{−1}T ϕ)

= \frac{∓\sqrt{−1}}{2π}(z_{i} ∓ \sqrt{−1}0)^{−1}δ(z_{1}, …, ẑ_{i}, …, z_{n}) ∗ f(z)

= \frac{±1}{2π\sqrt{−1}}(z_{i} ∓ \sqrt{−1}0)^{−1} ∗ f(z).

Here the first two convolutions are for n variables z_{1}, …, z_{n}, while the last one for only z_{i} ∈ ℝ. In view of the definition of ϕ_{±}^{(i)} and f_{±}^{(i)}, we have proved

Lemma 6.5. If f = F^{−1}T ϕ then f_{±}^{(i)} = F^{−1}T ϕ_{±}^{(i)}.

\underline{6.6.} We are now ready to complete the proof of Theorem 6.2. By using Lemma 6.4 and Lemma 6.5, the Plancherel formula for ℝ^{n−1}, and the integration formula (6.4.3), respectively, we have

(f_{±}^{(i)}|_{z_{i}=0}, \frac{∂f_{±}^{(i)}}{∂z_{i}} |_{z_{i}=0})_{L^{2}(ℝ^{n−1})} = \frac{±\sqrt{−1}}{16π^{2}}(F_{ℝ^{n−1}}^{−1}(\frac{φ_{±}^{(i)}}{\sqrt{Q^{(i)}}} )), F_{ℝ^{n−1}}^{−1}(φ_{±}^{(i)}))_{L^{2}(ℝ^{n−1})}

= \frac{±\sqrt{−1}}{16π^{2}(2π)^{n−1}} (\frac{φ_{±}^{(i)}}{\sqrt{Q^{(i)}}} , φ_{±}^{(i)})_{L^{2}(ℝ^{n−1})}

= \frac{±\sqrt{−1}}{2(2π)^{n+1}} ∥ϕ_{±}^{(i)}∥_{L^{2}(C)}^{2}.

Hence,

(f, f)_{W} = \frac{1}{2(2π)^{n+1}} (∥ϕ_{+}^{(i)}∥_{L^{2}(C)}^{2} + ∥ϕ_{−}^{(i)}∥_{L^{2}(C)}^{2}) = \frac{1}{2(2π)^{n+1}} ∥ϕ∥_{L^{2}(C)}^{2}.

This finishes the proof of the formula (6.2.5) and hence Theorem 6.2. □

\underline{6.7.} The main content of Theorem 6.2 is to give yet another realization of the inner product and of the Hilbert space. This is very close to the form most known in the case of the wave equation, where one integrates Cauchy data on the zero time hypersurface to get the inner product. Note the connection to the theory of conserved quantities for the wave equation - we end the paper by making more explicit this final remark:

When p = 2 let us introduce time and space coordinates (t, x) by

(t, x) = (t, x_{1}, …, x_{n−1}) = (z_{1}, …, z_{n})

and the dual variable k to x so that positive-energy solutions to the wave equation are given by the Fourier transform (i = \sqrt{−1})

u^{+}(t, x) = ∫_{ℝ^{n−1}} e^{i(kx−t|k|)} \frac{φ^{+}(k)}{|k|}dk

and similarly for negative-energy solutions

u^{−}(t, x) = ∫_{ℝ^{n−1}} e^{i(kx+t|k|)} \frac{φ^{−}(k)}{|k|}dk

where kx denotes the usual scalar product, |k| the Euclidean length, and dk = dk_{1} ⋯ dk_{n−1}. Any solution is the sum of two such: u = u^{+} + u^{−}. The energy of the wave u is given by

E(u) = \frac{1}{2} ∫_{ℝ^{n−1}}(|u_{t}|^{2} + |∇u|^{2})dx

and is a conserved quantity, i.e. independent of which constant-time hyperplane we integrate over. It is easy to see that cross-terms drop out, so that on the Fourier transform side we obtain

E(u) = (2π)^{n−1} ∫_{ℝ^{n−1}}(|φ^{+}(k)|^{2} + |φ^{−}(k)|^{2})dk

for the energy. Note that this energy only differs from our inner product by a density factor of |k|, and that it may be thought of as an integral over C. In terms of our conformally invariant inner product (6.2.4) this is up to a constant just

(u, |H|u) = (u^{+}, Hu^{+}) − (u^{−}, Hu^{−}),

where H = i∂_{t} is the energy generator (infinitesimal time translations). In the same way, we have the analogous “conserved quantities” for the ultrahyperbolic equation and the inner product (6.2.4), namely: Let H_{j} = i∂_{z_{j}} be the generator of translations in the coordinate z_{j}, then for a solution f in the Hilbert space

E_{j}(f) = (f, |H_{j}|f)

is invariant under translations in the coordinate z_{j}. Furthermore, the quantity E_{j}(f) can be expressed in terms of an integral of local quantities. In particular we may use this to study uniqueness and decay properties of solutions to □_{ℝ^{p−1,q−1}}f = 0. Since this is outside the scope of the present paper, we shall not do so here; but note that one easy consequence is the fact, that if a solution and its normal derivative vanish on a coordinate hyperplane, then it is identically zero - a classical fact about the energy (time zero hyperplane).

References

[1] M. Abramowitz and I. A. Stegun (eds.), Handbook of mathematical functions with formulas, graphs, and mathematical tables, Dover, 1992, Reprint of the 1972 edition.

[2] W. N. Bailey, Some infinite integrals involving Bessel functions, Proc. London Math. Soc. 40 (1935-36), 37–48.

[3] B. Binegar and R. Zierau, Unitarization of a singular representation of SO(p, q), Comm. Math. Phys. 138 (1991), 245–258.

[4] A. Dvorsky and S. Sahi, Explicit Hilbert spaces for certain unipotent representations II, Invent. Math. 138 (1999), 203-224.

[5] A. Erdélyi, Higher Transcendental Functions, vol. I, McGraw-Hill, New York, 1953.

[6] A. Erdélyi, Tables of Integral Transforms, vol. II, McGraw-Hill, New York, 1954.

[7] I. M. Gelfand and G. E. Shilov, Generalized Functions, I, Academic Press, 1964.

[8] V. Guillemin and S. Sternberg, Variations on a theme by Kepler, AMS Colloquium Publications, vol. 42, 1990.

[9] L. Hörmander, Asgeirsson’s mean value theorem and related identities, J. Func. Anal. 184 (2001), 337–402.

[10] T. Kobayashi, Discrete decomposability of the restriction of A_{𝔮}(λ) with respect to reductive subgroups and its applications, Invent. Math. 117 (1994), 181–205 ; Part II, Ann. of Math. 147 (1998) 709–729; Part III, Invent. Math. 131 (1998) 229–256.

[11] T. Kobayashi and B. Ørsted, Analysis on the minimal representation of O(p, q) – I. Realization via conformal geometry, preprint.

[12] , Analysis on the minimal representation of O(p, q) – II. Branching laws, preprint.

[13] B. Kostant, The vanishing scalar curvature and the minimal unitary representation of SO(4, 4), Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory (Connes et al, eds.), vol. 92, Birkhäuser, Boston, 1990, pp. 85–124.

[14] I. T. Todorov, Derivation and solution of an infinite-component wave equation for the relativistic Coulomb problem, Lecture Notes in Physics, Group representations in mathematics and physics (Rencontres, Battelle Res. Inst., Seattle, 1969), vol. 6, Springer, 1970, pp. 254– 278.

RIMS, Kyoto University, Sakyo-ku, Kyoto, 606-8502, Japan

Department of Mathematics and Computer Science, SDU-Odense University, Campusvej 55, DK-5230, Odense M, Denmark

E-mail address: toshi@kurims.kyoto-u.ac.jp; orsted@imada.sdu.dk