Introduction to visible actions on complex manifolds and multiplicity-free representations ^{∗}

京都大学・数理解析研究所 小林俊行 (Toshiyuki Kobayashi)

Research Institute for Mathematical Sciences, Kyoto University

Abstract

Recently, we established the theory of multiplicity-free representations based on visible actions on complex manifolds ([Ko97, Ko05a, Ko06a, Ko06d]).

The purpose of this article is to give guidance on this subject with short comments on the references.

Contents

1 Multiplicity-free representations 2 1.1 What is a multiplicity-free representation? . . . . . . . . 2 1.2 Why are multiplicity-free representations interesting? . . 3 1.3 Known examples of multiplicity-free representations … 3 1.4 A new approach to multiplicity-free representations … 4

2 Visible actions on complex manifolds 5 2.1 Visible actions on complex manifolds . . . . . . . . . . . 5 2.2 Strongly visible actions . . . . . . . . . . . . . . . . . . . 5 2.3 Examples of visible actions . . . . . . . . . . . . . . . . . 5 2.4 Visible actions on symmetric spaces . . . . . . . . . . . . 6 2.5 Coisotropic actions on symplectic manifolds . . . . . . . 7 2.6 Polar actions on Riemannian manifolds . . . . . . . . . . 7 2.7 Coisotropic, polar, and visible actions . . . . . . . . . . . 8 3 Multiplicity-free theorems 8

∗Proceedings of the RIMS Symposium on “Developments of Cartan Geometry and Related Mathematical Problems”, on 24–27th October 2005, organized by Professor T. Morimoto.

Partly supported by Grant-in-Aid for Exploratory Research 16654014, Japan Society of the Promotion of Science.

3.1 Multiplicity-free theorems . . . . . . . . . . . . . . . . . 8 3.2 Representation theoretic meaning of S . . . . . . . . . . 10 3.3 Applications to concrete multiplicity-free theorems …. 10 3.4 Explicit decomposition formulae . . . . . . . . . . . . . . 11 3.5 Classical limits — Orbit method . . . . . . . . . . . . . . 11

1 Multiplicity-free representations

1.1 What is a multiplicity-free representation?

Suppose π : G → GL(H) is a representation of G on a finite dimensional vector space H. If (π, H) is completely reducible, we have an irreducible decomposition:

(1.1)  π ≃ ⨁_{μ} \underbrace{μ ⊕ ⋯ ⊕ μ}_{m_{π}(μ)},

where μ runs over irreducible representations of G. The non-negative integer m_{π}(μ) is equal to dim Hom_{G}(μ, π), and is called the multiplicity of μ in π. We say π is multiplicity-free if m_{π}(μ) ≤ 1 for any irreducible representation μ of G.

More generally, for an infinite dimensional representation π, we may not have a discrete direct sum decomposition like (1.1) (see [Ko94, Ko98a, Ko98b, Ko02] for the criteria for discrete decomposability of a unitary representation π). Still, we can define the concept of multiplicityfree representations as follows. Suppose π : G → GL(H) is a unitary representation of a group G on the Hilbert space H over ℂ. We denote by End_{G}(H) the ring of continuous endomorphisms of H that commute with G-actions. For example, if (π, H) has the irreducible decomposition (1.1), then we have an isomorphism of rings:

End_{G}(H) ≃ ⨁_{μ} M(m_{π}(μ), ℂ)

by Schur’s lemma. In particular, the ring End_{G}(H) is commutative if and only if the direct summand M(m_{π}(μ), ℂ) is commutative, that is, m_{π}(μ) ≤ 1. This brings us to the following:

Definition 1.1. (π, H) is multiplicity-free if the ring End_{G}(H) is commutative.

If G is a type I group (e.g. algebraic group, reductive group, etc.), then any unitary representation π of G can be decomposed uniquely into the direct integral of irreducible unitary representations:

π ≃ ∫Ĝ m_{π}(μ)μ dσ(μ),

where Ĝ denotes the unitary dual of G (the set of equivalence classes of irreducible representations of G), dσ is a Borel measure on Ĝ, and m_{π} : Ĝ → ℕ ∪ {∞} is the multiplicity function. Then, it follows from Schur’s lemma for unitary representations that (π, H) is multiplicity-free in the sense of Definition 1.1 if and only if m_{π}(μ) ≤ 1 for almost every μ ∈ Ĝ with respect to the measure dσ.

1.2 Why are multiplicity-free representations interesting?

A distinguished feature of a multiplicity-free representation is that it has a canonical decomposition into irreducibles, and consequently, any operator that respects the group action can be diagonalized according to the irreducible decomposition.

Multiplicity-free representations appear in various contexts of mathematics, though we may not be aware of even the fact that the representation is there. For further perspectives, we refer the reader to [Ko05a, Section 1.1].

1.3 Known examples of multiplicity-free representations

Over many decades, numerous examples of multiplicity-free representations have been found implicitly/explicitly in various contexts including:

• Taylor series expansion,

• Fourier expansion,

• theory of spherical harmonics,

• the Peter–Weyl theorem, • the Cartan–Helgason theorem on compact symmetric spaces

• branching laws for GL_{n} ↓ GL_{n−1} and O_{n} ↓ O_{n−1},

• the Clebsh–Gordan formula for SL_{2},

• Pieri’s law,

• GL_{m}–GL_{n} duality,

• Plancherel formula for Riemannian symmetric spaces,

• the Gelfand–Graev–Vershik canonical representations,

• the Hua–Kostant–Schmid K-type formula,

• Kac’s classification of linear multiplicity-free spaces,

• Krämer–Brion’s classification of spherical varieties,

• Panyushev’s classification of spherical nilpotent orbits,

• Stembridge’s classification of multiplicity-free tensor products. Accordingly, various techniques that explain multiplicity-free prop-

erty of those representations have been developed such as computational combinatorics, algebraic geometry (in particular, actions of Borel subgroups), the Iwahori–Hecke algebra (e.g. [Ge50]), the Schur–Weyl–Howe duality (e.g. [Ho89, Ho95]), etc (see [Ko05a] and references therein).

However, no single known-method seems to cover all of the above multiplicity-free examples.

1.4 A new approach to multiplicity-free representations The aim of our paper [Ko06a] is to present a simple principle based on complex geometry that yields various kinds of multiplicity-free representation such as the above examples.

Our machinery is explained in Section 3. It is based on the concept of visible actions (see Section 2).

2 Visible actions on complex manifolds

2.1 Visible actions on complex manifolds

Let D be a connected complex manifold with complex structure J. Suppose a Lie group G acts holomorphically on D.

Definition 2.1 ([Ko04, Definition 2.3]). The action is previsible if there exists a totally real submanifold S such that D′ := H · S is open in D.

We say that a previsible action is visible if

J_{x}(T_{x} · S) ⊂ T_{x}(H · x) for any x ∈ S.

2.2 Strongly visible actions

Definition 2.2 ([Ko05a, Definition 3.3.1]). A previsible action is strongly visible if there exists an anti-holomorphic diffeomorphism σ of D′ such that σ|_{S} = id and that σ preserves every H-orbit in D′.

It is proved in [Ko05a, Theorem 4] that a strongly visible action is visible.

2.3 Examples of visible actions

In the papers [Ko06b, Ko06c, Ko06e] and [Ko05a, Section 5], we considered the question about which actions on complex flag varieties are strongly visible, and tried to understand relevant geometric properties. The classification results on visible actions produce various multiplicityfree theorems as applications of the machinery, which we shall explain in Theorem 3.1.

We begin with the simplest example of visible actions. Let us consider the natural action of the one-dimensional toral subgroup 𝕋 = {t ∈ ℂ : |t| = 1} on ℂ given by

𝕋 × ℂ → ℂ, (t, z) ↦ tz.

Then, this action is visible as one can see from the following figure where we set S := ℝ\{0}:

𝕋-orbits on ℂ S

This action is also strongly visible. In fact, we can take the complex conjugation σ to be σ(z) := z̄.

Next, the following SL_{2}-example is taken from [Ko05a, Example 5.4.1]:

Example 2.3. Let G = SL(2, ℝ) and we define the following onedimensional subgroups of G:

K := \{(\begin{matrix}cos θ & − sin θ \\ sin θ & cos θ \end{matrix}) : θ ∈ ℝ/2πℤ\} ,

H := \{(\begin{matrix}a & 0 \\ 0 & a^{−1} \end{matrix}) : a > 0\} ,

N := \{(\begin{matrix}1 & x \\ 0 & 1 \end{matrix}) : x ∈ ℝ\} .

Then, both (G, K) and (G, H) are symmetric pairs, while N is a maximal unipotent subgroup of G.

Let us consider the actions of subgroups of G on the Hermitian symmetric space G/K. Then, all of the actions of the subgroups K, H and N on G/K become strongly visible, as one can easily see from the following figures where G/K is realized as the Poincaré disk:

K-orbits  H-orbits  N-orbits

2.4 Visible actions on symmetric spaces

In the papers [Ko06b, Ko06e], we studied visible actions, particularly in the case where D is a symmetric space. One of the main results of [Ko06b] is:

Theorem 2.4 ([Ko06b]). Let G/K be a Hermitian symmetric space, and (G, H) an arbitrary semisimple symmetric pair. Then the H-action on G/K is strongly visible.

Theorem 2.4 is a generalization of the first two cases (i.e. the K-action and the H-action) of Example 2.3.

Applications to representation theory are discussed in [Ko06d, Theorems A–F] for both finite and infinite dimensional representations.

2.5 Coisotropic actions on symplectic manifolds

In contrast to visible actions on complex manifolds, let us consider a symplectic manifold (D, ω). A submanifold S is called coisotropic if

(T_{x}S)^{⊥ω} ⊂ T_{x}S

for every x ∈ S. Here, (T_{x}S)^{⊥ω} := {u ∈ T_{x}D : ω(u, v) = 0 for any v ∈ T_{x}S}. For a submanifold S satisfying 2 dim S = dim D, S is coisotropic if and only if S is Lagrangean.

Definition 2.5 (Guillemin–Sternberg). Suppose a compact Lie group G acts on D by symplectic automorphisms. The action is called coisotropic (or multiplicity-free) if all principal orbits G · x are coisotropic with respect to the symplectic form ω.

2.6 Polar actions on Riemannian manifolds

Relevant concept is also known for Riemannian manifolds. Let (D, g) be a Riemannian manifold, and G a compact Lie group acting on D by isometries.

Definition 2.6 (e.g. [PT02]). The action is called polar if there exists a properly embedded submanifold S with the following two properties:

S meets every G-orbits.

T_{x}S ⊥ T_{x}(G · x) for any x ∈ S.

2.7 Coisotropic, polar, and visible actions

Kähler manifolds enjoy all three geometric structures: symplectic, Riemannian, and complex structures. The concepts of coisotropic, polar, and visible actions can be compared on Kähler manifolds.

Suppose G is a compact Lie group acting on compact Kähler manifolds by holomorphic isometries. Then, the following implications hold:

Riemannian Polar

Visible

& 2 dim S = dim D

Coisotropic

(multiplicity-free)

Complex  Symplectic

Strongly Visible Complex

See [PT02] and [Ko05a, Theorems 7, 8, 9] for precise statements and their proofs.

3 Multiplicity-free theorems

Finally, we explain how the concept of strongly visible actions on complex manifolds is used in the formalisation of our multiplicity-free theorem.

3.1 Multiplicity-free theorems

Suppose we are given a strongly visible action of a Lie group G on a complex manifold D. A group automorphism σ̃ of G is compatible if

σ̃(g) · σ(x) = σ(g · x) (g ∈ G, x ∈ D′).

Then, the following result is a most general form of our multiplicity-free theorem (cf. [FT99, Ko97, Ko00]):

Theorem 3.1 ([Ko06a, Theorem 4.3]). Let V → D be a G-equivariant Hermitian holomorphic vector bundle. Assume the following three conditions are satisfied:

1) (Base space) The G-action on the base space is strongly-visible with a compatible automorphism.

2) (Fiber) The isotropy representation of G_{x} on V_{x} is multiplicity-free for any x ∈ S.

We write its irreducible decomposition as

V_{x} = ⨁_{i=1}^{n(x)} V_{x}^{(i)}.

3) (Compatibility) σ lifts to an anti-holomorphic endomorphism (we use the same letter σ) of the G-equivariant Hermitian holomorphic vector bundle V such that

σ(V_{x}^{(i)}) = V_{x}^{(i)} for 1 ≤ i ≤ n(x), x ∈ S.

Then, any unitary representation which can be realized in the space O(D, V) of global holomorphic sections is multiplicity-free.

This theorem can be regarded as a propagation theorem of multiplicityfree property from fibers to the space of global sections. It is noteworthy that propagation theory of unitarity is one of the most fundamental results in representation theory. It was established by Mackey for induced representations in 1950s and by Vogan and Wallach for cohomologically induced representations in 1980s. From the viewpoint of ‘propagation’ of multiplicity-free property, strongly visible actions (i.e. the condition 1) in Theorem 3.1) has a key role in the geometry.

Another important aspect of Theorem 3.1 is that we are dealing with global analysis on manifolds having infinitely many orbits (in contrast to the usual sense of non-commutative harmonic analysis, where we basically deal with manifolds having only one orbit, namely, homogeneous spaces). The anti-holomorphic automorphism σ in the definition of strong visibility takes control of infinitely many orbits.

3.2 Representation theoretic meaning of S

Suppose S is taken as small as possible in the setting of Theorem 3.1. Then, we raise a conjecture on the relation between the multiplicity-free decomposition of a unitary representation in O(D, V) and the slice S for strongly visible actions as follows:

Conjecture 3.2. 1) The dimension S should not exceed the number of essentially independent parameter of irreducible representations that occur in the irreducible decomposition of a unitary representation π realized in the section space O(D, V).

2) For ‘non-degenerate’ representation π, these two numbers should coincide.

Here are some evidence:

1) S = {one point}.

In this case, any unitary representation realized in O(D, V) is irreducible if it is non-zero. Hence, Conjecture holds.

2) Suppose G/K is a Hermitian symmetric space and (G, H) is a semisimple symmetric pair. Then, the H-action on G/K is strongly visible with the slice S of dimension k := ℝ-rank G/H ([Ko06b]). On the other hand, the branching formula of the restriction of holomorphic discrete series representations of G to H contains k independent parameter (see [Ko97, Ko06d]).

3.3 Applications to concrete multiplicity-free theorems

Applications to various multiplicity-free theorems (e.g. those listed in Section 1) are studied in the papers [Ko04, Ko05a, Ko06d].

For example, the paper [Ko04] gives a new geometric explanation of the list of all pairs (π_{1}, π_{2}) of irreducible finite dimensional representations of GL(n) such that π_{1} ⊗π_{2} is multiplicity-free by geometric consideration (see also [Ko03]). (Such pairs were first classified by Stembridge [St01] by a combinatorial argument.)

The paper [Ko05a] collects a number of applications of Theorem 3.1 to multiplicity-free theorems, including Panyushev’s list of spherical nilpotent orbits of GL(n). Various aspects of multiplicity-free theorems for the restriction with respect to semisimple symmetric pairs are discussed in [Ko06d] as its main theme.

3.4 Explicit decomposition formulae

If a representation is known a priori multiplicity-free, one may be tempted to find its irreducible decomposition explicitly. As a matter of fact, such formulae often have a beautiful nature because of multiplicity-free property. Various new explicit multiplicity-free irreducible decompositions and their further analysis were found in the last decade. Here is a sample of the references: [A06, Bn02, BH98, DP01, Ko97, Ko06d, KoØ03, Kr98, Nr02, O98, ØZ97, Pz96, Pz05, Z01, Z02].

3.5 Classical limits — Orbit method

By the spirit of the Kirillov–Kostant orbit method, Theorem 3.1 predicts that ‘geometric multiplicity-free theorems’ should hold for the coadjoint orbits. The paper [KoN03] addresses this problem for semisimple symmetric pairs (G, H).

References

[A06] Alikawa H, Multiplicity free branching rules for outer automorphisms of simple Lie algebras, to appear in J. Math. Soc. Japan

[Bn02] Ben Saïd, S (2002) Weighted Bergman spaces on bounded symmetric domains. Pacific J. Math. 206:39–68

[BH98] Bertram W, Hilgert J (1998) Hardy spaces and analytic continuation of Bergman spaces. Bull. Soc. Math. France 126:435–482

[DH97] van Dijk G, Hille SC (1997) Canonical representations related to hyperbolic spaces. J. Funct. Anal. 147:109–139

[DP01] van Dijk G, Pevzner M (2001) Berezin kernels of tube domains. J. Funct. Anal. 181:189–208

[FT99] Faraut J, Thomas E (1999) Invariant Hilbert spaces of holomorphic functions. J. Lie Theory 9:383–402

[Ge50] Gelfand IM (1950) Spherical functions on symmetric spaces. Dokl. Akad. Nauk. SSSR 70:5–8 [Hi] Hilgert J, Reproducing kernels in representation theory, in prepration

[Ho89] Howe R (1989) Transcending classical invariant theory. J. Amer. Math. Soc. 2:535–552

[Ho95] Howe R (1995) Perspectives on invariant theory: Schur duality, multiplicityfree actions and beyond. Israel Math. Conf. Proc. 8:1–182

[Hu63] Hua LK (1963) Harmonic Analysis of Functions of Several Complex Variables in the Classical Domains. Amer. Math. Soc.

[JV79] Jakobsen HP, Vergne M (1979) Restrictions and expansions of holomorphic representations. J. Funct. Anal. 34:29–53

[Ko94] Kobayashi T (1994) Discrete decomposability of the restriction of A_{𝔮}(λ) with respect to reductive subgroups and its applications. Invent. Math. 117:181– 205

[Ko97] Kobayashi T (1997) Multiplicity free theorem in branching problems of unitary highest weight modules. Proceedings of the Symposium on Representation Theory held at Saga, Kyushu 1997, Mimachi K (ed) 9–17

[Ko98a] Kobayashi T (1998) Discrete decomposability of the restriction of A_{𝔮}(λ) with respect to reductive subgroups, Part II — micro-local analysis and asymptotic K-support. Ann. of Math. 147:709–729

[Ko98b] Kobayashi T (1998) Discrete decomposability of the restriction of A_{𝔮}(λ) with respect to reductive subgroups, Part III — restriction of Harish-Chandra modules and associated varieties. Invent. Math. 131:229–256

[Ko00] Kobayashi T (2000) Multiplicity-free restrictions of unitary highest weight modules for reductive symmetric pairs, Preprint Series 2000–1 of UTMS. 39 pages

[Ko02] Kobayashi T (2002) Branching problems of unitary representations. Proc. of I.C.M. 2002 at Beijing, vol. 2, Higher Ed. Press, Beijing, 615–627

[Ko03] Kobayashi T (2003) Multiplicity one theorem on branching laws and geometry of complex manifolds. Surikaiseki Kokyuroku, RIMS 1348:1–9 (in Japanese), Exposition of Lie Theory and New Advances (organized by S. Ariki).

[Ko04] Kobayashi T (2004) Geometry of multiplicity-free representations of GL(n), visible actions on flag varieties, and triunity. Acta Appl. Math. 81:129–146

[Ko05a] Kobayashi T (2005) Multiplicity-free representations and visible actions on complex manifolds. Publ. Res. Inst. Math. Sci. 41:497–549 [Ko05b] Kobayashi T (2005) Theory of discretely decomposable restrictions of uni-

tary representations of semisimple Lie groups and some applications. Sugaku Expositions 18:1–37, Amer. Math. Soc.

[Ko06a] Kobayashi T, Propagation of multiplicity-free property for holomorphic vector bundles, preprint.

[Ko06b] Kobayashi T, Visible actions on symmetric spaces, preprint.

[Ko06c] Kobayashi T, A generalized Cartan decomposition for the double coset space (U(n_{1}) × U(n_{2}) × U(n_{3}))\U(n)/(U(p) × U(q)), preprint.

[Ko06d] Kobayashi T, Multiplicity-free theorems of the restrictions of unitary highest weight modules with respect to reductive symmetric pairs, to appear in Progress in Math., T. Kobayashi, W. Schmid, and J.-H. Yang (eds), Birkhäuser

[Ko06e] Kobayashi T, Visible actions on flag varieties of SO(n) and multiplicity-free representations, in preparation.

[KoN03] Kobayashi T, Nasrin S (2003) Multiplicity one theorem in the orbit method. Lie Groups and Symmetric Spaces: In memory of F. I. Karpelevič, S. Gindikin (ed), Translation Series 2, Amer. Math. Soc. 210:161–169

[KoØ03] Kobayashi T, Ørsted B (2003) Analysis on the minimal representation of O(p, q). II. Branching laws. Adv. Math. 180:513–550

[Kr98] Krattenthaler C (1998) Identities for classical group characters of nearly rectangular shape, J. Algebra 209:1–61

[Nb97] Neeb K-H (1997) On some classes of multiplicity free representations. Manuscripta Math. 92:389–407

[Nr02] Neretin YA (2002) Plancherel formula for Berezin deformation of L^{2} on Riemannian symmetric space. J. Funct. Anal. 189:336–408

[O98] Okada S (1998) Applications of minor summation formulas to rectangularshaped representations of classical groups. J. Algebra 205:337–367

[ÓØ96] Ólafsson G, Ørsted B (1996) Generalizations of the Bargmann transform. Lie Theory and its application in physics (Clausthal, 1995) Dobrev and Döbner (eds) World Scientific 3–14

[ØZ97] Ørsted B, Zhang G (1997) Tensor products of analytic continuations of holomorphic discrete series. Canad. J. Math. 49:1224–1241

[Pz96] Pevsner M (1996) Espace de Bergman d’un semi-groupe complexe. C. R. Acad. Sci. Paris Sér. I Math. 322:635–640 [Pz05] Pevzner M (2005) Représentations des groupes de Lie conformes et quantifi-

cation des espaces symétriques. Habilitation, l’université de Reims, 36pp.

[PT02] Podestà F, Thorbergsson G (2002) Polar and coisotropic actions on Kähler manifolds. Trans. Amer. Math. Soc. 354:1759–1781

[Rp79] Repka J (1979) Tensor products of holomorphic discrete series representations. Canad. J. Math. 31:836–844

[Sc69] Schmid W (1969–70) Die Randwerte holomorphe Funktionen auf hermetisch symmetrischen Raumen. Invent. Math. 9:61–80

[St01] Stembridge JR (2001) Multiplicity-free products of Schur functions. Ann. Comb. 5:113–121

[Wo80] Wolf J (1980) Representations that remain irreducible on parabolic subgroups. Differential geometrical methods in mathematical physics (Proc. Conf., Aix-en-Provence/Salamanca, 1979), pp. 129–144, Lecture Notes in Math. 836, Springer

[Z01] Zhang G (2001) Tensor products of minimal holomorphic representations. Represent. Theory 5:164–190

[Z02] Zhang G (2002) Branching coefficients of holomorphic representations and Segal-Bargmann transform. J. Funct. Anal. 195:306–349