On infinitely many foliations by caustics in strictly convex open billiards

Ergodic Theory and Dynamical Systems. 2024. Vol. 44. No. 5. P. 1418–1467.

Glutsyuk A.

Reflection in a strictly convex bounded planar billiard acts on the space of oriented lines and preserves a standard area form. A caustic is a curve C whose tangent lines are reflected by the billiard to lines tangent to C. The famous Birkhoff conjecture states that the only strictly convex billiards with a foliation by closed caustics near the boundary are ellipses. By Lazutkin’s theorem, there always exists a Cantor family of closed caustics approaching the boundary. In the present paper, we deal with an open billiard, whose boundary is a strictly convex embedded (non-closed) curve γ . We prove that there exists a domain U adjacent to γ from the convex side and a C∞-smooth foliation of U ∪ γ whose leaves are γ and (non-closed) caustics of the billiard. This generalizes a previous result by Melrose on the existence of a germ of foliation as above. We show that there exists a continuum of above foliations by caustics whose germs at each point in γ are pairwise different. We prove a more general version of this statement for γ being an (immersed) arc. It also applies to a billiard bounded by a closed strictly convex curve γ and yields infinitely many ‘immersed’ foliations by immersed caustics. For the proof of the above results, we state and prove their analogue for a special class of area-preserving maps generalizing billiard reflections: the so-called C∞-lifted strongly billiard-like maps. We also prove a series of results on conjugacy of billiard maps near the boundary for open curves of the above type.

Keywords: foliation, слоение, бильярды, billiards, caustic, каустика, symplectomorphisms, симплектоморфизмы