Cloud Theatre
A virtual reconstruction of a theatrical performance

Principal Contributors

  • Graphic model, animations, integration, & project coordination - David Poirier-Quinot
  • Geometrical acoustic model - Bart Postma
  • Project manager - Brian F.G. Katz
  • On-site support - Dominique Lemaire, Technical director, Athénée Théâtre Louis-Jouvet
  • Geometrical Acoustic simulations - All acoustical calculations were carried out using CATT-Acoustic™ v9
Athénée screenshot

This project was carried out in the context of the ANR-ECHO project concerning Digital Heritage and Historic Auralizations. From 2013 to 2017, this work was carried out at LIMSI. Since 2017, it is being led by the Sorbonne University, at the Institut d'Alembert, in the the group LAM.

Project Description

The project proposes to create realistic auralizations for applications in historic research in the field of the performing arts. The French ECHO project studies the use of voice in the recent history of theater. It is a multi-disciplinary project which combines the efforts of historians, theater scientists, and acousticians. In the scope of this project an audio-visual simulation was created which combines auralizations with visualizations of former configurations of the Théâtre de l'Athénée over a series of renovations, enabling researchers to realistically perceive theater performances in foregone rooms. Simulations include the room, 2 actors on stage, and an audience. To achieve these simulation, architectural plans were studied from archives providing various details of the different theater configurations, from which the corresponding visual and room acoustic geometrical acoustics (GA) models were created. The resulting simulations allow for 360° audio-visual presentations at various positions in the theater using commercial standard hardware.

Overview of the project

The Cloud Theatre virtual reconstructions comprise a number of elemetns, combined to cteate the final result.

  • Geometrical acoustic model of the theatre in is current state. This model is created and subsequently calibrated using on-site acoustic measurements following established procedures.
  • Modifications of the Geometrial Acoustic model to retroactively undo the series of rennovations carried out since the theatres construction.
  • Room impulse responses simulated using the Geometrical Acosutic model. These simulations were carried out using a composite source method which allows for the reconstruction of different source direcitivites as a function of frequency and dynamic source orientation without the need to re-run the entire simulation.
  • A Visual 3D model of the theatre, to provide a multimodal experience. This model is based on the actual theatre, but has been simplified. The overall impression of the room design is maintained but finish details such as moldings and sculputures have not been included, as the focus of the project is on the acoustics of the space.
  • Recordings of a theatical presenation. A theatrical piece was performed, with actors equipped with head-worm microphones to achiece a relatively dry audio, limiting the captured room response. This same performance was captured using a video and depth camera in order to create a visual representation of the actors in any virtual theatre construction. The video data further provides continuous position and orientation information for the actors, to be used in the processing of dynamic source directivity.
  • Audience presence. Audience ambient noise was created using studio recording of typical audience member mouevments, like shifting in chairs, head scratching, and coughing. A set of audience source poisitions were included in the simulation to provide realistic room impulse reponses for a distributed audience. For visual representation, wooden mannequins were placed throughout the seating area with associated animations used for typical mouvements. This addition provides a sense of realism to the performance, while not distracting from the actors, or requiring sophisticated human avatar modeling and animations.

Overview of technical details

The objective of the Cloud Theatre project is to create a realistic auralization of a theatre play in the Athénée Theatre (Paris). Two actors are recorded (audio and video) performing ''Ubu Roi, Act I'' in an anechoic environment. The audio recordings are used to create an auralization of the actors, using a 3D calibrated model of the Theatre (CATT Acoustics). The visual recordings, realised with a Kinect v2 (depth and RGB feeds), are used to create 3D point clouds of the actors, projected in the virtual environment. The position and orientation of actors' head is extracted from the video recordings and used to simulate head position and voice directivity in the auralization. A virtual audience is added to the theatre, simulated based on a procedural set of sounds and animations. The final 360° and flat renderings are created using Blender Cycles. The overall simulation is repeated for several spectator positions in the Theatre, the final material is used to study the impact of said position on visual and acoustic perception of the theatre play.

Integration of the model into Blender, texturing, compositing, lighting, and rendering was realised by David Poirier-Quinot. Some more discussion on the visual elements can be found on his webpage.

Example renderings

The 360° image renderings combined with the 3D Higher Order Ambisonic audio streams, were combined into a publicly accepsible format via the Facebook 360 platform. See all available videos on the project's Facebook playlist.

Cloud Theatre 2015 configuration - front row

Cloud Theatre original configuration - mid room

Cloud Theatre 2015 configuration - balcony


Associated works

Journal publications

  • B. F. G. Katz and M.-M. Mervant-Roux, “Comment entendre le passé ? Quelques leçons d’une collaboration de recherche entre acousticiens et spécialistes d’études théâtrales,” Revue Sciences/Lettres, no. 6, pp. 1–14, 2019, doi:10.4000/rsl.1645. SI: L’Écho du théâtre 2. English title: How to hear the past? Some lessons from a collaborative research between acousticians and theatre studies specialists.
  • B. N. Postma and B. F. Katz, “Perceptive and objective evaluation of calibrated room acoustic simulation auralizations,” J. Acoust. Soc. Am., vol. 140, pp. 4326–4337, Dec. 2016, doi:10.1121/1.4971422.
  • B. N. Postma and B. F. Katz, “Creation and calibration method of virtual acoustic models for historic auralizations,” Virtual Reality, vol. 19, no. SI: Spatial Sound, pp. 161–180, 2015, doi:10.1007/s10055-015-0275-3.

Conference proceedings

  • B. F. G. Katz, D. Poirier-Quinot, and B. N. J. Postma, “Virtual reconstructions of the Théâtre de l’Athénée for archeoacoustic study,” in Intl Cong on Acoustics (ICA), (Aachen), Sept. 2019, (url).
  • B. F. Katz, B. Postma, D. Thery, D. Poirier-Quinot, and P. Luizard, “Objective and perceptive evaluations of high-resolution room acoustic simulations and auralizations,” in Euronoise, (Crete), pp. 2107–2114, May 2018, (url).
  • D. Poirier-Quinot, B. N. Postma, and B. F. G. Katz, “Augmented auralization : Complimenting auralizations with immersive virtual reality technologies,” in Intl. Sym on Music and Room Acoustics (ISMRA), (La Plata), pp. 14:1–10, Sept. 2016, (url).

Contact

For any questions, please contact the head of the project: Brian F.G. Katz