PANOPTES - Integrated Computational Plenoptic Imaging for Large-Volume High-Quality Immersive Visual Content
Building the foundations for immersive free-viewpoint experiences by capturing and reconstructing the full visual richness of the real world.
Towards truly immersive visual experiences
The PANOPTES project aims to overcome the scientific and technological barriers that currently prevent the widespread deployment of immersive visual content with six degrees of freedom (6DoF). Such experiences would allow users to freely move through recorded environments and observe scenes from arbitrary viewpoints, creating the sensation of truly being present inside captured real-world content.
At the heart of the project lies the concept of the plenoptic function, a complete description of all light rays traveling through a scene. PANOPTES investigates how this information can be efficiently captured, represented, modelled, and rendered using novel computational imaging approaches. By combining advanced signal processing techniques with state-of-the-art machine learning methods, the project seeks to establish a unified framework for large-scale dynamic light field acquisition and rendering.
The research focuses on three major challenges. First, PANOPTES develops multimodal sensing systems based on light field cameras, RGB cameras, and depth sensors that can efficiently capture complex dynamic environments. Second, the project investigates novel semantically enriched representations for dynamic light fields, enabling robust modelling even when captured data is incomplete or inconsistent. Third, the project explores real-time rendering and view synthesis techniques that generate photorealistic virtual viewpoints while preserving geometric and semantic consistency across views.
At the scientific level, PANOPTES advances the theory of computational plenoptic imaging by introducing new methods for acquisition, representation, modelling, and rendering of immersive visual content. At the technological level, it aims to establish kernel-based scene representations capable of supporting future immersive applications, including virtual reality, extended reality, telepresence, digital twins, and free-viewpoint video.
Through the creation of new datasets, algorithms, and evaluation frameworks, PANOPTES will contribute fundamental knowledge to the rapidly evolving domains of computational imaging, computer vision, and immersive media, helping to shape the next generation of visual communication technologies.
The consortium
The academic partners are:
- IDLab-MEDIA, Ghent University – imec
- ETRO, Vrije Universiteit Brussel
The project is coordinated by Peter Lambert (Ghent University – imec) and Adrian Munteanu (Vrije Universiteit Brussel), two research groups with long-standing expertise in multimedia signal processing, immersive media, light field technologies, computer vision, and machine learning.
For this project, we received funding through the FWO Senior Research Projects – Fundamental Research programme. The project will run for four years and will strengthen the collaboration between Ghent University and Vrije Universiteit Brussel while reinforcing Flanders’ international position in immersive visual communications and computational imaging research.
