COURSE DETAILS

Before starting the Master's program, students will attend a Cinema Essentials Bootcamp (xx hours), designed to provide foundational knowledge of camera operation and cinematic practices relevant to virtual production.

Before starting the Master's program, students will attend a Computer Graphics Essentials Bootcamp (12 hours), designed to provide foundational knowledge of computer graphics and real-time 3D programming.

Course Overview: This foundational course is designed to introduce students to the basics of computer graphics and real-time 3D programming. Tailored for students with a background in mathematics and computer science, it assumes no prior experience in computer graphics, starting from scratch to build the essential knowledge required for the program's advanced courses.

Objectives: Students will gain an understanding of:

• The graphics pipeline and its role in rendering 3D scenes.
• The use of matrices for transformations in 3D space, including scaling, rotation, and translation.
• Fundamental concepts of real-time rendering (rasterization, ray-tracing, etc.).

Practical Work (TP): Hands-on sessions will reinforce theoretical concepts, with exercises that include:

• Modeling simple objects.
• Animating transformations in 3D.
• Generating and rendering procedural terrains using Perlin noise.
• Programming shaders on GPU.

Technology: The course uses a simplified custom graphics library to introduce core concepts, while simultaneously exposing students to OpenGL as a parallel industry-standard tool for real-time graphics programming.

This bootcamp serves as a comprehensive introduction and a stepping stone, preparing students to confidently tackle more advanced courses in the program.

This course will introduce the fundamental concepts for creating and analyzing shapes on the computer. We will start with generating and representing smooth curves in 2d using splines and Bézier curves. We will then move to various techniques for shape representation in 3d with special emphasis on triangle meshes and associated methods. At the same time, we will introduce methods for shape *analysis* and in particular defining and computing similarity between shapes, and shape matching (establishing correspondences between points on shapes). Topics will include:

• Polynomial, Spline and Hermite interpolation for 2D curves
• Bézier curves and the de Casteljau Subdivision Algorithm.
• Triangles meshes and Subdivision Surfaces
• Point cloud representation and processing algorithms
• Shape reconstruction from point cloud data.
• Shape Processing and Analysis – Simplification, segmentation, curvature and feature detection.
• Rigid Registration.
• Shape retrieval, non-rigid matching and correspondence.

Descriptif: This 3D Computer Animation course teaches the methods of animation and deformation of 3D shapes used in video games, special effects, and animated films, or more generally in animated and interactive virtual worlds.

Content: The course details geometric animation approaches as well as simulation methods for modeling physical phenomena. The course prepares students for a specialization in computer graphics, it prepares students to pursue technical development in R&D companies as well as research. The applications illustrated in the tutorials are mainly related to the field of entertainment (animated films, video games, virtual and/or augmented reality), or real-time simulation for modeling physical phenomena, but the underlying mechanisms can be applied in other disciplines (medical, biology, etc.). Examples of cases treated in the course and practical exercises: Implementing animation and interactive deformation of an articulated character, Modeling the deformation of a garment and managing collisions in real time, Simulating the surface of a moving fluid, Managing crowds of characters moving coherently, etc.

Organization: The course is largely practice-oriented with half lectures and half TD/TP. The TD/TP are done in the form of programming exercises in the machine room in C++ language, with OpenGL. A short project involving an animated virtual scene is carried out in the last sessions.

Syllabus:

• Geometric Animation
  • Kinematics: Procedural, keyframing
  • Geometric deformations
  • Articulated characters, Skinning
  • Production Pipeline (cinéma, VFX)
• Physically-Based Simulation
  • Simulation models (particles, rigid, continuum)
  • Elastic deformation, Cloth
  • Fluids (grids and particles)

Learning Objectives: Knowledge of the industry-standard methods for real-time and interactive animation methods for VFX, animation cinema, and video games, as well as research oriented approaches. Fundamentals of physically-based simulation as well as advanced particle-based methods. Being able to implement in C++/OpenGL several of these methods in an interactive context.

Descriptif: The goal of Computer Vision is to compute properties of the world from digital images. Problems in this field include recovering 3D shapes, estimating motion and object recognition, all through analysis of images and video. This course provides an introduction to image analysis and computer vision, including topics such as feature detection, motion estimation, image mosaics and 3D vision.

• Light, color and images
• Filtering, resampling, edges and corners
• Feature description and matching, Image Warping, Ransac, Panoramas
• 3D Vision : Stereo
• Deep Learning
• Dense Matching and MultiView Stereo
• Structure from Motion
• Image Based Rendering
• Graph Cut

These topics will be discussed in terms of algorithms and mathematical tools. The applications may be developed in python or C++.

Evaluation: 5 Labs + 1 project

Real-time 3D programming with OpenGL (4.6) and C++, programmable shaders (GLSL), GPGPU, etc.

Descriptif: Image synthesis a.k.a. “rendering”, is a central theme of 3D computer graphics which combines a set of artificial imaging methods to automatically generate digital images from virtual 3D scene models. Rendering is a transdisciplinary topic located at the interface between computer science, physics, applied mathematics and perception. It is used extensively in the areas of computer-aided design, virtual and augmented reality, visual special effects, digital animation, video games, simulation and architecture.

This course presents the principles, algorithms and techniques of image synthesis. It deals in particular with digital models of shape, appearance, lighting and sensors present in a 3D scene. The rendering equation, as well as standard illumination, shading and reflectance models are presented. Various rendering algorithms based on these models are detailed, including rasterization (projective rendering) and ray tracing. Real-time rendering, GPU programming and hierarchical spatial data structures are also covered. Finally, an opening towards global illumination concludes the course.

This course has a strong practical dimension, where students implement the models and algorithms throughout the quarter, using the C++ language and the OpenGL API (mandatory technical notions are recalled during the course). At the end of the course, students have acquired the ability to develop complete interactive 3D rendering systems as well as a detailed knowledge of the process of light transport simulation and digital image formation from a 3D scene.

All information about the courses can be accessed via this link: http://www.enseignement.polytechnique.fr/informatique/INF584/

Unreal Engine is a complete game engine and the world leader in Augmented Reality for real-time VFX on set for movies. This course focus on exploring the engine and its features for developing tools in research, game and AR applications.

A first part of the course will be dedicated to basic project creation such as plugin and game template. We will discover how a complete video game can be created using the Unreal Editor. Simple and concrete examples will be given all along the course, including materials, lighting, participating media (fire, water,…), animation and basic rendering features.

The second and major part will focus the blueprints which are a sort of graph system allowing to develop powerful features without a line of code. This is where the course will start investigating more complex projects especially using C++ and the photo-realistic rendering capabilities of the engine and how to get the best of it. We will enjoy together creating real-time VFX such as a simple green stage keyer and AR compositer for example, using the plugin management and development system.

Other practical studies will be achieved to show that UE can also be a powerful support to Research.

A final project will be asked to groups of students as the final evaluation of this course.

We have entered the Big Data Era. The explosion and profusion of available data in a wide range of application domains rise up new challenges and opportunities in a plethora of disciplines – ranging from science and engineering to business and society in general. A major challenge is how to take advantage of the unprecedented scale of data, in order to acquire further insights and knowledge for improving the quality of the offered services, and this is where Machine and Deep Learning comes in capitalizing on techniques and methodologies from data exploration (statistical profiling, visualization) aiming at identifying patterns, correlations, groupings, modeling and doing predictions. In recent years Deep learning is becoming a very important element for solving large scale prediction problems.

The Introduction to Machine and Deep Learning class will cover the following aspects:

• The Machine Learning Pipeline
• Unsupervised Learning
• Data Preprocessing and Exploration
• Feature Selection/Engineering & Dimensionality reduction
• Supervised Learning
• Deep and Reinforcement Learning

Detailed syllabus of the course: (minor changes may apply during the course progression.)

• General Introduction to Machine Learning
  • Machine Learning paradigms
  • The Machine Learning Pipeline
• Supervised Learning
  • Generative and non generative methods
  • Naive Bayes, KNN and regressions
  • Tree based methods
• Unsupervised Learning
  • Dimensionality reduction
  • Clustering
• Advanced Machine Learning Concepts
  • Regularization
  • Model selection
  • Feature selection
  • Ensemble Methods
• Kernels
  • Introduction to kernels
  • Support Vector Machines
• Neural Networks
  • Introduction to Neural Networks
  • Perceptrons and back-propagation
• Deep Learning I
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Applications
• Deep Learning II
  • Modern Natural Language Processing
  • Unsupervised Deep Learning
  • Embeddings, Auto-Encoders, Generative Adversarial Networks
• Machine & Deep Learning for Graphs
  • Graph Similarity
  • Graph Kernels
  • Node Embeddings

This course presents recent advances in 3D computer graphics, and more specifically in the subfields on modeling and animation, which rely on artificial intelligence. We first introduce user-centered Creative AI, i.e. smart 3D models - either based on knowledge or on deep learning from examples, designed to help users creating 3D virtual environments. Second, we focus the use of AI - from light models to deep reinforcement learning - in Character Animation, i.e. towards the training of autonomous 3D characters able to navigate and interact with such environments.

The lab sessions are held on Unity, based on C#.

3D Scene Capture and New Forms of Neural Scene Representations. NeRFs/GS are the new primitives of today. They are set to radically change the methods of capture, as well as the generation processes through AI. Cinema inevitably requires capturing reality, manipulating it, and transforming it with an artistic vision. This can include sets, objects, or characters (actors). This course unit includes:

1. Photogrammetry, reconstruction, and SLAM (Simultaneous Localization and Mapping) using traditional techniques.
2. Neural reconstructions of scenes or characters (NeRF, Mip-NeRF 360, iNeRF) as well as Gaussian Splats (3DGS, relighting, etc.).

This unit is at the heart of future challenges in cinema.

Augmented Reality is everywhere these days. You will see it in every single movie using VFX, whether real-time or not real-time. Recent TV shows such as The Mandalorian from Disney are mostly created using real-time AR technology. We will explore all the essentials and basic of AR together and all its essential scientific foundations.

AR is the mix of Computer Vision and Computer Graphics (CG) techniques, and all the bricks required to create a complete AR system will be described: camera tracking (estimating the position, orientation, zoom, focus, iris and distortion of a real camera/lens), compositing (mixing CG objects with real videos), depth keying (being able to handle occlusion between CG and real objects), lighting estimation (seamlessly integrating CG objects into real scenes using real lights, casting shadows from CG objects onto real objects and vice-versa).

The entire course will be illustrated with examples and making-off videos coming from famous movies.

The presenter of that course has lead for 8 years the technology of the #1 real-time AR technology used by all major movie company (Disney, Marvel, Lucasfilm, Sony, …) and has participated to more than 20 movies including Thor Ragnarok, Avengers 2, Edge of Tomorrow, Avatar 2, etc.

It is a unique journey that will be provided here. and both academic of industrial points of view of research in AR will be shared.

Motion capture is now essential across various fields, including 3D cinema, video games, healthcare, and sports.

After an introduction to different models of motion representation, this module provides a detailed overview of existing 3D motion capture systems. The goal is to give students a comprehensive understanding of the available technologies (active or passive optical sensors, magnetic, inertial, computer vision-based systems, etc.), covering both theoretical foundations and hands-on experiments in dedicated practical sessions.

Part of the course will also be dedicated to recent advancements in AI/deep learning methods and their applications in 3D pose estimation and the generation of animated virtual content.

Finally, to give students a full-scale experience, a motion capture mini-project under real-world conditions will be conducted in a professional studio. Students will have the opportunity to create their own scenario, carry out the necessary motion captures, and then transpose them into a virtual environment populated with 3D avatars, applying the required post-processing phases to ensure high-quality rendering.

Many films require the generation of imaginary landscapes and their animation in striking conditions, which can be a key element of storytelling: the action can take place against an impressive backdrop of cliffs and eroded mountains crossed by networks of rivers, lakes and waterfalls, within meadows and wind-blown forests, above a tumultuous ocean under a stormy sky, or even on flying rocks that do not actually exist, but which should still appear plausible… In addition to creating such settings, spectacular dynamic effects may be necessary, such as explosions, landslides, tornadoes, hurricanes, tsunamis and their impressive waves, volcanic ejections and lava flows, as well as their effects on surrounding buildings and vegetation. Rather than precise simulation methods, art directors and FX engineers are looking for effective, yet easily controllable, methods that provide both visual realism and artistic freedom.

This course will present and compare a variety of methods to achieve these goals, through the introduction of dedicated procedural models for the generation of natural scenes, effective simulation methods enabling their animation, and their combination with expressive user control to enable inverse procedural modeling - i.e. direct high-level control of the resulting scenes and animations. Finally, this framework will be compared to the combination of user control with modern machine learning techniques trained on simulation results, which recently emerged as a possible alternative for generating and controlling special effects.

Course description in progress.

This course explores film analysis from an algorithmic perspective, focusing on its application to the development of generative tools (e.g., camera trajectories, automated video editing). The objective is to extract key features from film sequences—such as shot scale, composition, focus, depth, and movement—and examine how this information can be utilized through simple generative approaches, database querying, similarity searches, and more.

The course includes extensive practical sessions, offering hands-on experience and fostering innovative ideas for leveraging extracted data.

In the second part, the course introduces applications involving drones, using Unreal Engine to simulate and program advanced camera movements in dynamic environments.

Subject to Change: This course introduces students to the techniques and skills essential for roles such as Technical Director and Creative Technologist—key positions that bridge the gap between artists, developers, and researchers. The focus is on procedural content creation and advanced programming techniques, including parallel and multi-thread programming, equipping students with a versatile skill set to innovate at the intersection of art, technology, and science. Tentative topics include:

• Procedural Generation:
  • Objective: Explore techniques to create complex visuals, models, and environments algorithmically rather than manually, leveraging mathematical and rule-based systems.
  • L-systems: Algorithmic models for plant generation and fractals.
  • Shaders: Techniques for visual effects, such as noise generation and inspirations from the Demoscene.

• Parallel and Multi-thread Programming:
  • Languages: Python, C++.
  • Techniques and Applications:
    • Introduction to multi-threading for optimizing concurrency within a process.
    • Parallel programming concepts for high-performance tasks.
  • GPGPU (General-purpose Computing on Graphics Processing Units):
    • Compute shaders using OpenGL to leverage GPU resources for creative and technical tasks.

• Extending Tools with Plugins:
  • Objective: Introduce students to the development of custom plugins to extend the functionality of industry-standard tools like Maya and Blender.
  • Skills Covered:
    • Understanding the plugin architecture of 3D applications.
    • Scripting and programming for tool customization (e.g., Python, C++).
    • Enhancing workflows by automating repetitive tasks and adding new features tailored to production needs.
  • Applications: Practical exercises include writing plugins for tasks such as procedural modeling, animation tools, or pipeline automation within Maya and Blender.

Example of Project: Create a unique, interactive video game experience that blends cutting-edge technology with creative design. Students could, for instance, develop a custom Mario Bros-style game from scratch using C++ and OpenGL, integrating innovative controls and visual effects. Possible features include:

• Interactive controls: Replace traditional gamepads with face gestures using machine learning (e.g., the Dlib C++ library) or custom touch-based controls via TouchOSC on a mobile device.
• 2D and 3D integration: Start with a 2D tile-based game, then enhance it with dynamic 3D elements for a hybrid gameplay experience.
• AI-driven maps: Use procedural generation or AI algorithms to design complex, playable maps, making each game session unique.
• Advanced visuals: Implement GPGPU post-processing effects for real-time visual enhancements, such as dynamic lighting, particle systems, or stylized filters inspired by the Demoscene.

By the end of the course, students will have gained hands-on experience in programming, procedural content creation, and tool customization, bridging the technical and artistic realms. They’ll leave with a portfolio-ready project showcasing their ability to innovate and push the boundaries of interactive media.

Course description in progress.

Subject to Change: This course equips students with advanced techniques in GPU programming, preparing them for pivotal roles such as Technical Director and Creative Technologist—key positions that bridge the gap between artists, developers, and researchers. Through hands-on projects and real-world applications, students will develop a versatile skill set to innovate at the intersection of art, technology, and science.

Tentative topics include:

• Introduction to Vulkan: Fundamentals of 3D real-time programming using Vulkan for high-performance graphics rendering.
• GPGPU Programming: Introduction to general-purpose computing on GPUs (GPGPU) using CUDA and OpenCL to leverage the power of parallel processing.
• Graphics Interoperability: Exploring the integration and interoperability of CUDA, OpenCL, OpenGL, and Vulkan to create seamless workflows and advanced visual applications.
• Using AI: Leveraging machine learning and deep learning techniques to enhance real-time rendering, animation, and other advanced visual applications.

During each academic period (3 months), students have the flexibility to choose the subject and technology they want to learn, progressing at their own pace, one half a week.

While students are free to select their focus, we recommend following an academic track that aligns with tools commonly used in the film industry, complements the science and technology courses taught during each period, which will enhance their Technical Director and Creative Technologist skills. Below is a suggested track:

• Period #1: Apply your scientific knowledge to modeling and animation using Maya and/or Blender.
• Period #3: Develop your real-time 3D skills with Unreal Engine.
• Period #5: Begin exploring VFX techniques with Houdini.
• Period #7: Dive into compositing workflows with Nuke.

Additionally, students interested in research can choose to analyze a scientific paper, reproduce its findings, and potentially propose innovative extensions. For those with an interest in game development, it’s also possible to dedicate several periods to creating a complete video game project.

Requirement: At the end of each period, students must submit a comprehensive report documenting their learning process, challenges, and outcomes.

P3 and P7 Specialized Seminar: During this seminar, students focus on language skills relevant to the film industry. The seminar includes tailored sessions on industry-specific terminology, communication techniques, and cultural insights. At the end of the seminar, students are required to write a report summarizing their learning and its practical applications in the context of cinema and audiovisual production.

Potential List of Sport Activities:

• Laser Run
• Swimming
• Boxing
• Climbing
• Afro Dancing
• Rowing
• Badminton
• Weight Training / Cross Training
• Horseback Riding (Gallop 4 minimum level required)
• Volleyball
• Muscle Strengthening
• Rugby MScT
• Football MScT
• Basketball MScT
• Mixed Martial Arts (MMA)
• etc.