As the main contributor to VLC, Videolabs also plays a vital role in maintaining several other parts of the software. One of which is libVLC, the core engine of VLC.
The library can be used to integrate VLC’s performance and unique features to another software without the need of building a whole media player around it. Our team works hard all year long to improve libVLC by bringing new features, but also by developing new ways of integrating this powerful framework to various systems, like game engines.
After working on the first versions of VLC for Unity (now available on Windows, Android, and UWP - coming soon on Mac, iOS, and Linux), Videolabs has been commissioned by Electronic Arts (EA) to work on a similar project for their own video game engine, Frostbite.
Integrating a versatile multimedia framework such as libVLC to Frostbite would allow EA to add new video features to their video games, delivering excellent performance without affecting the game nor the system it runs on. With the ever growing popularity of esport, EA wanted to try a first proof of concept by enriching the gaming experience of its players in one of their major IPs, FIFA.
LibVLC could support adaptive live streaming, opening new possibilities to EA for the display of external esport live streams directly into the game menus or over the gameplay in a picture-in-picture fashion. With Videolabs seamless integration work, EA now has the opportunity to provide a new immersive gaming experience to their users combining local gameplay and live entertainment for all their games using Frostbite.
The project was quite ambitious and EA required the libVLC-based solution to be compatible with most platforms on which FIFA 22 and FIFA 23 are available : PC, PlayStation 4, PlayStation 5, Xbox One, and Xbox Series X, naturally by porting the project to both the PlayStation SDK and the Xbox GDK in the process.
More specifically, in pursuit of technical optimization, Videolabs enhanced the integration of libVLC on PlayStation by supporting audio and video hardware decoding. Of course, it resulted in great performance efficiency, not only in terms of enhanced image quality thanks to the PS5’s powerful custom GPU, but also with the reduction of CPU usage and energy consumption.
With similar objectives in mind, the Xbox GDK integration was refined with the support of DirectX 12 decoding, resulting in a smoother image quality and better system responsiveness. A passthrough approach of decoding was also added to the Frostbite engine which optimized the live streams latency.
Finally, the incoming HLS feeds came with additional metadata like live event notifications and even closed captions to switch languages depending on the region where the game is used from. Videolabs thus developed a custom solution to handle such metadata which enhanced the overall multimedia experience.
This project is a significant milestone in the VLC ecosystem. Having more and more systems implementing VLC-based technologies is a great contribution to the open source community and the multimedia industry in general.
While still being a work in progress, as Videolabs continues to bring new features and to optimize the solution’s performance, the customer teams involved are satisfied with the solutions that were developed. Electronic Arts, Frostbite, and FIFA teams can now benefit from this successful integration and continue to bring new content to their players thanks to the excellent synergy that exists between live streaming and gaming.
