Sony’s Beyond Sports Teams up with ESPN for First Real-time Animated Alternate Presentation of an NBA Game

Sony’s Beyond Sports is collaborating with ESPN, Disney, and the NBA for Dunk the Halls, the first-ever real-time, animated alternate presentation of an NBA game. The special Christmas Day alternate presentation of the New York Knicks and San Antonio Spurs game 12 p.m. ET, Wednesday, Dec. 25 will use data processing and real-time visualization technology from Beyond Sports, as well as optical tracking technology from Hawk-Eye Innovations, to insert iconic Disney characters into the game action using 3D animation, while the two teams play in real time.

Dunk the Halls is the third major sports telecast in 2024 to use Sony’s Beyond Sports’ real-time visualization engine, including the NHL Big City Greens Classic 2 on Mar. 9 and the NFL The Simpsons Funday Football on Dec. 9. The technology was also used for the three-time Sports Emmy Award-winning Toy Story Funday Football, featuring the Atlanta Falcons vs. Jacksonville Jaguars game in London in Oct. 2023.

The increasing popularity of these special presentations highlights the changing landscape of sports broadcasting and continuing shifts in audience behaviors. With heightened competition for viewers and advertisers from new content services and platforms, more broadcasters are creating new experiences that attract and retain new and existing viewers, including multi-generational audiences, in exciting and entertaining ways.

“Real-time visualization and virtual alternate productions have clearly emerged from being a novelty into an established tentpole of broadcast and streaming programming,” says Sander J. Schouten, Managing Director and Co-Founder at Beyond Sports. “Every time, we’re ready to deliver an innovative, immersive, and interactive altcast that puts viewers right in the heart of the action. This isn’t just the future of sports consumption—it’s the trend reshaping the industry today. With our latest telecast, fans can expect all of that, plus an added touch of Christmas magic.”

The Christmas Day telecast marks the first time that Beyond Sports technology will be used for basketball. The production will combine Beyond Sports’ data processing and real-time visualization technology with Sony’s Hawk-Eye Innovations’ optical tracking, allowing each Spurs and Knicks player to appear as a motion-enabled, animated player.

Beyond Sports developed a unique data processing engine to optimize basketball optical tracking data from Sony’s Hawk-Eye Innovations. By validating and enriching the data with advanced enhancement technology, they create more precise and lifelike animations. This process ensures the animations achieve a heightened level of realism and fluidity, making the content not only more visually compelling but also more engaging and immersive for end-viewers.

“Adapting our data enhancement system to these new sports formats was essential,” says Nicolaas Westerhof, CTO and Co-Founder at Beyond Sports “For example, the original ankle and toe data points can result in unnatural movements. By adding details like foot roll dynamics, we ensure smoother, more realistic visuals, significantly improving the viewer’s experience.”

In addition to planned visual elements and creative features like gingerbread referees, Beyond Sports’ virtual commentator technology will insert the ESPN announcers calling the game into the animated production. The virtual commentator technology uses a virtual reality headset to capture the real commentators’ upper body movements and facial expressions and create 3D animations. The technology brings the commentators closer to the action, enabling them to engage with the gameplay and audience like never before.

Beyond Sports’ technology has had a transformative impact on the sports production world for several years, supporting high-profile projects for major sports leagues worldwide.

Sony’s Beyond Sports’ technology makes it possible to combine multiple data sources, analyze, validate, enhance, translate to a 3D environment, and stream to a desired platform in real-time, with a delay of less than a second. Using virtual visualizations allow for the addition of more storytelling aspects to the broadcast and engaging wider audiences. This can be done through the addition of a new 3D environment or characters, educational or entertaining clips, replays from all angles, and live virtual commentators. Special visuals, events, and animations can be triggered either manually or through the data, adding additional storytelling layers.