Virtual humans steal the show at Seoul basketball game

Virtual humans took center stage at a basketball game in Seoul, Korea. Created through a project by Onmind and SK Square, the virtual characters resembled the players and coaches and entertained fans during the game with dances and cheers.

A digital twin modeled after Seoul SK Knights' Kim Sun-hyung is projected on the roof of Jamsil Students' Gymnasium in Songpa District, southern Seoul, on Sunday. [ONMIND]

It seemed like a normal Sunday afternoon at the Jamsil Students’ Gymnasium in southern Seoul, the basketball court brimming with the palpable tension of excited crowds and players before the start of the second game of the 2023-24 KBL season between Seoul SK Knights and Suwon KT Sonicboom.

Everything was the same as before, but with one obvious distinction: Virtual humans also took center stage at this game. Before the actual game started, a two-minute video featuring virtual twins modeled after the 22 players and coaches including Heo Il-young, Kim Sun-hyung, Jameel Warney and Leon Williams, filled the roof of the stadium, The twins were portrayed as knights in shining armor, their faces filled with staunch commitment to defeat their opponents represented as ogres.

Another virtual human named Sua, also dressed in red and white — the two team colors of the Seoul SK Knights — filled the stadium’s display screens between the game quarters as she danced alongside human cheerleaders to cheer for the team.

From their physical appearance to their movements, everything about the virtual humans felt natural. Any SK Knights fan would have been able to immediately distinguish who the virtual characters were based on, out of the 22 team members.

This realization is the result of a two-month-long project by virtual human production company Onmind, which SK Square, an investment subsidiary of SK Group, has been in partnership with since 2021.

The introduction video was created under a medieval theme where the players were recreated as knights — derived from the team name — who are on a mission to defeat their deadly opponents.

Digital twins of Chun Hee-chul, the head coach of the Seoul SK Knights and professional basketball player Kim Sun-hyung generated using Onmind's technology. [ONMIND]

“This project was sparked by an idea between SK Square and Onmind to create a video consisting of the players’ digital twins, utilizing Onmind’s technology and equipment,” said Onmind’s CEO Kim Hyung-il at a press event to introduce the company’s virtual technologies at Jamsil Students’ Stadium on Sunday. “The biggest obstacle was the time limit of two months. Given that it took about a month to capture the virtual twins’ motions, technically we had only a month to fully create all 22 players as virtual characters.”

Conventionally, it takes three months to a maximum of six months to create virtual humans, according to Onmind. However, Kim says that the project was possible despite the tight time constraint because Onmind’s studio is fully prepped with in-house technology and equipment needed to bring about virtual humans to life, therefore making it possible to fully generate the characters without outside assistance.

The vivid portrayal of the players was possible through a technology dubbed photogrammetry, a type of mapping of physical objects or environments to create 3D models.

The players sat on a stool in Onmind’s studio for 20 minutes, enveloped by 129 cameras and 50 lighting equipment, which would map each player’s facial features from shapes and colors to skin textures for high-resolution resemblance.

The technology that Onmind fronts is real-time rendering, the process of generating and displaying computer-generated graphics at a seamless speed.

“Real-time rendering is much more cost effective compared to the existing method, which is referred to as offline rendering,” Kim said. “To create a single frame of video, a 3D model is first made, and then rendering is applied frame by frame. As the length of the video increases, the cost increases exponentially using this method. With real-time rendering, the video length does not typically matter because once the settings are set, the technology enables smooth rendering without the added costs.”

BY LEE JAE-LIM [lee.jaelim@joongang.co.kr]