Rising deepfake cases, stagnant police funding fuel public criticism over legal measures

Public calls for the government to strengthen its measures against deepfake sex crimes are snowballing amid criticism that its current measures and law enforcement practices are insufficient to deal with the severity of the problem.

Protestors hold placards condemning deepfake sex crimes in front of the Government Complex Seoul in Jongno District, central Seoul, on Friday. [YONHAP]

Public calls for the government to strengthen its measures against deepfake sex crimes are snowballing amid criticism that its current measures and law enforcement practices are insufficient to deal with the severity of the problem.

According to the National Police Agency, reports of sex crimes involving generative AI, including deepfake videos, increased by over 65 percent this year from January to July — amounting to 297 cases. The numbers have steadily risen in recent years, with 156 cases in 2021, 160 in 2022 and 180 last year.

When the police started its intensive crackdown last month, 118 cases were reported in the five days between Aug. 26 and 30. Of the 33 suspects reported, 31 turned out to be teenagers. Of the seven suspects arrested, six were teenagers.

In contrast, the Ministry of Gender Equality and Family's budget for combating online sex crimes next year has been slashed by 31.5 percent, amounting to 841 million won, in comparison to this year’s 1.23 billion won. Specifically, operating expenses for related personnel were cut by 40.5 percent, from 494 million won this year to 294 million won next year.

The budget for the Women's Human Rights Institute of Korea, a public institution under the Gender Ministry, was also reduced by 6.5 percent, totaling 13.7 billion won. The Advocacy Center for Online Sexual Abuse Victims (Acosav), run by the institute, receives reports of digital sex crimes, including deepfakes, and deletes the illicit material from the internet.

Number of deepfake crime cases and arrests [NATIONAL POLICE AGENCY]

Acosav has been employing 39 people, eight of them part-time, for the past four years. While employee numbers have stayed consistent, reports of illegally filmed content have increased by 156 percent, from approximately 156,000 cases to 243,000 cases.

The police's digital sex crime unit also lacks adequate personnel numbers to deal with the issue. According to reports from the police, a total of 131 personnel were distributed across 26 teams nationwide as of June 5. The digital sex crime unit was founded in December 2018.

Thirty-two more people have been employed by the police for the digital sex crime unit since 2019. Of the increased personnel, 62.5 percent were assigned to Seoul and Gyeonggi, while personnel numbers decreased in Sejong and Gangwon.

Vice Gender Minister Shin Young-sook answers questions at a meeting held by the Gender Equality and Family Committee at the National Assembly in western Seoul Wednesday. [KIM SEONG-RYONG]

The total number of digital sex crimes doubled during this period, increasing from 9,430 to 20,127.

“As online sex crimes are occurring nationwide, it is necessary to examine the trends in each region and increase the number of personnel accordingly,” said Kwak Dae-kyung, professor at the College of Police and Criminal Justice in Dongguk University.

There are limits to solving the rising trend of digital sex crimes through restrictions to digital platforms, such as Telegram, where the illicit deepfake materials are circulated.

Telegram's app logo is seen in this illustration taken in August 2024. [REUTERS]

Legislation — commonly and collectively referred to as the “Nth Room Prevention Act” — was passed in response to the 2019 “Nth Room” incident, where multiple chat rooms on Telegram were reportedly being used to sexually exploit women and young girls. The girls were coerced to make pornographic images and videos of themselves to distribute through Telegram.

The legislation requires domestic telecommunications companies with more than 100,000 daily users and 1 billion won in revenue to screen illegal content. If the companies fail to comply, they could be fined up to 3 percent of their total revenue.

The acts failed to regulate Telegram, however, since the company is based in Tortola, British Virgin Islands, with its operational center being located in Dubai, United Arab Emirates. Telegram does not have a representative that is based in Korea either, making it impossible to impose penalties with the current laws.

Telegram also operates as a platform for private chat rooms, making it exempt from the act’s regulatory scope of “information that is openly distributed to the public.”

Experts say that effective, focused regulatory measures for Telegram are required to counter deepfake sex crimes. “It is time for Korea to come up with a model law that other countries can follow as well,” said Lim Jong-in, presidential secretary for cybersecurity. Lim called for forcing Telegram to place a representative in Korea, and starting undercover investigations to counter deepfake sex crimes.

BY LEE YOUNG-KEUN [kim.minyoung5@joongang.co.kr]