S. Korean government provided 170M facial images obtained in immigration process to private AI developers

Biometric data collected at Incheon International Airport was handed over to the private sector without subjects' consent, a Ministry of Justice document has shown

Automated immigration clearance system at Terminal 2 of Incheon International Airport (Yonhap News)

The South Korean government handed over roughly 170 million photographs showing the faces of South Korean and foreign nationals to the private sector without their consent, ostensibly for the development of an artificial intelligence (AI) system to be used for screening people entering and leaving the country, it has been learned.

As a form of biometric data, faces are considered an example of especially sensitive information, the handling of which is difficult to regulate.

The government is tracking additional biometric data through hundreds of cameras positioned at immigration in Incheon International Airport. Many observers are accusing the government of crossing a line in its efforts to foster the AI industry.

Over 100 million pieces of biometric data handed to the private sector

According to materials received Wednesday by Democratic Party lawmaker Park Joo-min from the Ministry of Justice (MOJ) and Ministry of Science and ICT (MSIT), the two ministries have been pursuing an “AI identification and tracking system development project” based on an April 2019 memorandum of understanding, with a target completion date in 2022.

For this project, the MOJ has been transferring information obtained during the immigration screening process to the MSIT, including facial images of South Korean and foreign passengers, along with their nationality, gender and age. The MSIT has, in turn, transferred that information to private businesses for the purpose of AI technology research.

The cited objective was to develop a more advanced immigration screening system that would allow for passengers to be identified without passport scans and to detect potentially threatening situations before they occur.

While the government did mention the project in June 2019 in a press release following a meeting of relevant ministers presided over by Deputy Prime Minister and Minister of Economy and Finance Hong Nam-ki, it did not disclose details about the project’s structure, including the scope and methods of data collection and the parties responsible for its development.

Of the facial data transferred from the MOJ for use by private companies last year as part of this project, around 120 million images were of foreign nationals.

Companies used 100 million of these for “AI learning” and another 20 million for “algorithm testing.” The MOJ possessed over 200 million photographs showing the faces of approximately 90 million foreign nationals as of 2018, meaning that over half of them were used for learning.

The faces of arriving and departing South Korean travelers were also used for AI learning. In a document sent to Park, the MOJ explained that it had “used around 57.6 million units of facial data [from South Korean nationals] for the AI identification and tracking system project.”

While the Immigration Act does not allow the MOJ to collect or store biometric data such as facial images from South Korean nationals during immigration screening, it does store fingerprints and facial photographs from the South Koreans who have applied to use the automated immigration clearance service introduced in 2008. The MOJ explained that it was the facial images and gender and age data collected from South Koreans through this program that were transferred to the MSIT.

Collected with intent to make smarter AI and algorithms

It was also confirmed that video images of South Korean and international travelers passing through immigration at Incheon International Airport have been gathered as of this year for the purpose of acquiring “real data.” This suggests that the amount of facial image information passed along to private businesses could increase in the future.

In 2020 alone, the MOJ installed a total of 88 cameras around the Incheon Airport immigration area, including 50 fixed facial recognition cameras, 26 four-sided omnidirectional cameras, and 12 rotating cameras.

A project proposal drafted this June by the MSIT-affiliated National IT Industry Promotion Agency (NIPA) included plans for the addition of over 100 cameras in the immigration area and its glass booths. The government’s plan involves acquiring data from a total of 400 cameras for a project period from 2019 to 2022, or roughly 100 cameras per year.

The companies are working on developing a “one-to-many” matching algorithm that compares data of South Korean and foreign nationals registered with the MOJ to images of people taken with cameras at immigration. The system is being taught to identify people behaving “unusually” at immigration through comparison with previously registered data.

It’s a more complex system than the “one-to-one” matching currently used in automated immigration inspections, which checks whether the person in front of the camera is the same one identified in the passport photograph.

NIPA, the organization that drafted the proposal, explained that this involved “location tracking and photography of the faces of people arriving and departing.”

“A comparison between photographs taken by closed-circuit cameras and existing database images is used to confirm whether [the person in the photograph] is a registered international passenger, and the database is searched until the person who appears in the captured image is found,” the agency said.

“A violation of digital rights at an unprecedented scale”

Facial images are considered a particularly sensitive form of personal information that can be used to uniquely identify an individual.

South Korea’s Personal Information Protection Act defines “sensitive information” as information produced for the purpose of identifying a specific individual through their physical, physiological, and behavioral characteristics. Separate consent must be obtained from the subject of the information for it to be provided to or processed by a third party.

But the MOJ and MSIT have been using this information without the consent of travelers.

In a document sent to Park Joo-min’s office, the MOJ explained, “After a careful review that included legal counsel from a law firm, we determined that this project fell within the [original] scope of information collection purposes for immigration review.”

“The consent of the subjects of the information was not obtained,” it also said.

The Personal Information Protection Act also said that personal information may be used without the subject’s consent “within the scope reasonably related to the initial purpose of the collection.”

The MOJ concluded that the images of travelers’ faces were collected for the purpose of confirming their identity, and that they may also be used for the AI project, which has the purpose of developing more advanced traveler confirmation methods.

Civic groups and others have criticized the information transfer as a violation of digital rights at an unprecedented scale.

“Internationally, it is difficult to find any precedent of actual immigration data from domestic and international travelers being provided to companies and used for AI development without any notification or consent,” said Chang Yeo-Kyung, executive director of the Institute for Digital Rights.

“This is a shocking incident,” she added.

Lawmaker Park Joo-min said, “An AI identification and tracking system may be helpful to some extent in making immigration reviews easier and maintaining security within airports.”

“But if this project involved the handling of individuals’ sensitive information without their consent and without any special regulations to serve as its basis, then we need to immediately reexamine its legality.”

By Cheon Ho-sung, staff reporter

Please direct questions or comments to [english@hani.co.kr]