Naver’s HyperCLOVA X outperforms rivals in Korean AI test

2024. 2. 27. 11:45
글자크기 설정 파란원을 좌우로 움직이시면 글자크기가 변경 됩니다.

이 글자크기로 변경됩니다.

(예시) 가장 빠른 뉴스가 있고 다양한 정보, 쌍방향 소통이 숨쉬는 다음뉴스를 만나보세요. 다음뉴스는 국내외 주요이슈와 실시간 속보, 문화생활 및 다양한 분야의 뉴스를 입체적으로 전달하고 있습니다.

[Courtesy of Naver Cloud]
Naver Cloud announced on Tuesday that its HyperCLOVA X has achieved higher scores than OpenAI and Google’s generative artificial intelligence (AI) models in the South Korean AI performance evaluation system called KMMLU, Measuring Massive Multitask Language Understanding in Korean.

KMMLU is a performance evaluation metric established by the prominent open-source language model research team HAE-RAE in Korea.

The evaluation consists of 35,030 questions of expert-level knowledge across 45 domains such as humanities, social sciences, and science & technology.

About 80 percent of the questions are related to globally applicable extensive knowledge, including mathematical inference capabilities. The remaining 20 percent assess the model’s ability to solve Korea-specific problems, such as the geography of the Korean Peninsula and domestic laws.

The test questions are composed in Korean, allowing for a more accurate assessment of the AI’s understanding of the language and evaluating its universal capabilities along with local knowledge to provide a comprehensive judgment for Korean users, according to Naver.

[Courtesy of Naver Corp.]
According to the KMMLU research paper, HyperCLOVA X achieved higher scores than OpenAI’s GPT-3.5-Turbo and Google’s Gemini-Pro. It also outperformed OpenAI’s GPT-4 based on Korea-specific knowledge criteria.

Building on the proven performance competitiveness through KMMLU, Naver Cloud plans to further develop HyperCLOVA X into a Sovereign AI solution that has both security and performance.

Copyright © 매일경제 & mk.co.kr. 무단 전재, 재배포 및 AI학습 이용 금지

이 기사에 대해 어떻게 생각하시나요?