Upstage’s LLM tops global open-source AI evaluation

An image of Upstage's CI

South Korea’s artificial intelligence startup Upstage said Thursday it has secured the top position in the global evaluation of large language models.

The company’s 30-billion-parameter LLM scored 67 points, ahead of Meta’s LLaMa-2 (66.8 points) on the Open LLM Leaderboard operated by the AI platform, Hugging Face, as of Wednesday.

Hugging Face’s leaderboard evaluates LLMs with less than 100 billion parameters on four key indicators: model’s reasoning challenge, common sense inference, contextual understanding and factual accuracy. It averages scores to determine the global ranking.

Despite the smaller LLM tool, in terms of parameters in size, the Korean firm recorded about 10 percent higher scores in performance than LLMs developed by global big tech firms such as Microsoft, Stability AI and Databricks, according to Upstage officials.

It also received a score of 56.5 points for its ability to mitigate AI hallucinations, surpassing Meta's score of 52.8 points.

“Using search is key to preventing hallucination,” an Upstage official said. “We are supplementing the basis for answers with vector search.”

The company said it developed the small but powerful model in just two months. The latest achievement marked a crucial turning point in establishing itself as a front-runner in the rising private AI market globally.

Upstage plans to increase the LLM’s Korean language performance and train more data to meet domestic customers’ needs.

“We will not only focus on enhancing the performance of the Korean language but also strive to become a global leader in the emerging private AI market for businesses,” said Kim Sung-hoon, the founder and CEO of Upstage.

By Jie Ye-eun(yeeun@heraldcorp.com)