[이데일리 한광범 기자] Artificial Intelligence (AI) language data specialist company Pelito announced on the 12th that it provided benchmark datasets for comparing and evaluating the performance of a Korean large language model on the ‘Open Ko-LLM Leaderboard.’
Open Ko-LLM Leaderboard is a performance evaluation platform for large-scale Korean language models (LLM) established and operated by the National Information Society Agency (NIA) and Upstage. The platform provides an environment for domestic companies and research institutions to register large-scale language models and compete in performance, thereby contributing to the development of Korean AI and natural language processing technology.
As a partner of Upstage, the operator of Open Ko-LLM Leaderboard, Pelito provided benchmark datasets in Korean that can evaluate common sense reasoning and contextual understanding abilities, mathematical inference and calculation abilities, and more. This enables more extensive performance testing comparisons beyond existing evaluation criteria such as reasoning ability, common sense ability, language understanding, hallucination prevention ability, and Korean common sense generation ability.
Pelito plans to accelerate the construction of high-quality language datasets for evaluating and improving the performance of Korean large language models following its participation in building this dataset. They are currently working with Upstage to construct benchmark datasets that can evaluate Korean language models applied in commercial fields and plan to release them through Open Ko-LLM by the end of the year.
Lee Jung-soo, CEO of Pelito, stated, “Through providing this benchmark dataset, the Korean large language model leaderboard will meet international evaluation standards, which is significant,” and added, “Based on our language data construction technology accumulated over many years, we will continue to contribute more to the development of the Korean AI ecosystem.”