Kakao has publicly released a technical report on their independently developed language model, ‘Kanana,’ by sharing the research outcomes on ArXiv. Additionally, in a bid to stimulate the domestic artificial intelligence (AI) ecosystem, they are distributing the ‘Kanana Nano 2.1B’ model as open-source on GitHub.
According to the technical report released by Kakao on the 27th, their large-scale language model, ‘Kanana Flag,’ completed its training by the end of last year. This achievement means Kakao has completed the entire lineup (Kanana Flag, Essence, Nano) of the ‘Kanana Language Model,’ which was unveiled at the ‘If Kakao 2024’ developer conference held last October.
The ‘Kanana Flag’ model has achieved top-tier performance in both Korean and English globally. Through the optimization of training resources, it has managed to reduce costs by over 50% compared to models of a similar size while simultaneously achieving state-of-the-art efficiency and performance.
Kakao applied various innovative training techniques to maximize the training efficiency of large language models (LLMs). This approach enabled them to efficiently develop a range of high-performance models from the lightweight ‘Kanana Nano 2.1B’ to the massive ‘Kanana Flag 32.5B,’ achieving less than half the training costs compared to global models of similar sizes.
Moving forward, Kakao plans to enhance the Kanana models by integrating the latest reinforcement learning and continual learning technologies to improve inference, mathematics, and coding capabilities. They also intend to advance alignment technologies to increase the accuracy of user request executions. This ongoing model refinement aims to support interaction in various forms, such as voice, image, and video, ultimately strengthening the technology’s competitiveness to add practical value to everyday life.
Simultaneously, Kakao has published the lightweight model ‘Kanana Nano 2.1B’ as open-source on GitHub. The base model, instruct model, and embedding model of ‘Kanana Nano 2.1B’ are available through the open-source community on GitHub.
The ‘Kanana Nano 2.1B’ model is suitable for use by researchers and developers, offering high performance in a compact size that can even operate in on-device environments. Despite its relatively small size, it boasts performance comparable to similar-sized global models. As mentioned in Kakao’s official tech blog last month, it has demonstrated outstanding results in handling both Korean and English, according to the company.
By releasing this model as open-source, Kakao aims to improve accessibility to AI technology and continuously support updates so researchers and developers can attempt various applications based on the model.
Kim Byung-hak, the leader of Kanana’s achievements at Kakao, stated, “We have efficiently secured a high-performance proprietary language model lineup capable of competing with global AI models such as Lama and Gemma, based on optimization and lightweight technology. We plan to keep developing practical and safe AI models focused on efficiency and performance, reinforcing AI competitiveness through ongoing technological innovation.”