Kakao Releases Open Source AI Models Including Korea's First MoE

Seoul, South Korea - Kakao Corp. released two artificial intelligence models as open-source on Thursday, the company announced.

The company published the models, named Kanana-1.5-v-3b and Kanana-1.5-15.7b-a3b, on Hugging Face. 

The release occurred approximately two months after Kakao released four language models in May.

Kanana-1.5-v-3b is a multimodal language model with 3 billion parameters that processes text and image inputs. The company said it developed the model using its existing Kanana 1.5 model as a foundation.

In Korean language benchmarks, the model achieved higher scores than domestic and international public models of similar size, according to Kakao. 

The company stated that the model reached 128 percent of the performance level of similar-scale domestic multimodal language models in instruction-following capability tests.

Kanana-1.5-15.7b-a3b uses a Mixture of Experts architecture with 15.7 billion total parameters. 

During inference, approximately 3 billion parameters are activated, the company said.

Kakao developed the MoE model by applying an "upcycling" method to its existing 3 billion parameter model Kanana-Nano-1.5-3B. 

The process converts existing multi-layer perceptron layers into multiple expert layers.

The company said the model matches or exceeds the performance of its Kanana-1.5-8B model while using fewer active parameters.

The company applied knowledge distillation and human preference alignment training to both models. Knowledge distillation involves training smaller models using larger models to achieve improved performance in compact architectures.

Kakao stated that the multimodal model can be utilized for image and text recognition, content generation, chart interpretation, and mathematical problem-solving.

Both models are released under the Apache 2.0 license, allowing commercial use.

The company stated that this allows researchers and startups to experiment with and deploy the models.

Kakao participates in the government's Independent AI Foundation Model Project.

The company said it will utilize its model development capabilities and experience operating large-scale services, including KakaoTalk, to contribute to the project.

Kim Byeong-hak, Kanana Performance Leader at Kakao, described the open source release as a significant step in technological development, specifically citing improvements in cost efficiency and performance.

Kakao stated that it plans to continue releasing models developed with proprietary technology and intends to develop large-scale models by expanding its scaling efforts.

Kakao said it will release performance results for reasoning models during the second half of the year.

The company has released AI models since introducing its Kanana lineup in 2024, with previous open-source releases including models published in February and May.