Tokyo, Japan - Rakuten has introduced a set of large language models (LLMs) optimized for Japanese.
The base model, Rakuten AI 7B, and its instruction-tuned and chat-tuned variants have achieved top scores among open Japanese LLMs in the LM Evaluation Harness benchmark.
Developed using the Mistral-7B-v0.1 open model from French AI startup Mistral AI, Rakuten AI 7B is a 7-billion-parameter model with extensive pre-training on large-scale Japanese and English data.
The pre-training process was performed on Rakuten's proprietary multi-node GPU cluster, which enables efficient processing of complex datasets.
The models excel in performance due to pre-training on high-quality data, optimized efficiency using a proprietary Japanese morphological analyzer, and have received top ratings among open Japanese LLMs.
Ting Cai, Rakuten's Chief Data Officer, expressed the company's commitment to sharing the knowledge gained through developing Rakuten AI 7B with the open-source community, fostering the growth and development of Japanese LLMs.
Rakuten aims to leverage its rich data and cutting-edge AI technology to advance its "AI-nization" initiative and create new value for people worldwide.
