SoftBank, Aizip Launch On-Device AI for Enterprise Privacy

Philip Lee -

CUPERTINO, CA / Tokyo, Japan - SoftBank Corp. and Aizip, Inc. have jointly developed a new Small Language Model (SLM) and Retrieval Augmented Generation (RAG) solution that operates locally on mobile devices and on-premises servers, the companies announced today.

The system, designed to address enterprise data privacy concerns, can be customized with domain-specific data and achieves accuracy comparable to 100 times larger cloud-based language models.

Testing on SoftBank's internal documents demonstrated that the system successfully answered 97% of employee questions when deployed as a mobile application running entirely on an iPhone 14. 

The solution's performance, evaluated across 22,000 questions, matched responses generated by a GPT-4-based RAG system.

Aizip's SLM customization pipeline incorporates data generation, fine-tuning, and evaluation processes that are executed iteratively until the target accuracy levels are reached. 

The development addresses reliability issues that have limited enterprise adoption of open-source Small Language Models.

According to the companies' June 2024 announcement, the technology's deployment capabilities extend beyond mobile devices to include on-premises servers, PCs, and IoT devices with microprocessors.

SoftBank vice president Katsuya Kitazawa, who heads the Information Technology & Architect Division, said the technology will benefit privacy-critical and offline-required use cases, including applications for flight attendants and field workers in remote locations.

The announcement comes as enterprise concerns about data privacy have led to restrictions on cloud-based language models. 

A Menlo Ventures survey indicated that 21% of failed AI pilots were attributed to data privacy challenges.

Aizip maintains its focus on developing production-grade AI solutions across its product lines, including Aizip Intelligent Audio, Vision, and Time-Series, emphasizing improvements in accuracy, reliability, speed, and development efficiency.