Apple Eyes PrismML’s On-Device AI for the iPhone
Apple is in talks with the startup PrismML about incorporating their 27B AI model onto an iPhone.
Key Takeaways:
- Early Discussions: Apple is in early stages of conversation with PrismML, according to CEO Babak Hassibi, who spoke with CNBC.
- On-Device Compression: PrismML has developed technology to shrink large AI models, like Alibaba's Qwen, from 54GB down to as little as 3.9GB for use on an iPhone.
- Performance Trade-offs: While the compressed models are faster and more energy efficient, they sacrifice a few percentage points in performance compared to full-size models.
- Siri Integration: This move aligns with Apple's recent Siri overhaul in iOS 27, aiming to compete with OpenAI and Anthropic's AI assistants.
- Cost Savings: By running more AI on device, Apple can reduce cloud costs and bolster its privacy pitch. Additionally, smaller models could help offset projected increases in memory costs.
The Tech Behind PrismML:
- Bonsai 27B: PrismML released a compressed version of Alibaba's Qwen model, called Bonsai 27B, which runs on an iPhone 17 Pro.
- Memory Efficiency: They achieve this by reducing internal values from 16 bits to just one or three possible values, significantly cutting memory use while maintaining about 90% of full performance.
Analyst Insights:
- Caution Required: Analysts note that the effectiveness of PrismML's technology will be proven through real-world usage and extensive testing.
- Power Use Concerns: Phil Solis from IDC raises questions about power consumption, as constantly running a resource-intensive model could offset any cost savings.