AI Model Costs and the Shift to In-House Models
AI model costs are pushing startups towards cheaper open weights, with many choosing to build their own models rather than rent from major providers.
The Situation at Harvey (discover more)
- Harvey's gross margin fell dramatically to minus 50% by June due to high AI agent usage.
- This margin improved significantly after they released their in-house model built on Moonshot’s Kimi K3.
- The company is now valued at $15.6B.
A Growing Trend
- Abridge, Decagon, Ramp, and Rogo are also developing their own models.
- Sequoia Capital and General Catalyst are backing this shift.
- This trend is also observed in the clinical model sector, where Abridge uses Nvidia’s open weights.
The Arguments for In-House Models
- Cost savings: Startups can avoid the high fees charged by renting models.
- Control: Building their own models gives startups more control over their technology.
- Performance: Custom models can be optimized for specific tasks, leading to better performance.
Challenges and Considerations
- Complexities: Training and maintaining models can be complex and resource-intensive.
- Leakage: There's a risk of model data leaking, which can compromise sensitive information.
- Regulatory: Compliance with data protection regulations like the EU's Data Act is crucial.
Perspectives
- Anthropic's View: Anthropic has noted the trend and highlighted the importance of Harvey still relying on their model for complex tasks.
- European Market: Europe has its own Harvey in Legora, a Stockholm-based startup that serves over 1,200 firms.
- Debate: Some, like Menlo Ventures' Matt Kraning, argue that having an in-house model is not always the best solution, describing it as "cosplay" in most cases.
Remember that the decision to develop or rent an AI model depends on various factors, including cost, performance requirements, and regulatory considerations.