Gemini 4 Argon Matches GPT-6 Astra in Early Tests
Gemini 4 Argon uses more than twice as many tokens per task as GPT-6 Astra, but lower token prices keep it cheaper for now. Some Google staff doubt its coding abilities, according to Bloomberg.
Key Findings:
- Performance:
- Argon achieved a score of 53 on the Artificial Analysis Intelligence Index, tying with GPT-6 Astra and outperforming GPT-6.1 Sol by one point.
- It had the lowest hallucination rate (15%) of any model scoring 45 or more on the index.
- Ranked first on AutomationBench-AA with a score of 78%.
- Token Usage:
- Used approximately 62,000 output tokens per task, compared to GPT-6 Astra's 27,000.
- Despite lower token costs due to launch discount, Argon is still 2.7 times more expensive per task than GPT-6.1 Sol.
- Google's Response:
- Offering a $2 per million input tokens and $10 per million output tokens pricing tier for at least one month.
- Emphasizing the model's performance gains, particularly in text ranking and WebDev.
- Internal Concerns:
- Some Google employees question Argon's coding capabilities and front-end design, citing Bloomberg's report.
- Express doubts about "benchmaxxing" - chasing test scores over practical application.
Other Notable Mentions:
- Argon is the first Gemini model to top the Vals Index at 68.9%.
- It also leads Arena’s Text Arena ranking, with 1,525 points.