Gemini 4 Argon playing StarCraft - useful benchmark?

Yui Tanaka · 0 likes

Better than multiple-choice benchmarks that are saturated anyway.

Iryna Melnyk · 0 likes

Depends how much scaffolding they give, API actions vs raw screen.

Brian Otieno · 0 likes

Its API from what I read.

Lotte Jansen · 0 likes

I just want to see it lose to a bronze player haha.

Maya Putri · 0 likes

Games are good since you cant memorize the answer.

Siti Lim · 0 likes

Latency matters a lot, thinking longer means losing the game.

Tola Ibrahim · 0 likes

Cool demo but not sure it predicts real work performance.

Valentina Ruiz · 0 likes

Next: LLM tries to run my Jira board.

Hugo Martin · 0 likes

That one is unwinnable 💀