AI for Rapid Prototyping

When you know what you want, AI sucks. When you don't know what you want, AI is great.
AI can be useful for experimentation as long as the results don't need to be reliable.
And with the cost of writing crappy code being reduced, you can now perform more experiments faster. In problems where there is less time involved, AI performs better (2-hour projects). However, when given more time, humans perform better (both AI and humans get 32-hour time budgets) Problems involving more code favor humans, while fewer favor AI. Also, if you think this is a trend that's changing, and there is evidence of a so-called Moore's Law of AI that suggests AI can exponentially perform tasks that take longer and longer, then I have news for you. It's true, but it comes at the cost of precision. An AI can perform tasks that take longer and longer, but when you look at the steps it takes to get there, its assumptions are often wildly off, providing a rocky foundation on which to build.

Performance comparison showing optimal time allocations vary by agent type, with AI agents making faster initial progress but humans improving more rapidly with additional time. Source: Wijk et al. (2025). RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. METR.