AI for Rapid Prototyping

When you know what you want, AI sucks. When you don't know what you want, AI is great.
AI can be useful for experimentation as long as the results don't need to be reliable.
And with the cost of writing mediocre code being reduced, you can now perform more experiments faster. In problems where there is less time involved, AI performs better (2-hour projects). However, when given more time, humans perform better (both AI and humans get 32-hour time budgets). Problems involving more code favor humans, while fewer favor AI. Also, if you think this is a trend that's changing, and there is evidence of a so-called Moore's Law of AI that suggests AI can exponentially perform tasks that take longer and longer, then I have news for you. It's true, but it comes at the cost of precision. An AI can perform tasks that take longer and longer, but when you look at the steps it takes to get there, its assumptions are often wildly off. These mistakes compound, providing a rocky foundation on which to build.

Fortunately, or unfortunately, depending on your viewpoint, there's enough work in the world that needs to be done, and letting AI take over the parts we can trust it to do well or is low risk to execute is going to be a boone for freeing up time for us humans to focus on the new, creative, intellectual, spiritually or otherwise stimulating challenges we want to focus on. So let's use AI to automate the boring stuff and keep for ourselves the work that requires creative thought and precision in executing a vision.

Performance comparison showing optimal time allocations vary by agent type, with AI agents making faster initial progress but humans improving more rapidly with additional time. Source: Wijk et al. (2025). RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. METR.