Automating an experiment loop can increase the number of ideas tried. It also makes the separation between choosing a candidate and evaluating it independently more important.
The organizational changes are documented
Discovery Loop identifies Jeff Dean, Sanjay Ghemawat, Quoc Le and Oriol Vinyals as founders. It describes plans to automate experimental loops, initially in machine learning, then use those capabilities on its own technology stack. These are company objectives, not published evidence that a fully autonomous discovery system has achieved them.
Google’s August 5 message confirms its investor and Cloud-partner relationship. Demis Hassabis becomes DeepMind chair and Alphabet chief scientist; Koray Kavukcuoglu takes the SVP role overseeing DeepMind operations. The announcement does not promise a new Gemini API lifecycle or more reliable deprecation schedules.
Keep the evaluation outside the search loop
Our conceptual diagram starts with a candidate idea, runs an experiment, scores it on a development set and uses that feedback to choose the next candidate. That loop can be repeated. A separate held-out evaluation is reserved for the selected result.
If the supposedly independent test repeatedly feeds the next design decision, it has become part of the search process. Its score alone can no longer demonstrate performance on untouched data. This is a methodological boundary, not a claim that Discovery Loop currently uses the depicted implementation.
An automated system should preserve the candidate definition, code revision, data split, random seed and resource budget for each trial. Keeping unsuccessful trials also matters: publishing only the best run hides the number of opportunities the search had to find a favorable result.
What a useful first result would show
For a hypothetical model-training improvement, compare against the same baseline under a stated compute budget. Report variability across repeated runs, evaluation contamination controls and the cost of the entire search, not just the winning training run. A cheaper winning model may still have required an expensive discovery process; the two costs answer different questions.
The founders’ prior work motivates interest, but it is not a substitute for those artifacts. A public-benefit corporate form likewise does not establish that code, data or future models will be openly licensed. The concrete evidence to follow is a result with a reproducible method and clearly bounded claims, rather than an inferred promise about how the company will publish.
September 8: verify founders and Google roles from company statements, remove stock-price causation and API-roadmap speculation, and add a conceptual evaluation boundary for automated research.