autoresearch is a deliberately tiny, single-GPU LLM pretraining setup built so that AI coding agents (like Claude or Codex) can run autonomous research experiments without human intervention. It is a simplified, cherry-picked subset of karpathy's nanochat project, reduced to three meaningful files: a fixed data-prep script, an editable training script, and a Markdown instruction file that humans tune to steer the agents.

The problem it addresses is the slow, manual iteration loop of neural network research. Traditionally a human researcher tweaks hyperparameters, architecture, and optimizer settings by hand. This repo reframes that loop: an AI agent edits train.py, runs a fixed 5-minute training experiment, measures validation bits-per-byte (val_bpb), keeps or discards the change, and repeats automatically — producing roughly 100 experiments overnight.

The intended users are ML practitioners and tinkerers with an NVIDIA GPU (the author tested on an H100) who want to experiment with agent-driven research. Instead of editing Python directly, the user programs the program.md file to define how the agent behaves, then spins up an AI agent in the repo with permissions disabled and prompts it to start running experiments.

Mechanically, prepare.py downloads parquet data shards from a HuggingFace dataset and trains a BPE tokenizer, caching results locally. train.py contains a full GPT implementation (rotary embeddings, grouped-query attention with FlashAttention-3 kernels, value residuals/ResFormer gating, Muon + AdamW optimizers, sliding-window attention patterns) wrapped in a training loop bounded by a fixed 300-second wall-clock budget. The fixed time budget makes architectural changes directly comparable on a given platform.

Notable patterns: the README is largely speculative narrative framing (a sci-fi vignette about future AI research) but the actual code is straightforward and well-scoped. One operational red flag worth noting is the documented instruction to run AI agents with 'all permissions disabled,' which grants an autonomous agent broad ability to modify and execute code on the user's machine — a deliberate design tradeoff the user should understand.