Specifications face a basic tradeoff: leave details out, and important questions go unanswered; record every detail separately, and the specification becomes large and prolix.
We investigate whether a language model can serve as a compact, living specification oracle by learning facts about a target and answering questions about it directly.
We compare two ways of storing the learned facts: external text notes and changes to the model's weights.
Results Across Four Families of 596-Fact Worlds
Across four families of 596-fact worlds and two Qwen2.5 model sizes, weight-only oracles benefited substantially more from structure:
- With the 7B model, their accuracy integrated across storage capacities was 18.5 percentage points higher on structured than unstructured worlds, compared with 1.1 points for note-sheet oracles.
This advantage came at a substantial storage cost, with the smallest adapter requiring approximately 175 KiB compared with a maximum note budget of 16 KiB.
Weight Adaptation vs. External Notes
Adapted weights therefore exploited latent structure more successfully, while external notes required substantially less object-specific storage.