We ran 18 experiments probing small language models (360M–1B parameters) with inputs ranging from random phonemes to Wierzbicka's universal semantic primitives.
The main finding: a consistent activation gap exists between what we term Layer 0a (scaffolding primitives: SOMEONE, TIME, PLACE) and Layer 0b (content primitives: FEAR, GRIEF, JOY, ANGER). The gap averaged +0.245 across all four tested architectures (Qwen 2.5, Gemma 3, LLaMA 3.2, SmolLM2) and was directionally consistent in every model.
Additionally, 11 pre-registered primitive compositions (operator + seed) matched predicted Layer 1 concepts in 3/4 models — e.g. WANT + GRIEF → longing/yearning, TIME + NOSTALGIA → memory/reminiscence, FEEL + GRIEF → heartbreak/sorrow.
The scaling pattern is the finding we're most uncertain about but find most interesting: the gap is largest in the smallest model and narrows as scale increases — not because content
primitives weaken but because larger models develop phenomenological access to scaffolding primitives too. This may partly explain capability jumps at scale.
All experiments are reproducible locally via Ollama. No API keys required. Code and data in the repo.
Paper: https://github.com/dchisholm125/graph-oriented-generation/blob/main/SRM_PAPER.md
Repo: https://github.com/dchisholm125/graph-oriented-generation
Limitations we're aware of: small n per primitive, the classifier is the same class of model being measured (circularity), and the mechanistic explanation is completely open. We're publishing preliminary findings, not definitive claims.
[link] [comments]
