Let me be direct: we’ve hit a wall with scaling, and the entire field is kind of bullshitting about what comes next. I’ve spent enough time in research circles to know this isn’t controversial, people just don’t say it publicly because there’s too much money involved.
Here’s the thing. Every major lab is operating under the same assumption: if we just throw enough compute at the problem, language models will eventually think. GPT-4 → GPT-5. Claude 3 → Claude 4. Llama keeps getting bigger. And yeah, there are improvements. But they’re getting marginal as hell, and nobody seems to want to talk about the ROI anymore.
We’ve spent the last three years making models that are incrementally better at pattern matching and retrieval. Revolutionary? No. Useful? Sure. A genuine step toward AGI? That’s where everyone’s lying to themselves.
The real problem is that scaling rewards the wrong things. You get better at predicting the next token, so you get better at autocomplete on steroids. You don’t necessarily get better at reasoning, planning, or handling novel problems. But those improvements are way harder to measure and fund, so… we just keep scaling.
Meanwhile, people are writing blog posts like “LLMs Have Achieved General Intelligence” after testing them on five cherry-picked examples. It’s embarrassing. It’s also lucrative, which is why nobody’s peer-reviewing this nonsense aggressively enough.
What would actually be useful:
• Research into modular architectures and compositional learning (unsexy, no massive compute requirements, hard to publish) • Better mechanistic understanding of what these models are actually doing (even harder to fund, requires careful experimental design) • Honest benchmarking instead of task-specific overfitting (kills your citations) • Actually proving that emergent abilities exist beyond statistical artifacts (lol good luck) What’s actually happening:
• More parameters • Bigger training sets (increasingly scraped into legal/ethical gray zones) • Flashier demos • Funding that goes to whoever can say “AGI” the most convincingly
Am I wrong? Probably not. Will anyone with skin in the game acknowledge this? Absolutely not. Too much money involved. Too many careers tied to “one more scaling paper.”
I’m not saying LLMs are useless. I use them. They’re tools. Good tools. But tools aren’t sentient, and we’re treating compute-heavy pattern matchers like they’re conscious because the alternative, admitting we’ve hit a local maximum, would tank stock prices and kill the hype cycle we’re all dependent on.
Five years from now, either we’ll have figured out something genuinely different (multimodal reasoning, world models, whatever), or we’ll all be very quietly accepting that the real breakthroughs require different approaches. And I’m putting money on the latter.
[link] [comments]
