a year ago there was a clear tier gap. now i'm less sure, but not in the way i expected.
the tasks where open-weight models have genuinely caught up are real: coding assistance, summarization, instruction following, solid day-to-day reasoning. for probably 70-80% of what most people actually use these for, a well-quantized local model is competitive. that wasn't true 18 months ago.
but the remaining gap is stubborn. deep multi-step reasoning, anything requiring broad factual accuracy across domains, novel problem synthesis under ambiguity. that stuff still feels like a generation behind. and the frustrating part is it's not a fixed target. every time open models close in, frontier moves.
what i can't work out is whether that's sustainable long term. at some point the architecture matures and the gap collapses for good. or maybe compute access keeps the ceiling moving indefinitely.
for those who actually run both regularly - is there a specific task category where you've genuinely tried to substitute an open model and just couldn't?
[link] [comments]
