I don't know about how you guys feel but even before LLM started, many papers are already leaning on empirical findings, architecture designs, and some changes to loss functions. Not that these does not need math, but I think part of the community has moved away from math heavy era. There are still areas focusing on hard math like reinforcement learning, optimization, etc.
And after LLM, many papers are just pipeline of existing systems, which has barely any math.
What is your thought on this trend?
Edit: my thoughts: I think math is important to the theory part but the field moving away from pure theory to more empirical is a good thing as it means the field is more applicable in real life. I do think a lot of people are over stating how much math is in current ML system though.
[link] [comments]
