M
Ask HN
Ask HN: Will transformer based LLMs hit an improvement ceiling?
A fundamental question I have is whether the current transformer based architectures are powerful enough to allow scaling towards AGI or are there ceilings here that are becoming more evident?

Though folks from the leading LLM companies are in the forefront of this, it’s hard to take their word due to an obvious conflict of interest. And it’s unclear if academia has more theoretical knowledge here as it seems these commercial companies are also leading the research work here.

What do you expect from AGI? What aspects of LLM have impressed you the most?
AGI is a vague term but a naive definition is smarter than any human in any fields/domain.

Clearly current LLMs are impressive in document/code generation but I’m having a hard time extrapolating that to AGI (assuming that it’s still reliant on current transformers arch).