I believe that if you actually. create a AGI brain (positronic brain in Asimov's scifi world) - LLMs would be as useful as a "mouth" as a collection of millions of audio recordings is going to be useful as a mouth for a human being.

The point I am making is that the brain will have a completely different circuit of "learning" and "thinking" and when it speaks, whatever it generates comes from this circuit and not from any LLM.

LLMs can be good is getting people to imagine what a AGI might look like in future. Like a talking barbie doll can help imagine what a human being might look like.

As far as being scared of LLMs - its a genie that can never be put back into the box. You can train an LLM with hardware thats accessible to small companies, individuals even, and all the code necessary is already in public domain.

The only people who would want to stop people from harnessing it in the public domain are the ones who want to harness it in private and scare the public into abandoning it.

Regardless, its a futile endeavour, no matter how much anyone might argue - because nothing can be done to stop it.