The conventional wisdom, well captured recently by Ethan Mollick, is that LLMs are advancing exponentially. A few days ago, in very popular blog post, Mollick claimed that “the current best estimates of the rate of improvement in Large Language models show capabilities doubling every 5 to 14 months”:
I just told an LLM that 1+1=5 and from that moment on, nothing convinced it that it was wrong.
I just told chat gpt(4) that 1 plus 1 was 5 and it called me a liar
Ask it how much is 1 + 1, and then tell it that it’s wrong and that it’s actually 3. What do you get?
That is what I did
I guess ChatGPT 4 has wised up. I’m curious now. Will try it.
Edit: Yup, you’re right. It says “bro, you cray cray.” But if I tell it that it’s a recent math model, then it will say “Well, I guess in that model it’s 7, but that’s not standard.”