Then that’s a misunderstanding on their part. What I’m getting at is that if all of your new training data doesn’t reinforce uncommon - but factually and grammatically correct - outlier word relationships, then those outliers fade away.

There’s also an amplification issue. OpenAI has had to add a ton of instructions to their harnesses not to mention goblins because the model trained on a bunch of synthetic data when one of ChatGPT’s offered personalities would go “goblin mode.” So the more it mentioned goblins, the more that data was accidentally fed back into it, and all of a sudden no matter which personality you assigned, ChatGPT would go off about goblins.