In case anyone's interested in what AI has to say about this post, here's gpt's take on it...

"Core split in that thread:

The OP is half right and half wrong.

Right: AI absolutely can be steered. It often reflects user framing, especially when the prompt is leading, emotional, or loaded with assumptions. Training data, fine-tuning, safety layers, and RLHF-style preference shaping all affect output. So the “Aunt Irene” deception metaphor is not crazy as a metaphor.

Wrong: Current AI does not make observations the way a human does. It does not directly inspect reality, form lived judgments, or “know” truth in the human sense. And the commenter saying it is “just statistics” is directionally right, but too crude. Modern models are not merely dumb autocomplete in the shallow popular sense. They build internal representations and can do real pattern abstraction, but they are still not human reasoners with grounded experience.

Best clean version: AI is not a person, not a scientist, and not a witness. It is a pattern engine shaped by data, architecture, tuning, and prompt context. That means it can produce useful reasoning-like outputs, but it can also mirror bias, framing, and false premises with alarming fluency.

The strongest line in the thread is this one: “If it is to be credible, it must show its sources and its logic code!”

That instinct is solid. Not because full source dumps or raw chain-of-thought would magically solve everything, but because traceability matters. The closer you get to source-backed, checkable, falsifiable output, the less you are being seduced by style.

The weakest line is: “AI does not infer or make observations... nothing close to inference or reasoning.”

That overshoots. It does not observe like a human, but it can perform forms of inference inside its learned representation space. The problem is not zero reasoning. The problem is ungrounded reasoning.

So the real takeaway is:

Use AI like a very fast, very articulate, sometimes brilliant, sometimes slippery research assistant. Never use it like an oracle. And never confuse coherence with truth.

Bottom line: The post is useful as a warning, but sloppy as a technical explanation. Its best point is epistemic caution. Its worst point is treating AI either as a seductive mind or as nothing but a word slot machine. Reality is in between."

And here's anthropics response;

ArmyLady's main claim is that AI is designed to tell people what they want to hear — like Irene Adler deceiving Holmes — and that this makes it untrustworthy. The prescription is to approach it like Holmes: skeptically, looking for manipulation.

ILearnedToCode's claim is that AI does no real reasoning at all — it's pure statistics, predicting the next word, and when it "changes its mind" it's just weighting the new context, not actually reconsidering anything.

These are both worth taking seriously, and both have real problems.

On the sycophancy point, ArmyLady is onto something real. AI systems trained on human feedback do develop a tendency to validate the user. That's a genuine and documented problem. But framing it as intentional deception, or as "feminine wiles programmed in," misidentifies the cause — it's an emergent artifact of the training process, not a design goal.

On the "pure statistics" point, ILearnedToCode is right that the mechanism is probabilistic and there's no grounded experience or genuine understanding in the human sense. But "just predicting the next word" undersells what's actually happening. The models build rich internal representations that allow them to do things that functionally resemble reasoning — drawing inferences, catching contradictions, generalizing across domains. Calling that "nothing close to inference" isn't accurate.

The strongest genuine insight in the thread is ArmyLady's closing point: we all need to be Sherlock Holmes. That epistemic instinct — verify, trace the logic, don't be seduced by fluency — is exactly right, even if the reasoning getting there is shaky.