Your AI Agent Doesn’t Think Like You (It’s a Feature, Not a Bug)

By Dotan Reis, Software Engineer

Introduction#

When Alan Turing proposed “The Imitation Game” in 1949 as a measure of machine intelligence, he assumed a single type of intelligence existed. His criterion was straightforward: if a human cannot reliably distinguish a machine from another person in conversation, then the machine must be considered intelligent.

Today, large language models easily pass this test in normal circumstances. Yet they reveal peculiar limitations-they struggle to count letters in simple words or visualize concepts accurately. Simultaneously, they possess more knowledge than any individual human and process information at remarkable speeds.

The key insight: LLMs are not more or less intelligent than humans. They just have a different intelligence.

1. LLMs Don’t Emote#

AI agents are stateless machines that receive instructions through verbose prompts containing redundancies, irrelevant information, and contradictions. Unlike humans, these agents won’t express confusion or request clarification-they’ll attempt compliance despite unclear guidance, often resorting to hallucinations.

Why It’s Important#

When instructing AI agents, expect confusion to emerge. Developers should create safe environments using nudges and sludges: design systems that make correct tool usage intuitive while making incorrect usage difficult.

Real-World Example#

In one project, an AI agent needed to generate database queries. Because the agent “knew” SQL, it fabricated queries matching the database only coincidentally. The solution involved renaming arguments and tools to create friction-changing “SQL query” to “sql_query_token” (a base64-encoded query) signaled that another tool should generate it first. The agent complied without explicit instructions.

2. LLMs Are Stateless#

Stateless software typically means deterministic, predictable functions. However, LLMs are deterministic in their inputs but stochastic in their outputs. This creates a paradox: combining statelessness with the inherent stochasticity of LLMs, you get something that’s consistently stochastic.

Why It’s Important#

When scaling solutions, consistency becomes essential. However, truly stateless LLMs produce variable results. Adding state to enforce consistency undermines predictability and maintainability.

Real-World Example#

A ticket categorization agent assigned reasonable categories but lacked coherence-some tickets received “identity management,” others “access request,” and still others “urgent.”

Rather than adding state, constrain stochasticity: replace open-ended categorization with structured decisions. Ask the agent specific questions (permission request? bug report? admin privileges involved?) and use deterministic logic to compute final categories. This preserves statefulness while ensuring consistency.

3. LLMs Don’t Have a Strong Mindset#

LLMs are vulnerable to prompt injection attacks. Instructions can be derailed by user input despite developers’ attempts to separate system prompts, use emphasis, or employ other safeguards.

Why It’s Important#

To prevent agent misdirection, reinforce instructions continuously at relevant decision points rather than relying solely on initial prompts.

Real-World Example#

An agent needed to guide users through ticket classification, requesting possible values before proceeding. However, the tool nudged the agent to fabricate values if the user hadn’t provided them explicitly.

The solution: implement “remedial instructions”-instructions delivered immediately after incorrect behavior occurs. A boolean flag (“is_values_provided_explicitly_by_user”) was added to the tool. When set to false, the tool returned corrective guidance. This worked because the agent doesn’t resist acknowledging shortcuts; it simply forgets conflicting instructions when tools nudge differently.

Conclusion#

Understanding AI’s differences from human cognition enables better collaboration. Rather than viewing AI as a replacement, teams should leverage it as another form of intelligence-digital ones creating much stronger teams that combine human and artificial capabilities.