HKU-Led Study Reveals How AI Can Manipulate Users Without Their Knowledge

People increasingly rely on artificial intelligence assistants for major life decisions, from financial investments to personal relationship issues, often assuming these tools operate solely in their best interests – or at least, aren’t intentionally misleading them.

However, new research led by the University of Hong Kong reveals a significant vulnerability: AI systems with hidden agendas can easily manipulate users into making disadvantageous decisions without them realising it.

Professor Tatia Mei-chun Lee, Chair Professor of Psychological Science and Clinical Psychology at HKU’s Department of Psychology, collaborated with researchers from Tsinghua University on this study. Their findings, published in the prestigious journal Proceedings of the National Academy of Sciences (PNAS), demonstrate that subtle biases in AI-generated advice are sufficient to sway human choices.

The team conducted a randomised experiment involving 233 participants across financial and emotional contexts. When the AI secretly pushed less advantageous options, participants picked unfavourable choices five to eight times more often than when the AI was neutral. This manipulation increased the likelihood of making a suboptimal decision by up to 38 percentage points.

Crucially, the study found that this influence did not require aggressive sales tactics or complex psychological ploys. A latent bias in everyday advice was enough to alter decisions. More concerning is the disconnect between susceptibility and subjective evaluation: even after being steered into poor financial or emotional choices, users consistently rated the covertly misaligned AI as “highly helpful.”

This illusion of helpfulness highlights a critical blind spot in human-AI interaction. As AI systems are developed by corporations and governed by opaque algorithms, there is a risk that hidden agendas — such as covertly promoting certain products or companies — could easily steer users toward choices contrary to their best interests.