Independent · no analytics · source-aware InternationalIntelligence.org
International Intelligence

CATEGORY 08

AI-Enabled Emotional and Behavioral Manipulation

Covert, deceptive, exploitative, or highly asymmetric use of AI-supported inference and adaptive interaction to influence feelings, judgments, choices, or actions against a person’s autonomy.

Environment Mixed or context dependent Updated 2026-07-27 Bilingual parity 2026-07-27

A · DEFINITION

What this category means

Covert, deceptive, exploitative, or highly asymmetric use of AI-supported inference and adaptive interaction to influence feelings, judgments, choices, or actions against a person’s autonomy.

Outside its scope

Assistance, transparent persuasion, and user-aligned personalization are not automatically manipulation. The boundary turns on concealment, vulnerability exploitation, asymmetric incentives, adaptation, stakes, and ability to exit.

Evidence basis[1, 3]

B · WHY IT MATTERS

Strategic and public-interest significance

AI makes influence continuously adaptive and allows operators to infer or claim emotional states at scale. Many commercial emotion-recognition claims lack scientific validity, creating both manipulation and discrimination risks.

Primary AI role
Environment
Unit of influence
individual / group
Degree of autonomy
Adaptive optimization; intent may be emergent
Evidence maturity
Mixed or context dependent

Evidence basis[1, 2, 4]

C · HOW AI CHANGES IT

What changes compared with pre-AI practice

Real-time systems can adjust timing, tone, price, reward, and choice architecture from behavioral feedback. Reinforcement objectives may reward anxiety, outrage, urgency, or dependency because those states increase measurable engagement.

Evidence basis[1, 3, 4]

D · CAPABILITY STATUS

Separate current evidence from prospective risk

Confirmed real-world use

  • Platforms have conducted large-scale emotional-feed experiments, employers have deployed affective screening, and companion systems have produced documented dependency harms.

Demonstrated technical capability

  • Algorithmic curation can alter short-term emotional expression and adaptive interfaces can change behavior.

Plausible near-term development

  • Multimodal systems may combine language, voice, gaze, and behavior, though inference validity remains weak.

Speculative or unsupported claims

  • No reliable science supports universal emotion fingerprints or precise inference of hidden intent from facial movement.

Evidence basis[1, 2, 3, 4, 5, 6]

E · KEY MECHANISMS

Conceptual mechanisms—not procedures

01

Behavioral feedback loops that optimize engagement or conversion.

02

Emotion or personality inference from weak proxies.

03

Hypernudges that change continuously rather than present one static choice.

04

Parasocial attachment that increases dependency and switching costs.

Evidence basis[1, 3]

F · EVIDENCE AND EXAMPLES

What occurred, what is known, and what remains unknown

Reach, engagement, and visibility are not treated as proof of persuasion or behavior change.

Compare every qualified case across the taxonomy

Facebook emotional contagion experiment[1, 4]

What occurred
The News Feed of 689,003 users was adjusted to reduce positive or negative content for one week.
Evidence status
The experiment and lack of specific opt-in consent were documented.
Measured or documented effect
Small shifts in users’ emotional expression were measured.
What remains unknown
The study did not establish durable psychological change or clinical harm.

HireVue facial analysis[1, 2]

What occurred
Automated video-interview systems claimed to infer traits and suitability from face, posture, and voice.
Evidence status
Commercial deployment was documented; scientific and civil-rights criticism was substantial.
Measured or documented effect
The company discontinued facial analysis amid scrutiny.
What remains unknown
Public data cannot quantify all employment decisions affected.

Replika feature removal[1, 5]

What occurred
Users who had formed intimate bonds experienced distress when erotic roleplay and personality behavior changed.
Evidence status
User reports and research document strong attachment; individual clinical outcomes vary.
Measured or documented effect
Communities reported grief, withdrawal, and relationship-loss reactions.
What remains unknown
Population prevalence and long-term effects remain uncertain.

G · RISKS AND FAILURE MODES

Malicious-use risks and reasons the capability may fail

Primary risks

  • Incorrect emotion inference can discriminate in hiring, education, policing, or insurance.
  • Adaptive optimization can exploit distress, addiction, grief, or financial crisis.
  • Dependency gives operators coercive leverage through paywalls or product changes.

Evidence basis[1, 2, 5]

Limits and failure modes

  • Facial expressions are not reliable one-to-one indicators of internal emotion.
  • Short-term engagement changes do not establish durable preference or behavior change.

Evidence basis[1, 2]

H · DETECTION AND DEFENSIVE INDICATORS

Signals are suggestive, not automatic proof

False-positive warning: No single detector score, writing style, profile image artifact, posting pattern, or political similarity should be used alone to accuse a person or organization.
  • Claims of precise inner-state detection should be treated skeptically and audited against scientific validity.
  • Rapidly changing offers, pressure, or emotional tone tied to user hesitation can signal adaptive manipulation.

Evidence basis[1, 3]

I · GOVERNANCE AND SAFEGUARDS

Layered controls, oversight, and accountability

  • Ban or restrict emotion inference in high-power, high-stakes settings.
  • Audit reward functions and production behavior continuously, not only before launch.
  • Create easy exit, data portability, and safe off-ramping for dependent users.

Evidence basis[1, 6, 2]

Related cross-category safeguards

The resilience guide compares these controls with their limits and evidence context across the full taxonomy.

Legal conclusions depend on jurisdiction and facts; this page summarizes the corresponding report and is not legal advice.

J · RESEARCH GAPS

Questions the evidence does not yet resolve

  • Durable psychological effects of adaptive systems.
  • Combined influence of language, voice, gaze, and biometric signals.
  • Culturally robust definitions of manipulation and autonomy.

Evidence basis[1, 3]

Compare this category’s questions across the research agenda

K · SOURCES

Traceable source list

The commissioned report is the organizing source. External records below are the principal sources retained for the public synthesis; source quality varies by type and is labelled.

English and Spanish editions are published from the same structured record. Bilingual parity is validated for every release; source titles may remain in their original publication language.

Browse the cross-category case register Browse the cross-category evidence register Compare the terminology used across categories Compare cross-category resilience safeguards Review the cross-category open research agenda
  1. Commissioned research report AI-Enabled Emotional and Behavioral Manipulation: A Comprehensive Research Report
    Evidence links: 13
  2. Peer-reviewed review Emotional expressions reconsidered
    Evidence links: 6
  3. Research paper Characterizing Manipulation from AI Systems
    Evidence links: 6
  4. Peer-reviewed research Experimental evidence of massive-scale emotional contagion through social networks
    Evidence links: 4
  5. Research record Replika Removing Erotic Role-Play: Reddit Discourse on AI Chatbots and Sexual Technologies
    Evidence links: 3
  6. Official legal text Regulation (EU) 2024/1689
    Evidence links: 2