文章背景与核心概要

现代推荐系统通常将用户的交互行为视为其稳定偏好的直接代理。然而，用户的实际行为往往具有探索性或比较性，并非对底层欲望的绝对表达。随着现代用户界面向生成式UI和沉浸式扩展现实（XR）环境演进，系统不仅需要理解用户“选择什么”，更需要理解用户“为何如此行动”。

本文引入了逆向心智理论（Inverse Theory of Mind, IToM）流水线，通过从用户的交互行为进行逆向推理，推导驱动其行为的信念、偏好和决策特征。该方法将反事实推理与多假设 abductive 推理（溯因推理）相结合，构建出强有力的、以证据为基础的用户画像。在 OPeRA 数据集上的评估表明，这些推导出的画像在多个任务（包括下一步动作预测、购物态度对齐、大五人格推断以及留一类别预测）中均达到或超过了真实画像（ground-truth personas），同时成功在 VisionOS 的空间银行应用中展现出了跨模态迁移能力。

Inverse Theory of Mind Modeling for Content Recommendation: From Web Browsing to Dynamic Intelligent Interfaces

arXiv: arXiv:2608.11354 [cs.AI]
DOI: 10.48550/arXiv.2608.11354
Related DOI: 10.1145/3773078.3831787 (Accepted at RecSys '26)
Submitted: August 11, 2026
Authors: Mengyu Chen, Feiyu Lu, Chun-Fu Chen, Lucas Vinh Tran, Jay Katukuri

📌 Summary

Modern content recommenders often treat user interactions as direct proxies for stable preferences. However, user behavior is frequently exploratory or comparative rather than a strict expression of underlying desires. As modern user interfaces shift toward generative UIs and immersive extended reality (XR) environments, systems must understand not only what users choose, but why they act.

This paper introduces the Inverse Theory of Mind (IToM) pipeline, which reasons backward from user interactions to infer the beliefs, preferences, and decision-making traits driving behavior. By combining counterfactual reasoning with multi-hypothesis abductive inference, the approach constructs robust, evidence-grounded user personas. Evaluations on the OPeRA dataset show that these inferred personas match or exceed ground-truth personas across multiple tasks—including next-action prediction, shopping attitude alignment, Big Five personality inference, and held-out category prediction—while successfully demonstrating cross-modal transferability in a spatial banking application on VisionOS.

📖 Abstract

现代推荐系统将观察到的行为视为用户偏好的可靠代理，然而交互往往反映了探索或比较，而非稳定的偏好表达。随着界面从静态布局向生成式 UI 和沉浸式扩展现实（XR）演进，对更深层次、与模态无关的用户理解的需求日益增长：这些自适应环境不仅必须决定呈现什么，还要决定在哪里、何时、以多显著的方式呈现，以及最重要的是用户为何采取行动。

Modern recommender systems treat observed actions as reliable proxies for user preferences, yet interactions often reflect exploration or comparison rather than stable preference expression. As interfaces evolve from static layouts toward generative UIs and immersive extended reality (XR), the need for deeper, modality-agnostic user understanding grows: these adaptive environments must decide not only what to present but where, when, how prominently, and most importantly why a user acts.

我们提出了一种逆向心智理论（IToM）流水线，通过从观察到的交互中进行逆向推理，推导解释行为的信念、偏好和决策特征。该流水线重建每个用户的决策上下文（包括选择的内容和可用的替代方案），应用大语言模型驱动的反事实推理来生成有证据支撑的自然语言信念陈述，并通过多假设溯因推理将这些信念综合成结构化的用户画像。

We propose an Inverse Theory of Mind (IToM) pipeline that reasons backward from observed interactions to infer the beliefs, preferences, and decision-making traits that explain behavior. The pipeline reconstructs each user's decision context, including what was chosen and what alternatives were available, applies LLM-driven counterfactual reasoning to produce evidence-grounded natural-language belief statements, and synthesizes these beliefs through multi-hypothesis abductive inference into a structured user persona.

我们在 OPeRA 数据集上针对真实个性评估、态度调查和基于访谈的用户画像进行了四项任务的评估：下一步动作预测、购物态度对齐、大五人格推断和留一类别预测。结果表明，推导出的画像匹配或超过了真实画像，且多假设推理对于准确的人格预测至关重要。我们进一步展示了在 VisionOS 空间银行应用中基于画像的跨模态迁移能力。

We evaluate on the OPeRA dataset against ground-truth personality assessments, attitudinal surveys, and interview-based personas across four tasks: next action prediction, shopping attitude alignment, Big Five personality inference, and held-out category prediction. Results show that inferred personas match or exceed ground-truth personas and that multi-hypothesis reasoning is essential for accurate personality prediction. We further demonstrate cross-modal transferability with a persona-driven spatial banking application on VisionOS.

🛠️ Key Pipeline Components

1. 上下文重建： 通过在决策上下文中捕捉所选动作与可用替代方案，分析用户的交互。

   1. Context Reconstruction: Analyzes user interactions by capturing both selected actions and available alternatives within the decision context.

2. 反事实推理： 利用大语言模型（LLMs）生成有证据支持的、解释用户信念的自然语言陈述。

   1. Counterfactual Reasoning: Leverages Large Language Models (LLMs) to generate evidence-grounded, natural-language statements explaining user beliefs.

3. 溯因推理： 通过多假设框架综合各种信念，构建连贯、结构化的用户画像。

   1. Abductive Inference: Synthesizes beliefs via a multi-hypothesis framework to construct a coherent, structured user persona.

4. 跨模态适配： 成功将传统的网页浏览行为与空间计算环境（如 VisionOS 应用）连接起来。

   1. Cross-Modal Adaptation: Successfully bridges traditional web browsing behavior with spatial computing environments (e.g., VisionOS applications).

📋 Document Metadata & Details

Field	Detail
Primary Subject	Artificial Intelligence (cs.AI)
ACM Classes	H.3.3; H.5.1; I.2.4; I.2.7
Conference Venue	20th ACM Conference on Recommender Systems (RecSys '26), Minneapolis, MN, USA
Full-Text & Resources	View PDF | HTML Version | TeX Source | View License