EMERGE-Policy:

Abstract

A robot's effective “mind” need not reside in a single policy. It can emerge when specialized components perceive, reason, predict, act, verify, and remember within a shared orchestration process. EMERGE-Policy turns this perspective into a graph-structured agentic framework that coordinates both capability invocation and information exchange.

A Main Agent retains task-level state within an active context window, while role-specific Sub Agents process perception, execution monitoring, verification, and memory consolidation in isolated contexts and return structured, task-relevant evidence. The functional Skill interface composes heterogeneous backends as Operational, Imagination, and Evaluation Skills. Criterion-grounded verification, textual failure diagnosis, Branch Stack recovery, and token-aware external memory provide localized correction throughout execution.

Without additional fine-tuning, EMERGE-Policy improves several public benchmark results and supports long-horizon real-robot experiments. The results suggest that dividing functional subtasks among collaborating agents, while treating models as callable skills, can extend robot policies beyond isolated rollouts.

What emerges?

Method

Given an instruction T and observation O_t, EMERGE-Policy maintains an active task context C_t, decomposes the instruction into criterion-grounded subgoals, and selects the next Sub Agent or Skill. Each subgoal is represented as ⟨Target, Change, Criterion⟩, making the intended state transition and its verification condition explicit.

Standardized interfaces and lazy loading

Every callable interface follows the schema K = ⟨I_meta, Θ_param, Φ_pre, Ψ_post⟩. The Main Agent initially sees only a compact capability signature. Full parameters, preconditions, and postconditions are retrieved only when the interface is selected, limiting context growth.

Results

We evaluate the complete orchestration system on LIBERO, LIBERO-Plus, LIBERO-Pro, and RoboDojo, using fixed action primitives and different VLA/WAM execution backends. The headline numbers below are system-level results: they reflect the complete interaction of planning, perception, skill invocation, verification, and recovery.

Selected benchmark results

Standard LIBERO

Success rates (%) across LIBERO-Goal, Spatial, Object, and Long. WM denotes the world-model pathway.

LIBERO-Plus

LIBERO-Plus robustness under seven perturbation types. Success rates (%) are reported for each perturbation, with the mean in Average. Colored values show the absolute change from the corresponding execution backend.

Real-robot deployment

We study a long-horizon “Multi-layered Cup Stacking” task in which the robot identifies and picks up six cups, then constructs a three-tier pyramid with three, two, and one cups. The task tests semantic grounding, repeated grasp-and-place actions, concurrent agent coordination, memory of progress, and recovery. Across 100 real-robot experiments, the closed-loop system maintained high completion rates under several interference settings.

Analysis

Three-tier memory and recovery

EMERGE-Policy externalizes long-horizon state instead of relying on an unbounded neural memory. PLAN.md stores the subgoal decomposition and progress, ENVIRONMENT.md records the latest physical state, and structured Sub-Agent evidence provides concise short-term facts. An asynchronous visual monitor compares outcomes with the active criterion and feeds corrections back to the Main Agent. When a local failure is detected, a recovery objective is pushed onto the Branch Stack before global replanning is considered.

Imagination and Evaluation Skills

The Imagination Skill generates candidate action chunks and predicts their future observations. The Evaluation Skill scores those predictions against the active completion criterion, and the Main Agent greedily dispatches the highest-scoring candidate. In the reported implementation, the system evaluates M = 5 candidates with horizon H = 16; this adds approximately 1.2 seconds per selection step while reducing risky physical trial-and-error.

Task-specific priors improve efficiency

Planning priors expose the coupling between high-level subgoals and the native capabilities of the execution skills. In the cup-stacking study, this removes redundant move_to_pose cycles, reducing planning steps per subgoal by 24.7% and wall-clock time per episode by 16.3%, while maintaining comparable success rates.

BibTeX

@article{fang2026emergepolicy,
  title   = {EMERGE-Policy: A Robot Mind Emerges Beyond a Single Policy},
  author  = {Fang, Zhirui and Yu, Qingchi and Chen, Ziyang and Li, Longfei and Ma, Haoran and Zhou, Keru and Xu, Xinrun and Va, Samith and Hu, Yuxuan and Song, Peixuan and Du, Qiang and Qian, Bin and Deng, Yongkang and Li, Xin and Wang, Yezhen and Li, Zhe and Luo, Hao and Li, Shuyan and Wang, Ziwei and Deng, Weijian and Li, Xiu},
  journal = {arXiv preprint arXiv:2608.29896},
  year    = {2026}
}