Qwen-Image-Agent — Bridging the Context Gap in Real-World Image Generation

While text-to-image (T2I) models have achieved remarkable progress, they struggle with real-world requests that are often underspecified, implicit, or dependent on up-to-date knowledge. We identify this challenge as the Context Gap: the mismatch between the user context and the sufficient generation context for T2I models.

To bridge this gap, we propose Qwen-Image-Agent, a unified agentic framework that integrates plan, reason, search, memory and feedback in a context-centric manner. Qwen-Image-Agent treats user input as partial context and progressively constructs the generation context through Context-Aware Planning and Context Grounding.

To evaluate agentic image generation, we further introduce Image Agent Bench (IA-Bench), a benchmark covering four core image agent capabilities: Plan, Reason, Search, and Memory. Experiments on IA-Bench, Mindbench and WISE-Verified show that Qwen-Image-Agent outperforms strong baselines and achieves state-of-the-art performance.

As these systems move into real-world applications such as marketing, product design, and slide creation, they are increasingly expected to solve practical visual tasks rather than merely render prompts.

Despite their generative ability, current T2I models remain limited on real-world tasks. A key reason is a structural mismatch between training and deployment: models are optimized for fully specified prompts, while real-world requests are often underspecified. In practice, successful generation may require inferring implicit user intent, retrieving up-to-date knowledge or visual references from web, and incorporating interaction history.

We refer to this mismatch as the Context Gap: the gap between the provided user context and the generation context required for T2I models. This gap motivates a paradigm shift from traditional direct image generation to agentic image generation.

Recent work has explored components such as plan, reason, search and tool use, memory, and self feedback, but these efforts remain fragmented and do not provide a unified framework for context-centered generation.

We formalize image generation and edit as a conditional rendering problem. Given a user context $c_u=(P,I_{\mathrm{ref}})$ with prompt $P$ and optional reference images $I_{\mathrm{ref}}$, direct image generation renders output image $y$ in a single forward pass: $y \sim p_{\mathrm{gen}}(\cdot \mid c_u)$.

We distinguish user context $c_u$ from the generation context $c_g$, which denotes the complete context needed for successful rendering. Agentic image generation introduces a context-construction process: at each step $t$, the agent maintains state $s_t$, takes action $a_t$, and receives observation $o_t$, forming a trajectory $\tau=\{(s_t,a_t,o_t)\}_{t=1}^{T}$.

The action space consists of basic operations to gather context, including plan, reason, search, rewrite, and evaluate. The agentic generation process is: $p_{\mathrm{agent}}(y \mid c_u)=\sum_{\tau} p(\tau \mid c_u)\, p_{\mathrm{gen}}(y \mid c_g=c(\tau))$.

Context-Aware Planning operates at three levels: information-level planning identifies the context gap and routes questions to grounding strategies; content-level planning assembles grounded context and rewrites the user prompt into a detailed specification (subject, attributes, layout, style, textual elements); generation-level planning allocates context in multi-image and multi-turn scenarios.

Context Grounding collects context through reason, search, memory and feedback. Reasoning covers commonsense, logical, and visual reasoning via VLM. Search uses web search for factual knowledge and image search for visual references. Memory incorporates conversation history, user profiles, and external multimodal retrieval. Feedback plans a checklist, uses VLM to evaluate outputs, and refines prompts iteratively.

IA-Bench covers four core capabilities: Plan, Reason, Search, and Memory. The benchmark consists of 4 tasks, 17 subtasks, 730 instances and 1801 evaluation checklist items.

Planning-driven tasks include Composition, Enumeration, and Multi-Panel. Reasoning-driven tasks include Math, Science, Commonsense, Maze, Map, and Geometry. Search-driven tasks cover IP-related entities (Game, Movie, Anime, Celebrity) and Information (Stock, Weather). Memory-driven tasks include User Profile and Conversation History.

We adopt checklist-based evaluation. Pass Rate (PR) requires all checklist items satisfied. Checklist Accuracy (CA) averages the proportion of satisfied items. IA-score = $0.3\times\mathrm{Plan}+0.3\times\mathrm{Reason}+0.3\times\mathrm{Search}+0.1\times\mathrm{Memory}$.

We employ Qwen-Image-2.0 as the image generation and edit backbone, and GPT-5.5-0424 as the MLLM backbone. We utilize Google Search API for web and image search (limit 5 each), and Jina API for visited web pages. All agentic baselines use the same MLLM and generation backbone for fair comparison. Feedback loop allows up to 3 attempts on IA-Bench; feedback is disabled on WISE-Verified and MindBench.

On IA-Bench (Table 1), Qwen-Image-Agent achieves IA-score 45.4 with PR 45.3/43.7/46.1/49.0 on Plan/Reason/Search/Memory, outperforming MindBrush (30.2), SCOPE (30.9), and Qwen-Image-2.0 direct (17.4).

On WISE-Verified, Qwen-Image-Agent reaches Overall 0.9020, surpassing Nano Banana Pro (0.8760). On MindBench Overall, Qwen-Image-Agent achieves 0.42 vs Qwen-Image-2.0 0.23 (+82.6%).

Ablation (Table 4): removing Reason drops Plan PR to 24.7 and Reason to 29.7; removing Search drops Search PR to 7.8; removing Memory drops Memory PR to 0.0; removing Feedback causes smaller drops (IA-score 42.1 vs 45.4). Replacing MLLM with Qwen-Plus/VL-Max drops IA-score to 27.8; replacing Gen backbone with Qwen-Image drops to 28.3.

In this work, we identify the context gap as a central challenge in real-world image generation. To address it, we propose Qwen-Image-Agent, a unified agentic framework that integrates plan, reason, search, memory and feedback in a context-centric manner. We further introduce IA-Bench, a benchmark for systematically evaluating four core capabilities of agentic image generation: Plan, Reason, Search, and Memory.