WebCatalyst

Catalyzing Interactive HTML Generation Capabilities
via High-Fidelity Reinforcement Learning

Xujia Wang1,2,*Yixuan Liu1,*Shaohan Huang2,†Furu Wei2Lei Hou1Bin Xu1,†Juanzi Li1
1 Tsinghua University2 Microsoft Research

* Equal contribution · † Corresponding authors

WebCatalyst research overview · 5:02 · English
Download 1080p ↓

Abstract

Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functionality and visual quality during actual interaction. Existing methods typically target task-specific webpage supervision, leaving demanding interactive artifact optimization underexplored. Using mini-games as a challenging testbed, we expose limitations in interaction experience, reward signal, and credit assignment fidelity that are obscured by shallow webpage interactions. We introduce WebCatalyst, a browser-interaction reinforcement learning framework with three complementary components: a game-level GUI assessor for sustained interaction and evaluation, Visual Anchor for mitigating VLM reward hacking, and Action-Trace Credit for grounding token-level credit assignment in runtime execution traces. Trained on fewer than 1K requests, predominantly for games, WebCatalyst raises the Cookie-Bench Mini Games score from 68.8 to 75.3 and also the WebGen-Bench accuracy from 48.8 to 56.9 on Qwen3.6-35B-A3B, outperforming Claude Opus 4.7 in average rank across evaluated metrics. Further analysis shows that Visual Anchor prevents aesthetic degradation, while runtime-grounded credit assignment accelerates functional improvement. Together, our results show that improving feedback fidelity is the key to substantially advancing interactive HTML generation.

Key result. With 640 training requests, WebCatalyst improves Qwen3.6-35B-A3B’s average rank from 14.0 to 4.0 within the evaluated 20-model pool, with performance comparable to Claude Opus 4.7.

Method

WebCatalyst combines experience-grounded assessment, visual reward calibration, and runtime-grounded credit assignment.

HTML request→Generated artifact→Browser interaction→High-fidelity feedback→Policy update
Figure 2 · WebCatalyst’s browser-interaction reinforcement learning framework.
01

Game-Level
GUI Assessor

Experience fidelity

Execute sustained interactions in a real browser. Evaluate state transitions, then aggregate action outcomes and visual judgments into a page-level reward.

Ei,t=(St,at,St+1,mt)
§3.1 · Ground judgments in actual execution.
02

Visual
Anchor

Reward signal fidelity

Close the loophole where easier interactions mask visual degradation. Penalize visual regression relative to a high-quality reference; apply no penalty to comparable or better visuals.

ri=RiGUI−Di100
§3.2 · Calibrate the reward against a reference.
03

Action-Trace
Credit

Credit assignment fidelity

Trace successful actions back to executed JavaScript. Map call stacks to generated tokens, then redistribute the advantage by call depth while preserving its mean.

ci,k=γdi,kmin
§3.3 · Assign credit using runtime evidence.
A closer look at sign-aware credit assignment+

Sign-aware, mean-preserving credit assignment

For a positive page advantage, tokens on successful execution paths receive stronger positive updates. For a negative advantage, they receive weaker negative updates relative to untraced tokens.

Normalized token advantages enter the clipped GRPO objective. Successful behavior changes where the signal goes, while the sample-level scale stays intact.

w~i,k=[1+βsgn(Ai)ci,k]+A^i,k=Aiw~i,k1Ni∑u=1Niw~i,uEq. (7) · Sign-aware modulation and mean-preserving normalization.

Main Results

Training on 640 requests improves both webpage generation and sustained interactive experiences.

Table 1 · Full comparison

Complete benchmark results

20 models 2 benchmarks

Scroll horizontally to view all metrics →

Table 1. Complete main results: model date, size, all WebGen-Bench and Cookie-Bench Mini Games metrics, average rank, and changes from the base model.
ModelDateSizeWebGen-BenchCookie-Bench Mini GamesAvg.
Rank ↓
Start Fail.
Rate
↓
Acc. ↑ Content
Pres.
↑
User
Inter.
↑
Data
Mgmt.
↑
Func.
Test
↑
Data
Display
↑
Design
Val.
↑
Aesth. ↑ Func. ↑ Total ↑

Bold marks group-wise best results, with WebCatalyst included in Other LLMs. The shaded rows identify the base model and WebCatalyst. Arrows indicate preferred directions.

Date is YY/MM. Size gives total/active parameters; undisclosed sizes are marked accordingly. Acc., Aesth., and Func. denote accuracy, aesthetics, and functionality. Average rank averages ranks across evaluation dimensions. Δ gives changes from Qwen3.6-35B-A3B as reported in Table 1 ↗.

Analysis

From reward design to interaction quality: what changes with high-fidelity feedback?

01 Training dynamics §4.4.1

Why each reward component matters

Functional gains can hide visual degradation.

With GUI-only rewards, functionality improves while aesthetics declines. The full method improves both, preserving the visual signal that task success alone can miss.

GUI-only · Mini Games aesthetics
81.571.3
Initial generation → training step 60
§4.4.1 · WebGen accuracy, Mini Games functionality, and Mini Games aesthetics.

02 Experience diagnostics §4.4.2

Better interactions, across dimensions

All six experience dimensions improve, alongside screenshot aesthetics. Gains also span every measured action type, indicating broader improvements in interaction quality.

Action success rate+19.4pp

48.0% 67.4%

Qwen3.6-35B-A3BWebCatalyst

Experience dimensions

All dimensions improve ↗

Action transitions

More successful interaction ↗

Figure 5 · Experience-dimension and action-transition scores before and after training.

View the numerical results
Reward components
DimensionQwenWebCatalyst
Visual stability88.394.9
Screenshot aesthetic75.585.1
Feedback clarity48.764.9
Interaction reliability56.069.7
State legibility76.988.9
Prompt alignment54.068.1
Task progress26.142.1
Action transitions
DimensionQwenWebCatalyst
Combo83.685.6
Mouse move / hover84.188.7
Press76.384.2
Terminate63.276.0
Type93.096.0
Wait74.384.5
Success rate (%)48.067.4

03 Iterative refinement §4.4.2

Single-shot gains persist under refinement

WebCatalyst’s initial generation is already comparable to the base model after two refinement rounds. It maintains a higher Mini Games Total at every corresponding round, reaching 76.9 at round 3.

WebCatalystInitial generation
75.3
Mini Games Total
75.2
Qwen3.6-35B-A3BAfter 2 refinement rounds
Qwen3.6-35B-A3BWebCatalyst

Table 3. Mini Games performance under iterative refinement.

Table 3. All aesthetics, functionality, and total scores for refinement rounds 0 to 3.
RoundQwen3.6-35B-A3BWebCatalyst
Aes.Func.TotalAes.Func.Total
081.556.168.887.563.275.3
185.157.971.588.163.475.7
287.762.875.289.561.875.7
386.262.174.291.162.776.9

Round 0 is the initial generation; rounds 1–3 revise the artifact using feedback from the GUI Assessor. Bold marks the better result for each metric within each round.

§4.4.2 · Table 3 · Mini Games performance with GUI-Assessor-guided refinement.

Interactive Demos

Compare original generated outputs for the same prompt. Watch recorded interactions or open the artifacts to try them directly.

3D flight / Prompt 17

Sky Runner

View the original prompt+

Loading the comparison gallery…

Recorded browser interaction · Original generated HTMLSelected examples illustrate behavior; benchmark results provide aggregate evidence.

Explore the examples

Full demo collection

50 prompts × 6 models · 300 generated artifacts

BibTeX

If you find this work useful, please consider citing our paper.

BibTeX
@misc{wang2026catalyzinginteractivehtmlgeneration,
  title={Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning},
  author={Xujia Wang and Yixuan Liu and Shaohan Huang and Furu Wei and Lei Hou and Bin Xu and Juanzi Li},
  year={2026},
  eprint={2611.05151},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2611.05151},
}
Interactive playground

Model comparison

Click inside a game to use its controls. Open an artifact in a new tab for the full-screen experience. Press Esc to release pointer lock.

Original request +

Enlarged research figure