ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning

SkillsBench performance versus model size, and an illustration of how evolving capability gaps and repeated rollout failures motivate adaptive rubrics, skills, and task selection. — **Learning needs evolve.** Fixed criteria can miss new capability gaps, while failed rollouts can hide partial behavioral progress. ARISE turns rollout evidence into evolving evaluation criteria, targeted guidance, and task priorities.

SkillsBench v1.1

23.4% → 45.6%

Qwen3.5-27B → ARISE
+22.2 percentage points

Terminal-Bench v2.1

41.6% → 50.6%

Qwen3.5-27B → ARISE
+9.0 percentage points

Training efficiency

50 vs. 150 steps

Matches outcome-only RL performance
with the same per-step rollout budget

Abstract

As a long-horizon agent improves through experience, previously observed weaknesses may recede while new limitations emerge, continually changing what it still needs to learn. Yet the learning process often remains tied to a static view of these needs: fixed behavioral criteria and training priorities can become misaligned with evolving agent capabilities, while sparse task-level feedback makes such misalignment more difficult to detect. Even when capability gaps are identified, rollouts from the current policy may repeatedly reproduce the same failures rather than explore better alternatives. To address this, we introduce Adaptive Rubric–Skill Co-Evolution (ARISE), a reinforcement learning framework that uses rollout evidence to continually adapt evaluation criteria, exploration guidance, and training priorities. Rubrics evolve to reward partial behavioral progress, while their paired skills are refined and selectively activated to guide exploration toward unresolved weaknesses. Alongside this co-evolution, capability-based adaptive sampling prioritizes tasks that target behaviors needing further improvement. Experiments on two challenging long-horizon agent benchmarks, SkillsBench and Terminal-Bench, demonstrate that ARISE successfully enhances both overall task performance and training efficiency.

Overview

A continuous feedback loop aligns behavioral evaluation, exploration guidance, and training data with the agent's changing capabilities.

ARISE framework: capability-based sampling selects tasks, activated skills guide agent rollouts, task verifiers and rubric judges evaluate trajectories, and the resulting evidence updates both the policy and the rubric-skill pool. — **The ARISE training loop.** Shared rollout evidence drives rubric–skill co-evolution and capability-based adaptive sampling. Task outcomes and behavioral feedback jointly inform policy optimization.

Methodology

Two coupled components keep training focused on behavioral gaps that still offer opportunities for improvement.

Evaluate & guide

Rubric–Skill Co-Evolution

Each behavioral criterion pairs a rubric for evaluating observable progress with a skill that provides actionable guidance. A reflection model uses recent rollout evidence to discover emerging gaps and add new pairs.

AddActivateRefineHideRetire

Low rubric pass rates activate paired skills; persistent failures prompt refinement. As performance improves, guidance is hidden while evaluation continues. Consistently mastered pairs are retired, making room for new learning needs.

Select & learn

Capability-Based Adaptive Sampling

Rubric judgments estimate capability performance within each task type, revealing behavioral weaknesses that task-level success rates alone can obscure.

DiscoverEstimatePrioritize

Coverage-guided discovery explores less-evaluated tasks. Adaptive selection prioritizes task-type–capability pairs with informative success probabilities, balancing learning potential against repeated failure. As rubrics evolve, unchanged evidence is retained and obsolete contributions are discarded.

Training strategy. The rubric–skill pool starts empty, so initial training uses task rewards alone. As criteria emerge, task rewards and per-rubric feedback are normalized separately and combined for GRPO-style policy optimization with importance-ratio filtering. Updates apply to model-generated tokens, excluding tool outputs and environment observations.

Results

ARISE achieves the highest overall pass rates among comparable-scale models on both benchmarks, with gains that transfer from the Kilo Code harness to Terminus-2.

Task pass rates in percent on SkillsBench v1.1 and Terminal-Bench v2.1.
Model	SkillsBench v1.1									TB v2.1
Model	SE	NS	OW	IP	FE	MR	CS	MC	Overall	Overall
Proprietary Models
GPT-5.4 Mini	27.1	35.7	45.2	21.4	25.9	41.7	33.3	66.7	34.5	59.2
GPT-5.5	63.4	77.9	76.2	57.5	37.0	95.0	69.0	60.0	67.3	84.3
Claude Opus-4.7	58.3	83.3	54.8	54.8	44.4	50.0	57.1	53.3	58.6	83.1
Open-Weight Models
GPT-OSS-120B	—	—	—	—	—	—	—	—	—	26.2
MiniMax-M2.7	14.6	59.5	28.6	19.0	33.3	29.2	23.8	13.3	28.7	55.4
GLM-5.1	41.7	76.2	69.0	45.2	33.3	37.5	47.6	80.0	53.6	61.8
Kimi-K2.6	39.6	76.2	57.1	50.0	44.4	66.7	47.6	66.7	55.2	65.9
DeepSeek-V3.2	—	—	—	—	—	—	—	—	—	46.8
DeepSeek-V4-Pro	37.5	73.8	52.4	42.9	29.6	66.7	42.9	80.0	51.3	64.8
DeepSeek-V4-Pro-0813	62.5	81.0	64.3	47.6	48.1	50.0	42.9	60.0	59.0	78.7
Qwen3.5-122B-A10B	18.8	40.5	23.8	16.7	22.2	33.3	19.0	13.3	24.1	47.6
Qwen3.5-397B-A17B	16.7	45.2	38.1	31.0	22.2	37.5	23.8	20.0	30.3	51.3
Nemotron-3-Ultra-550B-A55B	—	—	—	—	—	—	—	—	—	53.9
Comparable-Scale Models
Qwen3.5-27B (Base Model)	14.6	40.5	26.2	23.8	18.5	29.2	14.3	6.7	23.4	41.6
VADE	25.0	54.8	38.1	31.0	33.3	50.0	23.8	73.3	38.7	43.8
OnlineRubrics	29.2	64.3	38.1	31.0	37.0	33.3	28.6	73.3	40.2	46.1
RuscaRL	33.3	57.1	42.9	26.2	40.7	29.2	23.8	73.3	39.5	44.9
ARISE	41.7	59.5	52.4	38.1	40.7	33.3	23.8	80.0	45.6	50.6

Pass rates (%); higher is better. Bold red values mark the best results within comparable-scale models. Dashes indicate unreported results.
SE: software engineering; NS: natural science; OW: office and white collar; IP: industrial and physical systems; FE: finance and economics; MR: mathematics and operations research/formal reasoning; CS: cybersecurity; MC: media and content production.

Evaluation protocol. SkillsBench uses Kilo Code with benchmark-provided curated skills. These differ from the rubric-paired skills evolved by ARISE, which are withheld on both evaluation benchmarks. Terminal-Bench uses Terminus-2. Our evaluations average three independent runs. SkillsBench GPT-5.5 results and Terminal-Bench results outside the comparable-scale group come from official leaderboards.

Analysis

Each component contributes to learning

Dynamic rubric evolution, skill-guided exploration, and capability-based sampling each contribute to performance. Under the same per-step rollout budget, ARISE reaches the step-150 performance of outcome-only RL by step 50 and continues improving. This comparison measures training steps and rollout efficiency.

Ablation comparison of ARISE against frozen rubrics, no skill injection, alternative sampling, and outcome-only RL, alongside SkillsBench pass rates over training steps. — **Ablations and training efficiency.** Variants use matched initialization, data, and training budgets. Even the full lifecycle rubric pool, when kept fixed, does not replace dynamic evolution.

Behavioral feedback evolves with the policy

Starting from an empty pool, ARISE introduces 36 criteria and retires 21, leaving 15 active. Supervision changes through replacement, not just accumulation. On a fixed task set, skill guidance reduces all-failure rollout groups compared with the variant without skill injection, with the largest gap early in training.

Training dynamics showing active, added, and retired rubrics; rubric reward and task pass rates; and lower all-failure rollout group rates with skill injection. — **Co-evolution dynamics.** The active criteria change as capabilities develop, while paired skills help the policy explore beyond repeated failures.

Training priorities shift toward emerging gaps

Early adaptive sampling emphasizes execution and verification. Later stages increasingly target debugging and efficiency as new criteria enter the pool. Blank cells indicate limited coverage or absent active rubrics; discovery sampling continues to explore these tasks.

Heatmaps of task-type and capability sampling probabilities across early, middle, and late training, showing changing priorities from execution and verification toward debugging and efficiency. — **Adaptive data allocation.** Probabilities exclude discovery sampling. Early, middle, and late stages correspond to steps 1–50, 51–100, and 101–150.

Citation

If you find ARISE useful in your research, please cite our preprint.

@misc{feng2026arise,
  title={ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning},
  author={Feng, Kun and Fang, Yuchen and Tan, Yiyang and Gu, Shuqi and Zhao, Yongxiang
          and Liu, Yu and Lu, Xingyu and Ma, Lintao and Ren, Kan},
  year={2026},
  eprint={2609.35532},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.35532}
}