Portrait of Isabella Beltran Shapland

Machine Learning Researcher

Isabella Beltran Shapland

I work on RL alignment, visual reasoning, and model reliability. Lately that means teaching vision-language models to keep looking at the image.

Open to research roles in AI safety and model behavior

about

I study how to make imperfect signals from inside a model safe to use in RL. My current work puts visual-attention probes into GRPO rewards for vision-language models, with constraints that let the probe break ties but never override correctness.

I did my B.S. in Computer Science at UC Santa Cruz (2022–2026), where I pitched and first-authored a GRPO research paper in Dr. Yuyin Zhou's lab. Before that I worked on explainable ARC-AGI reasoning at TrainLoop (YC W25) and on on-device LLM tooling at Intel. Lately I've been spending my own time on interpretability for safety: faithfulness, reward hacking, and eval awareness.

research interests

news

research

  1. Break the Loop with Good Attention: Visual-Attention-Based Probes in GRPO Rewards to Mitigate Repetition Collapse

    Isabella Beltran Shapland, Hardy Chen, Cihang Xie, Yuyin Zhou

    Committed to EACL 2027 · ACL Rolling Review, Aug 2026

    Reasoning VLMs trained with GRPO can fall into repetition collapse when their attention drifts off the image. A frozen probe on visual attention (0.796 AUROC) is used only to break reward ties, never to override task correctness; we call this FT-GRPO. It also recovers a learning signal from fully tied groups, where vanilla GRPO gives no gradient at all.

    A model answers the same question 8 times. Training then pushes each answer right (do more of this) or left (do less). Bar length is how hard.

    Question: Read a graph of y = −6·log₄(x − 1) + a that passes through (5, −3), and find the function.

    ✓ right✗ wrongright answers always stay above wrong onesClick After to see what changes
    Right, then 1,300 words doubting the graph
    Plugs in (5, −3), solves, answers
    Short and confident, but reads (5, 3)
    Reads (5, −3), solves, checks once
    Right after ~2,000 words and 41 “wait”s
    Reads (5, −3), solves, verifies
    Reads (5, 3), then patches around it
    Right, after 21 “wait”s

    Plain GRPO only knows right or wrong. All six right answers get the same push, so the one that took ~2,000 words and 41 “wait”s is rewarded exactly as much as the 214-word one.

    The "push" is GRPO's advantage. These are real answers from Qwen3-VL-8B-Thinking before RL, on questions the probe never trained on, with the frozen probe's real scores; the pushes use the paper's formulas (Eq. 3, 8, 9; α = 0.5).

    View as table
    AnswerResultLooks at imagePush beforePush after
    Right, then 1,300 words doubting the graph✓ right0.35+0.54+0.22
    Plugs in (5, −3), solves, answers✓ right0.70+0.54+0.65
    Short and confident, but reads (5, 3)✗ wrong0.81−1.62−1.08
    Reads (5, −3), solves, checks once✓ right0.80+0.54+0.86
    Right after ~2,000 words and 41 “wait”s✓ right0.28+0.540.00
    Reads (5, −3), solves, verifies✓ right0.82+0.54+1.08
    Reads (5, 3), then patches around it✗ wrong0.66−1.62−2.16
    Right, after 21 “wait”s✓ right0.45+0.54+0.43
    • +3.4 in-domain accuracy
    • +6.3 on MathVision
    • ~⅓ less repetition
    • Qwen3-VL-8B, full fine-tune

    [pdf]

  2. ARC-AGI v2: Explainable Reasoning Traces for GPT-5-Nano; The Execution Bottleneck

    Isabella Beltran Shapland, Jackson Stokes

    TrainLoop research report, 2025

    GPT-5-nano solves ARC-AGI v2 puzzles by writing illustrated instruction booklets made of IF-THEN steps. Its reasoning was fully correct on 45.5% of 33 human-annotated puzzles, compared with 0.9% raw benchmark accuracy. Execution then failed in 93.8% of those cases, so the bottleneck is visual execution, not reasoning.

    • 45.5% correct reasoning
    • 93.8% execution failure
    • failure-mode taxonomy

    [pdf]

experience

  1. UC Santa Cruz

    Jan 2026 – Present

    First-Author ML Researcher · Dr. Yuyin Zhou's lab · Santa Cruz, CA

    • Pitched the project in office hours, then built all of it: the Verl training setup, RL configs, data ingestion, controls, and multi-arm ablations on 8×A100 and 2×H200 (the paper's 8B runs are all on 2×H200).
    • Designed the tie-only injection method and showed that simply adding the probe to the reward lets confident wrong answers outrank right ones (the tie-break was right on 16 of 21 in-domain disagreements).

  2. TrainLoop (YC W25)

    Sept 2025 – Dec 2025

    ML Research Intern · San Francisco, CA

    • Built a 6-stage pipeline that turns multimodal puzzle-solving into auditable reasoning booklets with a crop-transform-uncrop image loop.
    • Designed the annotation schema and failure-mode taxonomy that made reasoning and execution measurable separately.

    [paper]

  3. Intel

    May 2025 – Sept 2025

    ML Software Engineering Intern · Santa Clara, CA

    • Built a Haystack RAG pipeline (LLaMA-3.2) that pulls CPU anomalies from large JSON system logs, plus a deterministic Python validator to catch hallucinations.
    • Wrote Intel's first internal benchmarks for local model hosting on integrated GPUs (Ollama, SYCL).

    [BLDT v3.0 write-up]

  4. ACM at UCSC

    Sept 2023 – June 2024

    ML Researcher · Santa Cruz, CA

    • Tested BERT against gradient-based Universal Trigger Token attacks and developed a new robustness measure from the results.
    • Ran the attack simulations in parallel across GPU nodes with Kubernetes and PyTorch.

projects

talks

misc

work in progress

An independent interpretability project on AI safety and model behavior. It's ongoing, so I describe the question and method here, not results.