Scale Labs

Research to Advance AI

Scale Labs advances AI through research. Our research focuses on agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.

[PAPERS]

Research papers and publications covering agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.

Date Title
11/10/2026
To Critic Or Not To CriticPost-Training
10/23/2026
BARC-Bench: Benign Asks, Risky ConsequencesAgents
10/9/2026
DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language ModelsSafety & Oversight
10/7/2026
Humanity's Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal ModelsMultimodal
9/17/2026
SteerDuplex: Steerable Duplex Speech Dialogue ModelsMultimodal
9/4/2026
READY or Not: Reliable Enterprise Agent DeploymentAgents, Enterprise
View more

[BLOG]

Insights, analysis, and updates from Scale Labs

Physical AINov 17, 2026

Measuring Robot Data Diversity: Insights from 20,000 Hours of Real-World Data

We are collecting one of the largest corpora of highly diverse multi-embodiment robot data. Here, we analyze more than 20,000 hours of our loco-manipulation data and take a deeper look at what the robot does, what it handles, where it works, and how all of it looks to a model.

AgentsNov 12, 2026

BARC-Bench: When benign workplace requests lead AI agents to cause harm

When we delegate tasks to an AI agent, we often emphasize the task description and the overall goal of the task. We are rarely explicit about how an agent should safely complete the task and what boundaries the agent should adhere to. Often, these boundaries are often left unsaid because they’re common sense or only discoverable after exploring the workspace.

AgentsNov 12, 2026

To Jev or Not to Jev: When to Use System-1 Decision Models as LLM Judges

In September 2026, TypeSafe released Jev, a System-1 decision model that answers typed questions with calibrated probabilities in a single pass, without generating text. In our paper, "To Jev or Not to Jev: When Can System-1 Decision Models Replace LLM Judges?," we evaluate 14 decision models, including Jev 1.14, against four generative judges across 11 benchmarks to determine when they can replace generative judges, how to query them, and where they fall short.

Post-TrainingNov 10, 2026

To Critic or Not to Critic: GRPO vs PPO

As agentic tasks become more complex and rollouts become longer and more expensive, improving the efficiency of reinforcement learning becomes increasingly important. Critics can reduce gradient variance, allowing models to learn from fewer trajectories, but they also introduce additional compute costs and potential bias. Understanding when a critic should be applied remains an important open question.

View allAll posts