Research to Advance AI
Scale Labs advances AI through research. Our research focuses on agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.
[LEADERBOARDS]
Benchmarks for frontier, agentic, and safety capabilities
[PAPERS]
Research papers and publications covering agents, post-training, reasoning, safety, evaluation, and alignment, and the science of data.






To Critic Or Not To Critic
[BLOG]
Insights, analysis, and updates from Scale Labs
Measuring Robot Data Diversity: Insights from 20,000 Hours of Real-World Data
We are collecting one of the largest corpora of highly diverse multi-embodiment robot data. Here, we analyze more than 20,000 hours of our loco-manipulation data and take a deeper look at what the robot does, what it handles, where it works, and how all of it looks to a model.
BARC-Bench: When benign workplace requests lead AI agents to cause harm
When we delegate tasks to an AI agent, we often emphasize the task description and the overall goal of the task. We are rarely explicit about how an agent should safely complete the task and what boundaries the agent should adhere to. Often, these boundaries are often left unsaid because they’re common sense or only discoverable after exploring the workspace.
To Jev or Not to Jev: When to Use System-1 Decision Models as LLM Judges
In September 2026, TypeSafe released Jev, a System-1 decision model that answers typed questions with calibrated probabilities in a single pass, without generating text. In our paper, "To Jev or Not to Jev: When Can System-1 Decision Models Replace LLM Judges?," we evaluate 14 decision models, including Jev 1.14, against four generative judges across 11 benchmarks to determine when they can replace generative judges, how to query them, and where they fall short.
To Critic or Not to Critic: GRPO vs PPO
As agentic tasks become more complex and rollouts become longer and more expensive, improving the efficiency of reinforcement learning becomes increasingly important. Critics can reduce gradient variance, allowing models to learn from fewer trajectories, but they also introduce additional compute costs and potential bias. Understanding when a critic should be applied remains an important open question.
View allAll posts