Foundry
An AI development platform for building, evaluating, and deploying AI that decides whether content violates Meta's policies.
My Role
Design Lead
Team
1 x Researcher
1 x Content Designer
1 x Product Manager
10+ Engineers
Year & Scope
2026
0→1 platform
My Contribution
Product Strategy
Product Design
Front-end Implementation
IMPACT
BACKGROUND
$5B a year, and still not enough.
Meta relies on 45,000+ human reviewers worldwide to decide what content stays up and what comes down across its platforms, including Facebook and Instagram. Even with over $5B in annual cost, efficiency and accuracy were hard to guarantee, especially for nuanced decisions.
$5B
per year
45K
human reviewers
80 - 85%
accuracy ceiling
BIG BET
Automating 80% of human review with AI
Meta launched AI for Integrity to reduce dependency on human review while improving speed and accuracy. The initiative aimed to automate 40% of human review in H1 and scale toward 80% by year-end, while surpassing human reviewers in accuracy and speed.
AI for Integrity was a large cross-org effort spanning multiple workstreams. I led design for two of them. This case study focuses on Foundry, the platform used to develop, evaluate, and deploy AI automation.
Automate 80% of Human Review
40% in H1 • surpass human accuracy & speed
Other workstreams
Infrastructure & Policy
Foundry
AI Development Platform
Example Center
Training & Evaluation Data
APPROACH
Two problems stood between us and our big bet
Given an ambitious goal and a vague ask to build LLM development tooling, I started by mapping how LLMs were developed for content review. I used Meta’s internal AI assistant to synthesize an initial lifecycle, then validated and refined it with users and tech leads:
LLM development lifecycle for content review
Steps
1. Create
prompt
2. Configure
schema & model
3. Evaluate
prompt & model
4. Analyze
disagreement
5. Iterate
prompt & model
6. Deploy
prompt & model
Tools
Google Docs
RAPID
Bento Notebook
Google Sheets
Google Docs
Deploy tool
Phabricator
Users
ML engineer
Policy expert
Review expert
ML engineer
Review expert
ML engineer
Policy expert
Review expert
ML engineer
The map surfaced two major barriers to the 80% automation goal:
Fragmented tooling
Six disconnected tools, with manual handoffs between each
Scattered data with no visibility across tools
No real governance in a high-stakes domain
2+ month development cycle
Prompt creation takes ~1.5 months
Disagreement analysis takes ~0.5 months
Too slow to reach 80% automation goal
Every month of delay leaves harmful content live
SOLUTION
One platform, two AI agents
Fragmented tooling had a clear product path — I proposed consolidating the key functions across six disconnected stages into one platform.
Current state
Six disconnected tools
Google Docs
Bento
RAPID
Google Sheets
Deploy tool
Phabricator
consolidated
Future state
One platform: Foundry
One place for all
six stages
For the two longest stages, I partnered with engineering to define where AI automation could safely accelerate them:
For prompt creation, engineering built an auto-prompt agent that turned raw policy into a structured, AI-ready prompt. We tested it on two policy areas, and the agent reduced 5–7 weeks of manual work to 5 minutes, with equal or better quality.
Manual prompt creation
1.5 months of manual collaboration
AI-generated prompts
Full automation
5 minutes with proven quality
For disagreement analysis, performance was harder to validate upfront. Instead of delaying launch until validation was complete, I suggested shipping it as an opt-in experimental feature alongside the manual flow. Teams could benefit early where it worked, while we gathered the data to make it reliable.
Disagreement analysis
0.5 months of manual investigation
AI root-cause analysis
Opt-in automation
with human analysis as fallback
DESIGN
One system — from prompt to deployment
With the full picture in hand, I designed Foundry as one governed system for creating prompts, configuring models, evaluating results, analyzing disagreements, iterating, and deploying automation. Working with engineering, I landed 20+ AI-generated diffs, helping close the gap between design and shipped code.
A unified workflow for LLM-based content review
from fragmented steps to one seamless process
1. Create
prompt
2. Configure
schema & model
3. Evaluate
prompt & model
4. Analyze
disagreement
5. Iterate
prompt & model
6. Deploy
prompt & model
Users create governed projects by policy area, scope each project to one safety problem, then add prompts with access limited to certified users.
HUMAN-AI INTERACTION
Designing for when AI is wrong
As AI took on more decisions, the design challenge shifted from helping reviewers decide to helping them evaluate, challenge, and correct model decisions. I focused on two moments where trust mattered most:
Explainability: In the evaluation flow, reviewers investigated cases where the model disagreed with ground truth. I surfaced the model’s reasoning alongside each decision, making outputs easier to understand, validate, and debug.
Model reasoning appears beside each decision so reviewers can verify and debug disagreements.
Graduated autonomy: For root-cause analysis, I designed graduated autonomy instead of a simple trust-or-reject flow. Users could accept AI recommendations, refine them through conversation, or fall back to manual analysis when needed.
The root-cause agent helps users refine the analysis and understand its findings.
IMPACT
Beyond a tool: scaling AI development
Foundry became a core part of Meta’s AI for Integrity ecosystem, with impact across operational, business, and strategic levels.
Operational
2+ Months to Days
Reduced model iteration cycles from 2+ months to days by unifying a fragmented workflow.
Business
40% Review Automated
Automated 40% of human review in H1, with models outperforming reviewers.
Strategic
Compounding AI Progress
Benefits from future model improvements without redesigning the workflow.
WHAT'S NEXT
UI as control center for agents
For the Foundry MVP, we focused on the core functions teams needed to build models. As the tool matures, teams need more advanced prompting techniques — such as prompt chaining and multi-pass evaluation — to produce reliable results for increasingly complex enforcement decisions. Meanwhile, engineering is exploring ways to automate model development and deployment end to end through Claude skills.
I see a larger opportunity for Foundry to evolve into a control center for AI agents — a no-code surface where policy and review experts can guide agents across the model-development lifecycle, while engineering focuses on deeper technical challenges.