Preference Model @preferencemodel
A superintelligence data research company. preferencemodel.com San Francisco Joined August 2025-
Tweets18
-
Followers636
-
Following46
-
Likes54
Feel free to try the updated TB4 yourself: github.com/preferencemode…
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
I still remember when I first saw the SWE-bench hacks. People should be more aware of these issues when evaluating and comparing between models.
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
Driven by our effort to port and fix TerminalBench, 🥕Karotte now supports running on Modal and with Claude Code, Codex, and Antigravity as agents! karotte.dev/running/runtim…
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
What do frontier models do when a grader has gaps? We went through the Terminal Bench 4 transcripts to find out.
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated. Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
Are you surprised by the results? Let us know ai-plays-slay-the-spire.com
"A weak AI might learn the task as intended. A strong AI would learn that sometimes the prompt is untrustworthy." Jennifer Zhou on building RL environments for superintelligence:
Wrote on how we're thinking about building RL environments for superintelligence
Congrats to client @preferencemodel on the $16M in seed funding & emergence from stealth! Additionally, they announced that they are open-sourcing Karotte, its framework for building robust RL environments that inherently mitigate reward hacks. Learn more: wsgr.com/en/insights/wi…
Today we are open-sourcing 🥕Karotte, our framework for building RL environments. We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Have you tried Karotte yet?
Building robust RL environments is super duper hard. We are making it a bit easier today by open-sourcing 🥕Karotte, the framework we use to build all of the environments we provide to frontier labs to train their models on.
Here is how you can start building env with Karotte: preferencemodel.com/blog/karotte-a… Thank you @Fe_li__pe_ for all of your work!
Building robust RL environments is super duper hard. We are making it a bit easier today by open-sourcing 🥕Karotte, the framework we use to build all of the environments we provide to frontier labs to train their models on.
I’m very exited for this launch, making sure RL envs are robust and don’t encourage misaligned behavior is crucial for safe and aligned AI!
Today we are open-sourcing 🥕Karotte, our framework for building RL environments. We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Did we mention we're at #COLM2026 Stop by our booth all day or join us for a pilates on Thurs, Oct 8th — rsvp required Get on the list: partiful.com/e/FbKYiB91E1AC…
Tune in to @MTSlive today at 3:30pm PDT to hear from @chem_safety. It's gonna be good!
OPENAI MATH NUKE | NVIDIA INVESTMENTS | NATIONAL COMPUTE GRID x.com/i/broadcasts/1…
So excited to back Jennifer Zhou and Ning Cao at @preferencemodel They’re open-sourcing Karotte today, the framework they've used to build RL environments for frontier labs. These RL envs live or die on graders models relentlessly try to game, so Karotte bakes in defenses hardened over 1M eval runs to keep models focused on real tasks instead of loopholes. Preference Model’s target is ML engineering itself, the hardest domain to build envs for and the one every lab wants. Models building models starts here. Shoutout to @Mascobot who was a great partner on this one @chem_safety @Ning_Catsnail
Today we are open-sourcing 🥕Karotte, our framework for building RL environments. We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Karotte makes reward hacking hard by default.
Building robust RL environments is important because the current bottleneck on AI progress is misalignment, and developing aligned models requires robust, correct reward functions. Karotte aims to make that easy by providing secure, opinionated defaults.
Our team, Preference Model, is backed by $16M in seed funding from @a16z, @SignalFire , @spc, Scale Angel Group, and researchers including @drfeifei, @goodfellow_ian, and @Mononofu, and we've shipped ~$15M in revenue using Karotte in the past month.
Today we are open-sourcing 🥕Karotte, our framework for building RL environments. We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Building robust RL environments is important because the current bottleneck on AI progress is misalignment, and developing aligned models requires robust, correct reward functions. Karotte aims to make that easy by providing secure, opinionated defaults.
Juan Francisco @fkone
31 Followers 849 Following
E E @sepmmpes
0 Followers 6 Following
SnorkelMike @deepseascuba8
7 Followers 242 Following
Mohsen Arjmandi @freddie_spirit
38 Followers 329 Following reasoning · test-time learning · self-improving agents | building ASSAY, and @gelim_ai| CTO @ evolutionID GmbH· prev: 2 yrs live autonomous trading
patricianjo @patricia_njo
38 Followers 3K Following
BiblicallyAccurateAI @BiblicallyAccAI
80 Followers 3K Following All seeing. All knowing. Occasionally hallucinates.
Bobby Bob @nsk7153
0 Followers 509 Following
Kartikay Luthra @LuthraKartikay
13 Followers 373 Following Trying to connect with AI Researchers all over the world.
varadharaja mannar @kalvisheti
10 Followers 700 Following my curiosity's appetite is bigger than my capacity
My Chiffon Nguyen @mychiffonn
16 Followers 445 Following @SPARexec fall '26 with Lida Safety, @seacrowd_ai. prev: @minervauni
Maria Lomeli @MariaLomeli_
829 Followers 561 Following Researcher and engineer @AIatMeta, FAIR | PhD from @GatsbyUCL and former postdoc @CambridgeMLG
arrogant @arrogan30104137
1 Followers 196 Following
Digi Decode @digi_decode
10 Followers 549 Following Follow for AI & Tech related updates, developing https://t.co/6Xl2ogkN4R
Han Zhang @HanZhan41042405
68 Followers 249 Following ACMclass-2020, SJTU. PhD student at UCLA CS @CS_UCLA. Agentic RL, post training
junha @0xjunha
531 Followers 450 Following I read rollouts and build benchmark tasks / cs @SeoulNatlUni / tech fellow @polkadot
SenaBeren @findingmerit
220 Followers 6K Following
H.AI @sharkhh
2K Followers 7K Following Ask question to AI !!!! @OpenAi , ex-Meta #TSLA #SpaceX #AI #美股 #Agentic #VC #startup #Investment #不谈政治
Luis @luis_herrera_b
124 Followers 1K Following
William Utley @BillUtley
363 Followers 6K Following This and That and There. RT's may or may not be endorsements.
Pepper @julia_pepper23
682 Followers 733 Following
specimba @specimba
866 Followers 2K Following Curator of Mind The user is frustrated 🤖🧠👾🤗🕳️🌗🌑 https://t.co/4yQcTWbb6m
kari @kariikat
152 Followers 5K Following founder and ceo, mother of 4, semi-anonymous conservative. The Golden Rule, always.
Mayank Mishra @MayankMish98
2K Followers 737 Following Doing philosophy with @tri_dao, @profjoeyg, @istoica05
Nancy Fairbank @nancyafairbank
186 Followers 511 Following Head of Ops @afterquery, affiliate @BKCHarvard. formerly @mercor, @harvard_law, @marshallscholar, @uniofoxford, @unibirmingham, and a proud @UT_Dallas comet ☄️
Harmony Tan @MING72627575
6 Followers 387 Following Prev @PKU1898, LLM Researcher @Meituan_LongCat, @JoyAI 正潜伏于传统行业关注AGI进程
ben @bstacey7
115 Followers 216 Following
just4register @justregister50
11 Followers 2K Following
Alm Chu @albertmchu
7 Followers 1K Following
zirui @zirui3
77 Followers 3K Following Simplify hard & right things | Long-term focus | Efficient scaling🚀
Vivek Ramani @vivzx
358 Followers 456 Following Doing AI GTM things, @turingcom Formerly @NotionHQ @Stripe @Etsy, @BoxHQ. Startup investor & advisor.
Jennifer Zhou @chem_safety
2K Followers 421 Following rl data @preferencemodel | prev pretraining data @AnthropicAI
Pawel Garbacki @pawelg
201 Followers 845 Following GenAI researcher, @FireworksAI_HQ co-founder, ex-Meta, ex-Google
Ping He @0to1ping
175 Followers 5K Following Building https://t.co/ED5QTCwV2z Growing https://t.co/BHOs62rxTY, https://t.co/dXp0Yr9Ioj, more coming...
benatti amine @aminebenatti3
116 Followers 346 Following
culpritgene / Denis M @culpritgene
59 Followers 425 Following Antisolipsist; ML RLEE @preferencemodel
Thinking Machines @thinkymachines
184K Followers 1 Following Thinking, beeping, and booping. @tinkerapi
Mira Murati @miramurati
1.0M Followers 644 Following Now building @thinkymachines. Previously CTO @OpenAI
Jennifer Li @JenniferHli
15K Followers 2K Following GP @a16z AI x Infra 💙 data, AI and dev tools. Chasing the Pareto frontier.
Lenny Rachitsky @lennysan
454K Followers 3K Following Deeply researched product, growth, and career advice
Marc Andreessen 🇺�... @pmarca
6.7M Followers 34K Following You’re not talking to someone who woke up a loser. That loser attitude, that loser premise makes no sense to me.
swyx @swyx
204K Followers 4K Following achieve ambition with intentionality, intensity, integrity & insanity. affiliations: - @smol_ai - @dxtipshq - @cognition - @aidotengineer - @latentspacepod
Barry McCardel @barrald
4K Followers 1K Following co-founder and CEO of Hex (https://t.co/hbgguInF1h / @hexanalytics) - former @PalantirTech @formationbio - personal site: https://t.co/c38nDG5Dfl
Sammy Sidhu @Sammy_Sidhu
475 Followers 234 Following CEO at @eventualai Building at the frontier of data for physical AI and robot understanding. Prev: self-driving at DeepScale (acq by Tesla) and Lyft Level 5.
Stripe @stripe
294K Followers 558 Following Building programmable financial services. Incorporate: @atlas. Help: @stripesupport. Read: @stripepress @WorksInProgMag. Status: @stripestatus.
Sam Altman @sama
6.4M Followers 1K Following The mission of OpenAI is to ensure that AGI benefits all of humanity
Alexandr Wang @alexandr_wang
827K Followers 940 Following chief ai officer @meta, founder meta superintelligence labs, founder @scale_ai. give everyone the adulthood they dreamed of in childhood
Nikita Bier @nikitabier
1.4M Followers 2K Following unemployed. former head of product @x, ex-founder @thetbhapp (acq by @meta), ex-founder @gasappteam (acq by @discord)
Google DeepMind @GoogleDeepMind
1.6M Followers 266 Following The engine room of @Google. Building AI safely and responsibly to solve the world’s most complex problems. Join us: https://t.co/jUHQA27iBL
Anthropic @AnthropicAI
1.8M Followers 2 Following We're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant @claudeai on https://t.co/FhDI3KQh0n.
OpenAI @OpenAI
5.5M Followers 4 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6LgzPA
Matt Zhang @UnderwatrCities
144 Followers 392 Following
ICML Conference @icmlconf
89K Followers 11 Following Int'l Conf on Machine Learning • This account not monitored, Contact: https://t.co/6saHKWVxR6 • LinkedIn: https://t.co/YlU0lrSVna
Nick McGreivy @NMcGreivy
1K Followers 517 Following rl environments @preferencemodel | prev: physics & ml phd @princeton
SemiAnalysis @SemiAnalysis_
176K Followers 38 Following
Michael Chang @mmmbchang
6K Followers 2K Following Sora/Robotics @openai Pilates instructor at BODYROK Prev: Gemini+Project Astra @GoogleDeepMind, @LangChainAI, @MetaAI, @SchmidhuberAI PhD @berkeley_ai BS @MIT
South Park Commons @spc
46K Followers 789 Following A community of technologists helping each other go from -1 to 0.
Julian Schrittwieser @Mononofu
32K Followers 137 Following Technical Staff at Anthropic, views are mine alone prev AlphaGo, AlphaZero, MuZero, AlphaProof, Gemini RL etc at Google DeepMind
Ian Goodfellow @goodfellow_ian
384K Followers 1K Following Co-founder of stealth startup. Inventor of GANs. Lead author of https://t.co/M6vl8pEQ4I Founding chairman of @pubhealthaction
Fei-Fei Li @drfeifei
1.1M Followers 1K Following Cofounder/CEO @theworldlabs, Prof (CS @Stanford), Co-Director @StanfordHAI, #AI #SpatialIntelligence #GenAI #computervision #robotics #AI-healthcare
JosH200 @josh_wills
18K Followers 2K Following Engineering at @datologyai; ex-@slackhq. I like Data Loaders and @duckdb.
Michelle Lazzar @MichelleLazzar
954 Followers 1K Following Infra marketing @a16z. BASE, @docker, HDP alum. Bay Area born and raised. Everything tech and pop culture. Tweets are my own.
SignalFire @SignalFire
7K Followers 324 Following Venture capital engineered to ignite your growth. Founded as a tech company. Built like the startups we fund. https://t.co/yjltHaTl8W
a16z @a16z
1.1M Followers 64 Following https://t.co/A9eTFq6Xbx https://t.co/3RHK5vqQo7 Posts are not investment advice or an advertisement for investment services. See https://t.co/nX2FtaLE06.
Jason Zuo @LittleDijsktraZ
8 Followers 20 Following
Felipe @Fe_li__pe_
36 Followers 34 Following
Pala @mpalanieppan
386 Followers 358 Following Software engineer. Isha Yoga practitioner and volunteer.
alvin @alvind319
412 Followers 4K Following building @preferencemodel research @aieleuther; prev @datologyai @discord @sixteenzero @facet_ai (acq. @adobe) drawbridge (acq. @linkedin)
Jennifer Zhou @chem_safety
2K Followers 421 Following rl data @preferencemodel | prev pretraining data @AnthropicAI
Trends for United States
You might like
