“Payroll professionals have spent years piecing answers together from government sites, union guidance, and dense regulatory text that leaves the final interpretation to them. An AI tool tested against expert-verified answers and current rates can shorten the path from question to resolution.”
Most AI benchmarks test models on questions written for the test. EmBench Payroll starts with the questions workers actually ask about their pay, covering everything from why a tax figure changed this month to how a bonus is taxed and what happens to unused leave on departure. These questions are grounded in Deel’s hands-on payroll expertise, built from seven years of serving 40,000+ customers in 150+ countries.
The benchmark covers 295 tasks across 8 countries, scored against 2,300 individual criteria. Each criterion is a specific point a correct answer has to include, so a model cannot earn credit for a fluent reply that skips the rule that decides the outcome.
Every question is answered and checked by Deel’s own payroll experts, the people who run payroll for 40,000+ customers in 150+ countries. They confirm the right answer, decide what a complete answer has to include, and check every point against the law. A second expert, working independently, then checks each answer key before it counts.
Payroll is local, which is why that expert layer is necessary. A model that understands US withholding can still miss how PAYE works in the UK, how provident fund rules apply in India, or how social security works in Germany. In this work, being nearly right is not good enough, because a wrong answer can mean someone does not get paid.
Payroll rules also change every year. This version of EmBench Payroll states the law as of 28 September 2026, and we will update it as tax rates, thresholds, and regulations change.
Deel is positioned to build this because payroll is work it carries out directly. Its experts know what a correct answer looks like in each country, and Deel finds, verifies, and pays them through its own contractor system. Anyone can write payroll questions, but few can say with confidence what the right answer is in every market.
Deel tested this approach on its own AI first. Emet helps build Pangaea, Deel’s family of LLMs, built on open-weight models for payroll and compliance, and Deel’s experts judged 2,163 tasks in Q3 across five internal projects.
Payroll is one of the first places where an AI error lands on a person’s income. EmBench Payroll shows labs where their models fall short, and Emet’s expert-verified data gives them a way to improve, so the people relying on AI for answers about their pay get the right ones.
To see how frontier models score on payroll and compliance, read the EmBench results at EmBench Payroll. AI labs and teams that want to improve a model on a specialized domain can talk to the Emet team.
