Description
Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma’s platform helps teams bring ideas to life—whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world. If you're excited to shape the future of design and collaboration, join us!
The Figma Research team is hiring an Director, Research - AI Evals to own how we measure the quality of Figma's AI-powered experiences. As Figma ships more AI capabilities across our products, the question "is this really good?" has never mattered more — and answering it rigorously is what this role exists to do. You'll define what "good" means for our AI features, build the frameworks and quality bars to measure it, and turn that into trusted signal that product teams rely on to decide what to ship.
The ideal candidate brings deep, hands-on experience evaluating AI/LLM-powered products — blending human evaluation with automated, model-based approaches — along with the product instinct and communication skills to make evaluation genuinely useful. Partnering with Product, Design, Engineering, and Data Science, you'll sit upstream of nearly every AI shipping decision at Figma and directly shape the quality of features used by millions of people.
This is a full time role that can be held from one of our US hubs or remotely in the United States.
What you'll do at Figma:
- Own AI evaluation methods and operations for Figma's AI-powered experiences — define quality dimensions, design how we measure them, and turn results into decision-ready signal
- Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars, combining human evaluation with automated/model-based approaches (e.g., LLM-as-judge) where appropriate
- Partner with engineering to stand up repeatable, reproducible evaluation pipelines and regression testing, so evaluation is a routine part of how AI features are built and shipped
- Produce clear readouts and dashboards that let stakeholders confidently make go/no-go and prioritization decisions
- Socialize a shared definition of quality so evaluation standards are adopted across teams rather than re-invented — and advocate for evaluation as a strategic partner in the product process
- Manage a small team to execute our AI evals in partnership with contractors, internal staff, and/or LLMs
We'd love to hear from you if you have:
- 10+ years of experience in product, research, applied research, or a closely related field, including 2+ years of management experience
- Direct, hands-on experience owning the evaluation of AI/LLM-powered products
- Expertise designing and running AI evaluation — human evaluation programs, rubric and benchmark/golden-dataset construction, inter-rater reliability — and sound judgment about when and how to apply automated/model-based approaches (e.g., LLM-as-judge), including their limitations
- Strength across both qualitative and quantitative methods, comfort with data and metrics, and the ability to reason about model behavior
- Demonstrated success in identifying the riskiest assumptions behind an ambiguous quality question, prioritizing them, and designing right-sized evaluation to build confidence
- A proven track record of gaining buy-in from executive and cross-disciplinary stakeholders — transcending methodology to articulate a larger user story and the "so what" to inspire action
While it's not required, it's an added plus if you also have:
- Experience building or co-building automated evaluation pipelines and regression testing in partnership with engineering, or familiarity with eval tooling (e.g., Braintrust, LangSmith, DeepEval, or equivalents)
- Experience standing up a new function, practice, or discipline from scratch
- 2+ years in product design, user-centric product management, data science, product development, and/or front-end engineering
- A familiarity and depth of experience using Figma's products
Pay Transparency Disclosure
Job level and actual compensation will be decided based on factors including, but not limited to, individual qualifications objectively assessed during the interview process (including skills and prior relevant experience, potential impact, and scope of role), market demands, and specific work location.
Figma offers equity to employees, as well as a competitive package of additional benefits, including health, dental, and vision coverage; retirement benefits with company contributions; parental leave and reproductive or family planning support; mental health and wellness benefits; and paid time off. Figma provides paid sick leave, holidays, and other leave benefits in compliance with applicable federal, state, and local laws, including the requirements of the Washington Minimum Wage Act and related regulations. Exempt employees are eligible for employer‑provided paid flexible PTO in addition to flexible paid sick leave. PTO is subject to manager approval. Additional benefits may include company recharge days, cell phone and home internet reimbursements, and a number of lifestyle spending accounts. Figma also offers sales incentive compensation for most sales roles and an annual bonus plan for eligible non-sales roles. All compensation and benefits are subject to applicable plan terms and may be modified by Figma at any time, consistent with applicable law.
At Figma we celebrate and support our differences. We know employing a team rich in diverse thoughts, experiences, and opinions allows our employees, our product and our community to flourish. Figma is an equal opportunity workplace - we are dedicated to equal employment opportunities regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity/expression, veteran status, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.
We will work to ensure individuals with disabilities are provided reasonable accommodation to apply for a role, participate in the interview process, perform essential job functions, and receive other benefits and privileges of employment. If you require accommodation, please reach out to accommodations-ext@figma.com. These modifications enable an individual with a disability to have an equal opportunity not only to get a job, but successfully perform their job tasks to the same extent as people without disabilities.
Examples of accommodations include but are not limited to:
- Holding interviews in an accessible location
- Enabling closed captioning on video conferencing
- Ensuring all written communication be compatible with screen readers
- Changing the mode or format of interviews
To ensure the integrity of our hiring process and facilitate a more personal connection, we require all candidates keep their cameras on during video interviews. Additionally, if hired you will be required to attend in person onboarding.
By applying for this job, the candidate acknowledges and agrees that any personal data contained in their application or supporting materials will be processed in accordance with Figma's Candidate Privacy Notice.
Similar jobs
Est. 170,000 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 292,000 USD
About Scale AI Scale AI accelerates the development and deployment of AI, from the high-quality training data and evaluation services that power the world's leading AI labs, to full-stack AI applications built for Enterp…
Est. 119,000 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 279,400 USD
Staff Software Engineer, Full-Stack - Enterprise Gen AI Scale GP (Scale Generative AI Platform) is an enterprise-grade AI platform providing APIs for knowledge retrieval, inference, evaluation, and more. We are looking f…
Est. 274,400 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 274,400 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 159,300 USD
Scale AI's Advisory practice is the most forward-leaning bet in our go-to-market motion. We're not waiting for clients to hand us a scope — we get in front of the decision, shape the right AI problems before a build begi…
Est. 154,800 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 139,000 USD
Scale AI’s customer-facing engineering organization is one of the fastest-growing and most technically demanding in enterprise AI. Frontier Agentic Engineers sit at the center of our most strategic relationships; buildin…
Est. 201,825 USD
About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative…
Est. 255,600 USD
About Scale Scale’s mission is to develop reliable AI systems for the world’s most important decisions. As the leading AI data foundry, we provide the high-quality data and full-stack technologies that power the world’s…
Est. 202,500 USD
Scale plays a critical role in powering the next generation of AI applications across the enterprise. As our customer base grows exponentially, Principal Solutions Engineers operate as senior technical leaders on the fro…
Est. 140,000 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 270,000 USD
Role Summary Scale is building the agentic operating system that powers AI deployment across the world's most important organizations. The Platform PM is the person who determines what that foundation needs to be — and h…
Est. 184,400 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 154,800 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 283,500 USD
Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignm…
Est. 85,000 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 243,225 USD
About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative…
Est. 202,500 USD
Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, perso…
Est. 170,000 USD
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 283,500 USD
About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with leading enterprises and government organizations to accelerate their AI i…
Est. 283,500 USD
Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, perso…
Est. 324,000 USD
About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with leading enterprises and government organizations to accelerate their AI i…
Est. 120,000 GBP
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital…
Est. 233,100 USD
The Enterprise Account Executive (Financial Services) will report to the Director of Enterprise GTM and own revenue growth across a portfolio of Scale AI’s most strategic financial services customers and prospects. This…
Est. 320,850 USD
About Scale AI Scale’s mission is to develop reliable AI systems for the world’s most important decisions. We work with leading enterprises to bring generative AI safely into production—powering real-world impact through…
Est. 243,225 USD
Scale GP (Scale Generative AI Platform) is an enterprise-grade Generative AI platform providing APIs for knowledge retrieval, inference, evaluation, and more. We are seeking a strong Senior Full-Stack Engineer to help us…
Est. 231,300 USD
Role Summary Scale is accelerating the development of AI across the world's most important organizations. The Forward Deployed PM is the person who makes enterprise deployments succeed from the product side — embedded wi…