Senior Machine Learning Engineer, Applied AI Quality Bay Area, CA, US

Block, Inc.; California (MO); On-site; USD 194,500 - 325,900; Full time

Job summary

Block, Inc. is seeking a Senior Machine Learning Engineer to lead the technical direction of AI quality systems and to build scalable evaluation frameworks powered by LLMs and AI agents.

You will work across engineering, product, platform, and leadership to define long-term technical direction and surface actionable product insights. You will lead complex cross-functional initiatives, mentor engineers, and drive architecture that enables reliable, trustworthy AI across Block’s products and

Qualifications

• 5+ years of experience in software engineering, machine learning engineering or applied AI.
• Deep experience designing and shipping large-scale AI/ML systems in production environments.
• Strong expertise with LLMs, agents, evaluation systems, retrieval architectures, and modern AI infrastructure.
• Proven ability to lead ambiguous, high-impact technical initiatives from concept through adoption across multiple teams.
• Strong systems thinking and architectural judgment, with the ability to balance experimentation, scalability, and operational rigor.
• Experience defining technical strategy and influencing roadmaps beyond your immediate team.
• Excellent communication and cross-functional leadership skills, with the ability to align engineering, product, and organizational priorities.
• A track record of creating leverage through platforms, frameworks, and systems that enable other engineers and teams to move faster and make better decisions.

Responsibilities

• Lead the technical strategy and architecture for AI-driven quality and evaluation systems used across products and teams.
• Drive the development of scalable systems that use LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights.
• Define long-term approaches for evaluation, measurement, and quality intelligence across complex product surfaces.
• Translate ambiguous organizational needs into clear technical direction, roadmap priorities, and platform capabilities.
• Influence engineering standards and best practices for building reliable, measurable, and trustworthy AI systems.
• Lead complex cross-functional initiatives spanning product, infrastructure, data, and applied AI teams.
• Mentor and level up engineers across the organization through technical leadership, design reviews, and systems thinking.
• Identify leverage opportunities where AI systems can fundamentally improve how teams understand, debug, and improve product behavior.

Skills

5+ years exp

Large-scale AI

LLMs

Agents

Evaluation systems

Retrieval architectures

AI infrastructure

Technical leadership

Systems thinking

Technical strategy

Roadmap influence

Cross-functional leadership

Platforms & frameworks

Job description

Senior Machine Learning Engineer, Applied AI Quality

Remote Bay Area, CA, US

The Role

At Block, we believe product quality is foundational to great user experiences, and AI is transforming how we measure, understand, and improve that quality at scale. Our team builds the intelligence layer that evaluates system behavior across millions of real-world interactions, helping ensure our products are reliable, safe, and continuously improving.

We’re looking for a Senior Machine Learning Engineer to lead the technical direction of next‑generation quality systems powered by LLMs and AI agents. You’ll drive the architecture and strategy behind systems that evaluate product behavior, surface emerging issues, generate actionable insights, and enable teams across Block to make higher‑confidence product decisions.

In this role, you’ll operate across ambiguous, high‑impact problem spaces and shape how quality is measured and operationalized across the organization. You’ll work across engineering, product, platform, and leadership teams to define long‑term technical direction, establish scalable evaluation frameworks, and build systems that become foundational infrastructure for AI‑driven product quality.

You Will

• Lead the technical strategy and architecture for AI‑driven quality and evaluation systems used across products and teams.
• Drive the development of scalable systems that use LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights.
• Define long‑term approaches for evaluation, measurement, and quality intelligence across complex product surfaces.
• Translate ambiguous organizational needs into clear technical direction, roadmap priorities, and platform capabilities.
• Influence engineering standards and best practices for building reliable, measurable, and trustworthy AI systems.
• Lead complex cross‑functional initiatives spanning product, infrastructure, data, and applied AI teams.
• Mentor and level up engineers across the organization through technical leadership, design reviews, and systems thinking.
• Identify leverage opportunities where AI systems can fundamentally improve how teams understand, debug, and improve product behavior.

You Have

• 5+ years of experience in software engineering, machine learning engineering or applied AI.
• Deep experience designing and shipping large‑scale AI/ML systems in production environments.
• Strong expertise with LLMs, agents, evaluation systems, retrieval architectures, and modern AI infrastructure.
• Proven ability to lead ambiguous, high‑impact technical initiatives from concept through adoption across multiple teams.
• Strong systems thinking and architectural judgment, with the ability to balance experimentation, scalability, and operational rigor.
• Experience defining technical strategy and influencing roadmaps beyond your immediate team.
• Excellent communication and cross‑functional leadership skills, with the ability to align engineering, product, and organizational priorities.
• A track record of creating leverage through platforms, frameworks, and systems that enable other engineers and teams to move faster and make better decisions.

We’re working to build a more inclusive economy where our customers have equal access to opportunity, and we strive to live by these same values in building our workplace. Block is an equal opportunity employer evaluating all employees and job applicants without regard to identity or any legally protected class. We will consider qualified applicants with arrest or conviction records for employment in accordance with state and local laws and "fair chance" ordinances. We believe in being fair, and are committed to an inclusive interview experience, including providing reasonable accommodations to disabled applicants throughout the recruitment process. We encourage applicants to share any needed accommodations with their recruiter, who will treat these requests as confidentially as possible.

Block takes a market-based approach to pay, and pay may vary depending on your location. U.S. locations are categorized into one of four zones based on a cost of labor index for that geographic area. The successful candidate’s starting pay will be determined based on job‑related skills, experience, qualifications, work location, and market conditions. These ranges may be modified in the future. To find a location’s zone designation, please refer to thisresource . If a location of interest is not listed, please speak with a recruiter for additional information.

Zone A:

$217,300 — $325,900 USD

Zone D:

$194,500 — $291,700 USD