Machine Learning Engineer

Coda; Porto; Hybrid; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Hybrid work model

Home office setup support

Wellness stipends

Learning and development opportunities

Relocation support to Berlin

Job summary

Superhuman is hiring a foundational Machine Learning Engineer to join our Porto team and build the evaluation foundation behind the next generation of AI-native productivity on our agent platform. You will own how we define, measure, and improve quality, building eval datasets, the infrastructure around them, and the metrics that tell us whether our models and agents are actually getting better for users.

This is a high-autonomy, high-impact role on a foundational team that moves fast from idea

Qualifications

• 5+ years of industry experience as a machine learning engineer or applied research scientist.
• Strong foundation in classic ML and NLP, including experimental design, statistics, and metric design, plus working knowledge of modern LLM approaches.
• Hands-on experience building evaluation datasets and the tooling and infrastructure to run evals at scale (data pipelines, labeling workflows, benchmarking frameworks).
• Experience using evaluation results to measure and improve model or product quality, whether for ranking, classification, or LLM-based systems.
• Strong Python and data engineering skills, with experience in reproducible, production-grade ML pipelines.
• Deep care about the end-user experience and a drive to ensure high quality.
• Ability to work independently with minimal guidance, manage priorities across multiple projects, and collaborate effectively with cross-functional teams in a fast-paced, results-driven environment.
• Nice to have: experience with LLM-as-judge, agent or workflow evaluation, or search/ranking systems; MS/PhD in Computer Science or a related field.

Responsibilities

• Build high-quality evaluation datasets: Design, source, and curate golden sets, labeled data, and synthetic data for agent behaviors, search ranking, and proactive suggestions.
• Build the eval infrastructure: Create scalable, reproducible pipelines for running offline evaluations, regression testing, and benchmarking across models, prompts, and agent configurations.
• Measure and improve quality: Define metrics that reflect real user value, from classic ML metrics to LLM-as-judge and human-eval approaches.
• Own classic ML systems: Develop and deploy ML models for core product experiences, including search ranking and proactive suggestions that anticipate user needs.
• Support the agent platform: Partner with the team building orchestration, routing, and planning to define success criteria and quality gates for multi-agent workflows.
• Close the loop with production: Connect offline evals to online experiments and real-world signals, so we catch regressions early and learn from user experiences.
• Ship fast: Move quickly from idea to impact, with no red tape or endless approvals, on a platform used by millions.

Skills

ML engineering

NLP fundamentals

Experimental design

Statistics

Metric design

Python

Data engineering

Independent work

Education

MS/PhD in Computer Science or related field

Tools

LLM evaluation

Job description

Superhuman offers a dynamic hybrid working model for this role. This flexible approach gives team members the best of both worlds: plenty of focus time along with in-person collaboration that helps foster trust, innovation, and a strong team culture.

About Superhuman

Superhuman is the AI productivity platform on a mission to unlock the superhuman potential in everyone. The Superhuman suite of apps and agents brings AI wherever people work, integrating with over 1 million applications and websites. The company’s products include Grammarly’s writing assistance, Superhuman Docs’s collaborative workspaces, Mail’s inbox management, and Go, the proactive AI assistant that understands context and delivers help automatically. Founded in 2009, Superhuman empowers over 40 million people, 50,000 organizations, and 3,000 educational institutions worldwide to eliminate busywork and focus on what matters. Learn more at superhuman.com and about our values here.

The Opportunity

We’re looking for a foundational Machine Learning Engineer to join our team in Porto and build the evaluation foundation behind the next generation of AI-native productivity at Superhuman, specifically our agent platform, where multiple AI agents collaborate to solve complex user problems. You’ll own how we define, measure, and improve quality. You’ll build the eval datasets, the infrastructure around them, and the metrics that tell us whether our models and agents are actually getting better for users.

This is a high-autonomy, high-impact role on a foundational team that moves fast from idea to production. As Superhuman Docs’ collaborative workspaces evolve so users can solve arbitrarily complex tasks with agents, rigorous measurement will be what lets us ship with confidence.

• In this role, you will:

  • Build high-quality evaluation datasets: Design, source, and curate golden sets, labeled data, and synthetic data for agent behaviors, search ranking, and proactive suggestions. Define annotation guidelines, manage labeling workflows, and ensure datasets are representative, versioned, and trustworthy.

  • Build the eval infrastructure: Create scalable, reproducible pipelines for running offline evaluations, regression testing, and benchmarking across models, prompts, and agent configurations. Make it easy for every team to run evals and trust the results.

  • Measure and improve quality: Define metrics that reflect real user value, from classic ML metrics (precision/recall, NDCG, calibration) to LLM-as-judge and human-eval approaches. Validate that automated metrics correlate with user outcomes, and use the results to drive model and product decisions.

  • Own classic ML systems: Develop and deploy ML models for core product experiences, including search ranking and proactive suggestions that anticipate user needs, with evaluation built in from day one.

  • Support the agent platform: Partner with the team building orchestration, routing, and planning to define success criteria and quality gates for multi-agent workflows.

  • Close the loop with production: Connect offline evals to online experiments and real-world signals, so we catch regressions early and learn from what users actually experience.

  • Ship fast: Move quickly from idea to impact, with no red tape or endless approvals, on a platform used by millions.

Qualifications

• 5+ years of industry experience as a machine learning engineer or applied research scientist.

• Strong foundation in classic ML and NLP, including experimental design, statistics, and metric design, plus working knowledge of modern LLM approaches.

• Hands-on experience building evaluation datasets and the tooling and infrastructure to run evals at scale (data pipelines, labeling workflows, benchmarking frameworks).

• Experience using evaluation results to measure and improve model or product quality, whether for ranking, classification, or LLM-based systems.

• Strong Python and data engineering skills, with experience in reproducible, production-grade ML pipelines.

• Deep care about the end-user experience and a drive to ensure high quality.

• Ability to work independently with minimal guidance, manage priorities across multiple projects, and collaborate effectively with cross-functional teams in a fast-paced, results-driven environment.

• Nice to have: experience with LLM-as-judge, agent or workflow evaluation, or search/ranking systems; MS/PhD in Computer Science or a related field.

Compensation and Benefits

• Professional Growth: We believe that autonomy and trust are key to empowering our team members to do their best, most innovative work in a way that aligns with their interests, talents, and well-being

• A connected team: Superhuman builds a product that helps people connect, and we apply this mindset to our own team. Our remote-first, hybrid model enables a highly collaborative culture rooted in our values. We work to foster belonging among team members in a variety of ways. This includes our team member resource groups, Superhuman Circles, which promote connection among those with shared identities including BIPOC and LGBTQIA+ team members, women, and parents. We also celebrate our colleagues and accomplishments with global, local, and team-specific programs.

• Comprehensive benefits for candidates based in Portugal: Superhuman offers all team members competitive pay along with a benefits package encompassing life care (including mental health care and risk benefits) and ample and defined time off. We also offer support to set up a home office, wellness and pet care stipends, learning and development opportunities, and more.

• Relocation Support: Superhuman provides relocation support to make your move to Berlin seamless. Our package includes visa assistance, destination services to help you and your family settle in comfortably, and a relocation bonus to cover additional expenses, such as temporary housing.

We encourage you to apply

At Superhuman, we value our differences, and we encourage all—especially those whose identities are traditionally underrepresented in tech organizations—to apply. Superhuman is an equal opportunity company. We do not discriminate on the basis of race or ethnic origin, religion or belief, color, gender expression or identity, sexual orientation, sexual identity, national origin, citizenship, age, marital status, veteran status, disability status, criminal prosecution, judgment in a criminal case, or any other characteristic protected by law.

For more details about the personal data Superhuman collects during the recruitment process, for what purposes, and how you can address your rights, please see the Superhuman Data Privacy Notice for Candidates here.