Senior AI/ML Engineer New Lisbon

Pager; Lisboa; On-site; EUR 100,000 - 140,000; Full time

Job summary

PagerDuty is seeking a Senior AI/ML Engineer to design and ship AI systems at scale, powering incident management agents, event intelligence, and LLM-powered capabilities across the platform. You will own the full lifecycle from problem framing to production serving, working with cross-functional teams to integrate AI responsibly and reliably.

You will build with modern AI, including LLMs, agents, and retrieval, and ensure systems stay high-throughput, fault-tolerant, and cost-conscious.

Qualifications

• 5+ years of software engineering experience with production distributed systems.
• Hands-on experience building and shipping AI systems in production.
• Strong programming fundamentals.
• Applied AI fundamentals: prompting, retrieval, agent patterns, and guardrails.
• Experience with cloud infrastructure (AWS, GCP, or Azure), containers, and Kubernetes.
• Reliability-minded mindset and ability to optimize for scalability and correctness.
• Strong communication and collaboration skills.

Responsibilities

• Design and build AI-powered features for high-volume, real-time event streams.
• Architect agent orchestration, retrieval pipelines, tool/API integrations, and low-latency inference.
• Ensure consistency, throughput, fault tolerance, and cost efficiency across services.
• Move AI features from prototype to production with robust evaluation and guardrails.
• Collaborate with platform, product, and applied-research teams to define success.
• Mentor teammates and shape the team's technical direction.

Skills

Distributed systems

AI in production

Kubernetes

LLMs / AI agents

Cloud infrastructure

Communication

Job description

PagerDuty (NYSE:PD) is a leader in Digital Operations Management. In an always-on world, organizations of all sizes trust PagerDuty to help them deliver a perfect digital experience to their customers, every time. Teams use PagerDuty to identify issues and opportunities in real time and bring together the right people to fix problems faster and prevent them in the future. Over 13,000 organizations (including 60 of Fortune 100) rely on PagerDuty to succeed with Digital Transformation, Cloud Migration, and DevOps Modernization. Notable customers include GE, Cisco, Genentech, Electronic Arts, Cox Automotive, Netflix, Shopify, Zoom, DoorDash, Lululemon and more. We are expanding rapidly as a platform for Digital Operations Management using AI/ML and Automation and growing our adoption by Development, IT, Customer Service, Security, and other teams across the organization.

About the role

PagerDuty’s Operations Cloud runs on a platform that ingests billions of signals and turns them into real-time action for thousands of customers. We’re looking for a Senior AI/ML Engineer who lives at the intersection of two disciplines: large-scale distributed systems and applied AI.

In this role you will design and ship AI systems that run in production at PagerDuty’s scale — powering Incident Management AI Agents, event intelligence, and the LLM-powered capabilities embedded across our platform. You’ll own the full lifecycle, from framing the problem to serving reliably at scale.

We are looking for a candidate who is genuinely passionate about building with modern AI — LLMs, agents, and retrieval — but grounded in the realities of building resilient, high-throughput systems.

What you’ll do

• Design and build AI-powered features — LLM agents, retrieval, and event intelligence — that operate on high-volume, real-time event streams, from problem framing through production deployment and monitoring.
• Architect and own the systems behind them: agent and prompt orchestration, retrieval pipelines, tool/API integrations, and low-latency inference and evaluation at scale.
• Reason about consistency, throughput, fault tolerance, and cost across services that must stay reliable under bursty, unpredictable load.
• Take AI features from prototype to production, establishing the evaluation, guardrail, observability, and improvement loops that keep them accurate and trustworthy over time.
• Partner with platform, product, and applied-research teams to define what “good” looks like and to integrate AI cleanly into existing services.
• Raise the bar through example, reviews and mentorship, and help shape the team’s technical direction.

What you’ll bring

• 5+ years of software engineering experience, with meaningful time spent building and operating production distributed systems (high-throughput services, streaming/event-driven architectures, or large-scale data platforms).
• Hands-on experience building and shipping AI systems in production — LLM-powered applications, agents, or retrieval — including the surrounding orchestration, serving, and evaluation, not just prototypes.
• Strong programming fundamentals and comfort moving between systems and AI/application code.
• Solid grounding in applied AI fundamentals: prompting, retrieval, agent patterns, and how to evaluate and guardrail LLM behavior.
• Experience with cloud infrastructure (AWS, GCP, or Azure), containers, and orchestration (Kubernetes).
• A pragmatic, reliability-minded mindset: you optimize for systems that work correctly at scale, and you can articulate the trade-offs behind your choices.
• Strong communication and collaboration skills, and a track record of raising the quality of the teams and systems around you.

Nice to have

• Experience with LLMOps tooling and patterns — evaluation harnesses, prompt/version management, tracing and observability for agents, and online/offline eval consistency.
• Deep experience serving LLM-based systems in production, including retrieval-augmented generation, multi-step agents, and tool use.
• Background in anomaly detection, event correlation, or applied problems in observability, AIOps, or reliability.
• Familiarity with the ecosystem — e.g. LLM APIs and frameworks such as LangChain or LlamaIndex, vector databases, and distributed data/compute tools such as Kafka, Airflow, or Spark.
• Contributions to open-source AI or distributed-systems projects.

Why PagerDuty

At PagerDuty, AI it’s core to how we help the world’s teams keep their digital services running. You’ll work on problems where scale, latency, and correctness genuinely matter, alongside engineers who care about building systems that people depend on in their most critical moments.

Where we work

PagerDuty operates a hybrid work model with offices in 8 major cities: Atlanta