Staff AI Engineer

mymoose; United States; On-site; USD 180,000 - 260,000; Full time

Job summary

Rapid7 is seeking a Staff AI Engineer to design, build, and ship LLM and agentic AI systems across security products. You will own technical solutions, provide architectural guidance, and mentor engineers while balancing quality, latency and cost in production deployments.

You will work hands-on with transformer internals, fine-tuning, distillation, and evaluation, shaping the AI strategy for Detection and Response and Exposure and Remediation areas.

Qualifications

• 8+ years of software or ML engineering experience
• 2+ years building production LLM or agentic AI systems
• Master's degree preferred or equivalent experience
• Track record delivering scalable AI systems across teams

Responsibilities

• Design, build and ship LLM and agentic AI systems end-to-end
• Lead design reviews and technical roadmaps for AI systems
• Fine-tune and post-train open-weight LLMs for security data
• Optimize inference for quality, latency and cost
• Mentor engineers and partner with product teams

Skills

Production AI systems

Leadership

LLM knowledge

PyTorch

Inference optimization

Context engineering

Python

Mentoring

Education

Master's degree in CS/ML

Tools

PyTorch

vLLM

TensorRT-LLM

DeepSpeed

Job description

Our Engineering organization is scaling its AI Engineering capabilities under new leadership, creating an opportunity to shape the next generation of AI at Rapid7. We're looking for Staff AI Engineers to drive the design and delivery of sophisticated LLM and agentic AI systems, influencing technical direction across teams and solving complex, cross-functional challenges. You'll establish engineering best practices, guide architectural decisions and mentor other engineers, helping translate customer needs into scalable AI solutions that advance our product and platform capabilities.

About the Team

AI is reshaping cybersecurity on both sides. Attackers can use AI to strike at machine speed, and only AI-powered defense can keep up. At the same time, organizations can only adopt AI with confidence if it is secure. Rapid7 sits at the intersection of security and AI, and we are building industry-defining products for this new era.

Rapid7's AI Engineering team builds the AI that powers our security platform. We are AI engineers and applied scientists who take models from research to production, combining open-source and frontier LLMs, AI agents, and fast predictive models with more than 20 years of Rapid7 security expertise. Our work drives core use cases across Detection and Response, such as alert triage, threat investigation, and automated response, and Exposure and Remediation, such as vulnerability prioritization, attack surface risk assessment, and guided remediation. Together, these help customers stop threats faster and fix what matters most.

About the Role

As a Staff AI Engineer, your primary responsibility will be to design, build and ship LLM and agentic AI systems across Rapid7's security products, owning the technical solutions for your area end to end. You understand LLMs from transformer internals to fine-tuning and inference optimization, stay hands-on with hard problems, and know where LLMs shine and where they fall short, choosing the right approach among LLMs, predictive AI and classical ML. Specifically, your focus will be to:

• Lead the design and architecture of LLM and agentic AI systems within Detection and Response and Exposure and Remediation, contributing to the technical roadmap for your area


• Recommend model strategy for your projects, including when to use frontier or open-source models and when to fine-tune, distill, or post-train


• Fine-tune, distill and post-train open-source LLMs on security data, from data curation through training, evaluation and deployment


• Apply and improve engineering practices for context engineering, inference optimization and evaluation, optimizing LLM inference to balance quality, latency and cost


• Establish evals as the release gate for the AI systems you ship


• Lead design reviews for your projects and work through complex technical trade-offs


• Stay hands-on by prototyping and building critical model, training and inference code


• Mentor engineers on your team and partner with Product and platform teams to align technical solutions with customer needs

The skills and qualities you'll bring include:

• 8+ years of software or ML engineering experience including 2+ years building production LLM or agentic AI systems. A Master's degree in Computer Science, Machine Learning, AI or a related field is preferred, or equivalent practical experience


• A track record of leading the delivery of scalable, production-grade AI systems used by multiple teams or products, with impact across the organisation through technical leadership and mentoring


• A deep understanding of LLMs, including transformer architecture, attention, tokenization and positional encodings, and how models are trained and served


• Advanced PyTorch skills, including custom model code, training loops, mixed precision and distributed training (e.g., FSDP, DeepSpeed)


• Hands-on fine-tuning and post-training of open-weight LLMs, including full and parameter-efficient methods (LoRA, QLoRA), distillation, and preference or RL-based methods (e.g., DPO, GRPO)


• Experience with context engineering (retrieval, memory, and context management for long-running agent workflows) and defining evaluation strategy and benchmarks for LLM and agent systems


• Strong knowledge of inference optimization, including prefill and decode, KV-cache management, continuous batching, speculative decoding and quantization, and serving at scale with vLLM, SGLang or TensorRT-LLM


• Strong predictive AI, classical ML, distributed systems and Python fundamentals, with the judgment to make efficient, well-reasoned decisions between these approaches and LLMs based on accuracy, latency and cost


• Lead AI projects or programs of moderate scope and risk, contributing to the execution strategy, breaking larger goals into milestones, and holding yourself and others to clear expectations and commitments


• Takes ownership of complex technical outcomes across teams, proactively managing risks and dependencies, following through on commitments and driving issues to resoluti