Principal AI Engineer

mymoose

United States

On-site

USD 180,000 - 340,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Rapid7 is expanding its AI Engineering leadership to shape the next generation of AI across security products. You will define architecture for LLM and agentic systems, guide model strategy, and mentor engineers to build scalable, production-grade AI that delivers real impact for global customers.

You will stay hands-on, prototype critical components, and help establish standards for context management, inference optimization, and evaluation across teams.

Qualifications

  • 12+ years of software or ML engineering experience, including production LLM or agentic AI systems.
  • Strong track record influencing multiple teams with technical leadership and standards.
  • Deep understanding of transformer architecture and model serving from scratch.

Responsibilities

  • Define architecture and roadmap for LLM and agentic AI across security products.
  • Lead model strategy: frontier vs open-source, fine-tuning, distillation, post-training.
  • Fine-tune/open-source LLMs on security data from data curation to deployment.
  • Set engineering standards for context engineering, inference, and eval across teams.
  • Optimize LLM inference at scale with techniques like speculative decoding and KV-cache.
  • Establish evaluation gates for releases and lead design reviews.
  • Hands-on prototype and build model, training, and inference code.

Skills

LLMs fundamentals
PyTorch
Distributed systems
Python
Mentoring/leadership
Transformer architectures
Model fine-tuning
Inference optimization

Education

PhD in CS/ML preferred

Tools

PyTorch (deep)
FSDP
DeepSpeed
TensorRT-LLM
vLLM

Job description

Our Engineering organisation is scaling its AI Engineering capabilities under new leadership, creating an opportunity to build the next generation of AI at Rapid7. We're looking for Principal AI Engineers to join us and help shape what comes next by defining and influencing the technical direction of our LLM and agentic AI systems. As a Principal AI Engineer, you'll raise the bar for engineering standards, and build truly cutting edge greenfield solutions that deliver real impact for our Global customers. Through your technical leadership and mentoring, you'll help shape our product and platform investments and influence how we build and scale AI across the organisation.

About the Team

AI is reshaping cybersecurity on both sides. Attackers can use AI to strike at machine speed, and only AI-powered defense can keep up. At the same time, organizations can only adopt AI with confidence if it is secure. Rapid7 sits at the intersection of security and AI, and we are building industry-defining products for this new era.

Rapid7's AI Engineering team builds the AI that powers our security platform. We are AI engineers and applied scientists who take models from research to production, combining open-source and frontier LLMs, AI agents, and fast predictive models with more than 20 years of Rapid7 security expertise. Our work drives core use cases across Detection and Response, such as alert triage, threat investigation, and automated response, and Exposure and Remediation, such as vulnerability prioritization, attack surface risk assessment, and guided remediation. Together, these help customers stop threats faster and fix what matters most.

About the Role

As a Principal AI Engineer, you will set the technical direction for how Rapid7 builds, tunes and serves LLMs and agentic systems across its security products. You understand LLMs from the ground up, from transformer internals and training to fine-tuning and inference optimization, and you stay hands‑on with the hardest problems. You know where LLMs shine and where they fall short, and you treat them as one tool in the AI toolbox, alongside predictive AI and classical ML, choosing the right approach for each problem. Specifically your focus will be to:

  • Define the architecture and technical roadmap for LLM and agentic AI systems across Detection and Response and Exposure and Remediation
  • Lead model strategy: when to use frontier or open-source models, and when to fine-tune, distill, or post-train
  • Fine-tune, distill and post-train open-source LLMs on security data, from data curation through training, evaluation and deployment
  • Set engineering standards for context engineering, inference optimization, and eval across teams
  • Optimize LLM inference at scale with techniques such as speculative decoding, KV-cache management, continuous batching, quantization and custom kernels, balancing quality, latency and cost
  • Establish evals as the release gate for every AI system we ship
  • Lead design reviews and resolve the hardest technical trade-offs
  • Stay hands‑on: prototype and build critical model, training and inference code yourself
  • Mentor Staff and Senior engineers and shape product and platform investment
The skills and qualities you'll bring include :
  • 12+ years of software or ML engineering experience, including 2+ years building production LLM or agentic AI systems. A PhD in Computer Science, Machine Learning, AI or a related field is preferred, or equivalent depth of expertise demonstrated through industry experience and technical contributions
  • A track record of building scalable, production‑grade AI systems used across multiple teams or products with a strong history of organization‑wide impact through technical leadership, mentoring and shared platforms and standards
  • LLMs from first principles: deep understanding of transformer architecture, attention, tokenization and positional encodings, and how models are trained and served, with the ability to implement core components from scratch
  • PyTorch: expert‑level, including custom model code, training loops, mixed precision and distributed training (e.g., FSDP, DeepSpeed)
  • Fine‑tuning and post‑training: hands‑on full and parameter‑efficient fine‑tuning (LoRA, QLoRA) of open‑weight LLMs, distillation, and preference or RL‑based methods (e.g., DPO, GRPO)
  • Context engineering: retrieval, memory, and context management for long‑running agent workflows
  • Inference optimization: deep knowledge of prefill and decode, KV‑cache management (e.g., PagedAttention, prefix caching), continuous batching, speculative decoding, FlashAttention and quantization, and serving at scale with vLLM, SGLang or TensorRT‑LLM
  • Evals: defining evaluation strategy and benchmarks for LLM and agent systems
  • Strong predictive AI and classical ML fundamentals, and the judgement to choose between them and LLMs based on accuracy, latency and cost
  • Strong distributed systems and Python fundamentals
  • Takes ownership of significant technical outcomes across multipl
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Manager - AI Engineering
Senior Manager - AI Engineering

mymoose • Boston (MA)

On-site
GBP 90,000 - 140,000
Principal AI Engineer
Principal AI Engineer

Rapid7 • United States

Remote
USD 221,000 - 299,000
Staff AI Engineer
Staff AI Engineer

Rapid7 • California

Hybrid
USD 187,000 - 252,000
Principal AI Engineer
Principal AI Engineer

GSquared Group • Atlanta (GA)

On-site
USD 180,000 - 260,000
Staff Machine Learning Engineer
Staff Machine Learning Engineer

Primer.ai • Pasadena (CA)

On-site
USD 180,000 - 260,000
Senior AI Engineer
Senior AI Engineer

Equs, Inc. • United States

Remote
USD 180,000 - 240,000
AI Engineer
AI Engineer

Data • Union City (NJ)

On-site
USD 120,000 - 150,000
Principal AI Engineer, Product Cybersecurity
Principal AI Engineer, Product Cybersecurity

Toyota Tsusho Systems • Plano (TX)

On-site
USD 150,000 - 190,000
Remote Principal AI Engineer: LLM & Agentic Systems Lead
Remote Principal AI Engineer: LLM & Agentic Systems Lead

Rapid7 • United States

Remote
USD 221,000 - 299,000
AI/ML Engineer
AI/ML Engineer

RiskForce LLC • United States

On-site
USD 120,000 - 180,000