LLM & Agentic AI Architect — Senior Principal Engineer

mymoose; United States; Hybrid; USD 210,000 - 320,000; Full time

Job summary

Rapid7 seeks a Senior Principal AI Engineer to define and influence the technical direction of LLMs and agentic AI across its security products. You will stay hands‑on with transformer internals, training, fine‑tuning and inference optimization, choosing the right approach for each problem.

You will set architectural principles, lead high‑impact initiatives, establish context engineering standards and mentor senior engineers while partnering with executives to shape product strategy.

Qualifications

• 14+ years of software or ML engineering experience, including 2+ years building production LLM or agentic AI systems.
• Track record of building scalable, production‑grade AI systems
• Experience leading multiple teams and setting engineering standards.

Responsibilities

• Set the AI technical vision, architecture and multi‑year roadmap for LLM and agentic systems across products.
• Define company‑wide model strategy and when to use frontier or open source models; determine fine‑tuning, distillation or post‑training decisions.
• Lead complex, high‑risk initiatives across teams including fine‑tuning and post‑training of LLMs on security data.
• Establish company‑wide standards for context engineering and evaluation as release gates.
• Lead optimization of LLM inference at scale balancing quality, latency and cost.
• Prototype and contribute critical model, training and inference code.
• Mentor Principal, Staff and Senior engineers to deepen AI talent across the org.
• Represent Rapid7's AI engineering to executives and customers.

Skills

LLMs knowledge

PyTorch

Distributed training

LoRA/QLoRA

Distillation

RL methods (DPO/GRPO)

Context engineering

Inference optimization

vLLM/TensorRT-LLM

CUDA/Triton

Python

Technical leadership

Mentoring

Architecture roadmapping

Education

PhD in Computer Science, Machine Learning, AI or related field

Tools

FSDP

DeepSpeed

Open‑weight LLMs

CUDA

Job description

Rapid7 seeks a Senior Principal AI Engineer to define and influence the technical direction of LLMs and agentic AI across its security products. You will stay hands‑on with transformer internals, training, fine‑tuning and inference optimization, choosing the right approach for each problem.

You will set architectural principles, lead high‑impact initiatives, establish context engineering standards and mentor senior engineers while partnering with executives to shape product strategy.