Engineering Manager, Forward Deployed AI & LLM Inference

BaseTen; New York, San Francisco (NY, CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive equity

Medical, dental, vision coverage

Flexible PTO

Parental leave

401(k)

Learning & networking opportunities

Job summary

Baseten is hiring an Engineering Manager (Player & Coach) to lead a team of Forward Deployed Engineers building and optimizing LLM inference workloads. You will blend hands‑on technical ownership with people leadership across design, deployment, and production monitoring.

You will partner with product and infrastructure teams to ensure high performance, reliable, and cost‑efficient AI services for Baseten customers, guiding strategic initiatives and customer engagements.

Qualifications

• Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, or related field.
• 4+ years of professional software engineering experience, including 1+ year in a leadership or mentorship capacity.
• Strong programming skills in Python, with production experience in building or optimizing ML inference systems.
• Proven experience with LLMs, inference optimization, or serving frameworks (e.g., vLLM, TensorRT, Triton, Hugging Face, Ray Serve).
• Familiarity with observability, profiling, and cost/performance tradeoffs in production ML systems.
• Excellent communication and collaboration skills—able to lead cross‑functional efforts and drive outcomes in ambiguous, fast‑paced environments.

Responsibilities

• Lead, mentor, and grow a team of Forward Deployed Engineers, providing guidance on technical direction, project execution, and professional development.
• Set clear goals and ensure timely, high-quality delivery across multiple customer‑facing projects involving LLM deployment and inference optimization.
• Collaborate with leadership to align team priorities with company and customer goals, balancing short‑term delivery, widely varying customer priorities, and long‑term technical initiatives.
• Player‑coach – drive strategic product initiatives and customer engagements, combining hands‑on work with management.

Skills

Python

LLMs

Leadership

Production ML

Education

Bachelor’s/Master’s/PhD in CS or related

Tools

vLLM

TensorRT

Triton

Hugging Face

Ray Serve

Job description

Baseten is hiring an Engineering Manager (Player & Coach) to lead a team of Forward Deployed Engineers building and optimizing LLM inference workloads. You will blend hands‑on technical ownership with people leadership across design, deployment, and production monitoring.

You will partner with product and infrastructure teams to ensure high performance, reliable, and cost‑efficient AI services for Baseten customers, guiding strategic initiatives and customer engagements.