Lead Machine Learning Engineer

Jobtailor; California (MO); On-site; USD 140,000 - 210,000; Full time

Job summary

Jobtailor is seeking an engineer to design and develop ML evaluation systems for agentic and LLM-based architectures. You will translate research ideas into production-grade ML systems with measurable impact and partner with Research, Product, and Platform teams to productize experiments into robust AI solutions.

Stay current with advancements in ML, NLP, and LLM systems and contribute to technical discussions.

Qualifications

• Deep experience in NLP and modern ML systems, including hands-on work with LLM-based or agentic systems.
• Strong architectural skills and proven experience designing complex software systems.
• Production experience with Python, AWS, Kubernetes, and/or Docker.
• Experience implementing technical solutions and tackling challenging engineering problems.
• Bachelor’s Degree in CS or another related field.
• Experience building and evaluating agentic systems at scale.
• Production experience with LLM-centric services, including inference, orchestration, evaluation, and monitoring.
• Familiarity with large-scale ML experimentation, benchmarking, or simulation frameworks.
• Knowledge of techniques for optimizing model architectures for faster inference.
• Experience with AWS, CI/CD, Kafka, and Athena.
• Proficiency in technical mentorship of junior and mid-level engineers.
• Willingness to learn, collaborate across teams, and teach others.

Responsibilities

• Design and develop ML evaluation systems for agentic and LLM-based architectures.
• Translate research ideas into production-grade ML systems with measurable impact.
• Partner with Research, Product, and Platform teams to productize experiments into robust AI solutions.
• Stay current with advancements in ML, NLP, and LLM systems and contribute to technical discussions.
• Mentor and support other engineers through design reviews, feedback, and knowledge sharing.
• Build and evaluate the core intelligence behind agentic AI systems.
• Design and own evaluation frameworks ensuring quality, safety, and performance across complex agentic systems.

Skills

NLP expertise

LLM-based systems

Architectural design

Python development

AWS

Kubernetes

Docker

CI/CD

Kafka

Athena

Education

Bachelor's Degree in CS

Tools

AWS

Kubernetes

Docker

CI/CD

Kafka

Athena

Job description

• Design and develop ML evaluation systems for agentic and LLM-based architectures
• Translate research ideas into production-grade ML systems with measurable impact
• Partner with Research, Product, and Platform teams to productize experiments into robust AI solutions
• Stay current with advancements in ML, NLP, and LLM systems and contribute to technical discussions
• Mentor and support other engineers through design reviews, feedback, and knowledge sharing
• Build and evaluate the core intelligence behind agentic AI systems
• Design and own evaluation frameworks ensuring quality, safety, and performance across complex agentic systems

Requirements

• Deep experience in NLP and modern ML systems, including hands-on experience with LLM-based or agentic systems
• Strong architectural skills and proven experience designing complex software systems
• Production experience with Python, AWS, Kubernetes, and/or Docker
• Experience implementing technical solutions and tackling challenging engineering problems
• Bachelor’s Degree in CS or another related field
• Experience building and evaluating agentic systems at scale
• Production experience with LLM-centric services, including inference, orchestration, evaluation, and monitoring
• Familiarity with large-scale ML experimentation, benchmarking, or simulation frameworks
• Knowledge of techniques for optimizing model architectures for faster inference
• Experience with AWS, CI/CD, Kafka, and Athena
• Proficiency in technical mentorship of junior and mid-level engineers
• Willingness to learn, collaborate across teams, and teach others

Core Competencies

Demonstrates expertise in designing and developing ML evaluation systems, particularly for agentic and LLM-based architectures, while effectively collaborating with cross-functional teams to translate research into impactful AI solutions. Proficient in mentoring engineers and ensuring quality and performance in complex systems.

Highest-signal resume keywords

• Deep Experience In NLP
• Production Experience With Python
• Architectural Skills In Complex Software Systems
• Experience With LLM-Centric Services
• Familiarity With Large-Scale ML Experimentation

ATS Optimization Keywords

Hard Skills

• Machine Learning
• Natural Language Processing
• LLM-Based Systems
• Software Architecture
• Model Optimization Techniques
• Technical Solution Implementation
• Evaluation Framework Design
• Benchmarking
• Simulation Frameworks
• Agentic Systems Evaluation

Soft Skills

• Mentorship
• Collaboration
• Knowledge Sharing
• Feedback Provision
• Willingness To Learn

Certifications & Qualifications

• Bachelor’s Degree In Computer Science

Industry Keywords

• AI Solutions
• Production-Grade Systems
• Technical Discussions
• Quality Assurance
• Performance Evaluation

Tools & Technologies

• AWS
• Kubernetes
• Docker
• CI/CD
• Kafka
• Athena