Associate Director, AI Engineering – Live Operations

Jobtailor; New Jersey; On-site; USD 180,000 - 240,000; Full time

Job summary

Jobtailor seeks a senior data science leader to guide a team responsible for the production lifecycle, health, and optimization of deployed models. You will ensure AI applications stay accurate, safe, and efficient, while proactively monitoring performance and managing costs, including drift detection and latency tracking.

You will design continuous evaluation loops, oversee incident response, and implement automatic safety guards.

Qualifications

• Bachelor’s degree or four or more years of work experience.
• Eight or more years of relevant experience required, demonstrated through work, military, or training.
• Direct experience managing teams of data scientists or research-oriented talent in production, LiveOps, or operations.
• In-depth working experience with deep learning frameworks (PyTorch), Generative AI, and distributed computing systems.
• Experience building and maintaining automated model monitoring, evaluation, and CI/CD pipelines in cloud or edge production.
• Proficiency in Python, Java, or R, plus AWS, Hadoop, or other distributed compute platforms.

Responsibilities

• Lead a team focused on production lifecycle, health, and optimization of deployed models.
• Ensure AI applications remain accurate, safe, and operationally efficient.
• Monitor real-world model performance to detect drift, track latency, and manage costs.
• Design and implement continuous evaluation loops to assess model accuracy daily.
• Oversee incident response protocols and automated safety guardrails.
• Establish fallback architectures for seamless operation if a model fails.
• Automate CI/CD pipelines for seamless deployments.
• Partner with data science teams and senior leadership as a technical advisor.

Skills

Team Management

Deep Learning Frameworks

Automated Model Monitoring

CI/CD Pipeline Development

Real-Time AI Systems

Model Drift Detection

Automated Safety Filters

Cloud Computing

Python

Generative AI

Education

Bachelor’s degree

Experience leading data science teams

Tools

AWS

Hadoop

Distributed Computing Systems

CI/CD Tools

Real-Time Monitoring Tools

Job description

• use technical expertise in data science and real-time AI systems to lead a team focused on the production lifecycle, health, and optimization of deployed models
• ensure commercial AI applications remain highly accurate, safe, and operationally efficient
• monitor and observe real-world model performance to proactively detect model drift, track latency, and manage computational expenses
• design and implement continuous evaluation loops to assess model accuracy on a day-to-day basis
• manage incident response protocols and automated safety guardrails
• establish fallback architectures to ensure a seamless experience if a model fails
• automate continuous deployment (CI/CD) pipelines
• operate as a technical thought leader and trusted advisor to partner with broader data science teams and senior leadership

Requirements

• Bachelor’s degree or four or more years of work experience
• Eight or more years of relevant experience required, demonstrated through one or a combination of work and/or military experience, or specialized training
• Direct experience managing teams of data scientists or research-oriented talent in a production, operations, or LiveOps capacity
• In-depth working experience with deep learning frameworks (such as PyTorch), Generative AI, and distributed computing systems
• Experience building and maintaining automated model monitoring, evaluation, and CI/CD pipelines in a cloud or edge production environment
• Proficiency in developing real-time safety filters, system guardrails, and automated fallback systems for conversational or predictive AI
• Coding proficiency in Python, Java, or R, alongside experience with AWS, Hadoop, or other distributed compute platforms

Core Competencies

Demonstrates expertise in managing data science teams and optimizing AI model performance through continuous evaluation and automated deployment processes. Proficient in developing safety protocols and fallback architectures to ensure operational efficiency in AI applications.

Highest-signal resume keywords

• Team Management
• Deep Learning Frameworks
• Automated Model Monitoring
• CI/CD Pipeline Development
• Real-Time AI Systems

ATS Optimization Keywords

Hard Skills

• Data Science
• Model Optimization
• Deep Learning
• Python
• Java
• R
• Generative AI
• Automated Safety Filters
• Model Drift Detection
• Cloud Computing

Soft Skills

• Technical Leadership
• Collaboration
• Problem-Solving
• Communication

Industry Keywords

• Production Lifecycle
• Operational Efficiency
• Incident Response Protocols
• Fallback Architectures
• LiveOps

Tools & Technologies

• AWS
• Hadoop
• Distributed Computing Systems
• CI/CD Tools
• Real-Time Monitoring Tools