AgentOps (Artificial Intelligence Engineer)

Applaudo; Estado de México; Presencial; MXN 1.085.187 - 1.446.916; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Celebrations

Entertainment area

Modern Workspaces

Great work environment

Private medical insurance

Descripción de la vacante

A leading tech company in Mexico seeks an AI Engineer for AI systems engineering. You will design observability, manage datasets, and implement deployment strategies in a cloud-native environment. The ideal candidate has a strong background in AI/ML evaluation and tools like AWS, Docker, and Kubernetes. Benefits include modern workspaces and private medical insurance. Join us to advance your career in a dynamic team dedicated to excellence.

Formación

• Strong experience with automated judge systems and evaluation pipelines.
• Experience translating product KPIs into measurable metrics.
• Proficient in English for global collaboration.

Responsabilidades

• Translate KPIs into measurable agent-level metrics.
• Design and implement end-to-end observability.
• Manage golden datasets and judge systems.

Conocimientos

AI/ML evaluation and metric definition

Observability and monitoring

CI/CD automation

AWS services

Docker and Kubernetes

Educación

Bachelor’s degree in computer science or related field

Herramientas

Docker

Kubernetes

AWS

OpenTelemetry

Descripción del empleo

AgentOps (Artificial Intelligence Engineer)

3 days ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

You are an engineer passionate about building reliable, observable, and secure AI systems at scale. You enjoy working at the intersection of product metrics, evaluation frameworks, and production‑grade infrastructure. You are comfortable translating abstract KPIs into measurable signals, designing deployment safeguards, and ensuring AI systems behave safely, efficiently, and consistently in real‑world environments.

You thrive in fast‑paced, cloud‑native environments, value automation and rigor, and enjoy collaborating with product, engineering, and AI teams to continuously improve system quality.

Qualifications

• Bachelor’s degree in computer science, Software Engineering, Computer Engineering, or related field — or equivalent professional experience.
• Strong experience with AI/ML evaluation, metric definition, evaluation pipelines, golden datasets, and automated judge systems.
• Proficiency in observability and monitoring, including structured logging, tracing, and OpenTelemetry.
• Solid background in CI/CD automation and modern deployment strategies (canary, blue‑green, gated deployments).
• Knowledge of AI safety practices, including PII scrubbing, deterministic guardrails, and secure handling of model inputs and outputs.
• Experience working with multi‑agent systems and translating product KPIs into measurable agent performance metrics.
• Hands‑on experience with AWS (CDK, ECS/ECR, WAF, SES, Bedrock, CloudWatch, DevOps Guru).
• Strong experience with Docker, Kubernetes, and cloud‑native tooling.
• Familiarity with Azure for identity management and basic exposure to GCP environments.
• Strong analytical thinking, attention to detail, and problem‑solving skills.
• Excellent communication skills to collaborate across product, platform, and AI teams.
• English proficiency required for collaboration with global stakeholders.

Responsibilities

• Translate product KPIs into measurable agent and system‑level metrics for effectiveness, efficiency, robustness, and safety.
• Design and implement end‑to‑end observability using structured logging, metrics, and tracing with OpenTelemetry.
• Curate and maintain golden datasets and manage judge systems for scalable, repeatable AI evaluation.
• Implement evaluation‑gated deployments within CI/CD pipelines.
• Orchestrate pre‑merge and post‑merge validation workflows to ensure quality before release.
• Apply canary and blue‑green deployment strategies, enabling fast and safe rollbacks.
• Enforce layered security controls, including PII scrubbing, deterministic guardrails, and AI‑based filtering of inputs and outputs.
• Monitor and analyse latency, error rates, token usage, and cost metrics across AI systems.
• Track production quality indicators such as correctness, relevance, and helpfulness.
• Convert failures, incidents, and negative feedback into automated regression tests.
• Manage multi‑agent interoperability and coordination across AI components.
• Continuously update and adapt guardrails and safety controls as new risks and threats emerge.

Additional Information

Here at Applaudo Studios values trust, communication, respect, excellence and teamwork are our keys to success. We know we are working with the best and thus treat each other with respect and admiration without asking.

Submit your application today, and don't miss this opportunity to join the Best Digital team in the Region!

We truly appreciate all the hard and outstanding work our team makes every day at Applaudo Studios, and that's why the perks we offer are thoughtfully designed as a way to thank them for their commitment and excellence.

Benefits

• Celebrations
• Entertainment area
• Modern Workspaces
• Great work environment
• Private medical insurance

*Benefits may vary according to your location and/or availability. Request further information when applying.

Seniority level

Mid‑Senior level

Employment type

Full‑time

Job function

IT Services and IT Consulting