Generative AI Agent Engineer (Hybrid, LA)

Technology Management Solutions LLC; Los Angeles (CA); Híbrido; USD 180.000 - 280.000; Jornada completa

Descripción de la vacante

TMS LLC is seeking an AI Engineer / Generative AI Engineer to help build production-grade AI agents and bring them through deployment, monitoring, and continuous improvement. This contract role is based in Los Angeles, CA with a hybrid work model, and focuses on agent development, evaluation, and observability in real environments.

You will work with Python and the Claude Agent SDK, deploy via AWS Bedrock AgentCore, and strengthen quality through golden datasets, LLM-as-a-Judge evaluations, and

Formación

• 15+ years of experience is required.
• 6–8+ years of software engineering, cloud engineering, or AI/ML experience.
• Hands-on experience with production AI agents and continuous evaluation.
• Strong problem-solving and troubleshooting skills.

Responsabilidades

• Develop AI agents using Python and Claude Agent SDK.
• Deploy agents via AWS Bedrock AgentCore Runtime.
• Design and maintain golden datasets for reliable evaluation.
• Implement LLM-as-a-Judge evaluations and calibrate against human labels.
• Integrate automated evaluation into CI/CD quality gates.
• Set up Langfuse and/or OpenTelemetry for observability, tracing, and monitoring.
• Evaluate agent outcomes including accuracy, task completion, tool execution, and performance.
• Support RAG pipelines and AWS data processing workflows.

Conocimientos

Python

agentic AI development

LLM evaluation design

CI/CD quality gates

Troubleshooting

Herramientas

Claude Agent SDK

AWS Bedrock AgentCore

Langfuse

OpenTelemetry

Golden datasets

LLM-as-a-Judge

Descripción del empleo

TMS LLC is seeking an AI Engineer / Generative AI Engineer to help build production-grade AI agents and bring them through deployment, monitoring, and continuous improvement. This contract role is based in Los Angeles, CA with a hybrid work model, and focuses on agent development, evaluation, and observability in real environments.

You will work with Python and the Claude Agent SDK, deploy via AWS Bedrock AgentCore, and strengthen quality through golden datasets, LLM-as-a-Judge evaluations, and