Founding AI Engineer

Aimhire; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Aimhire is recruiting a Founding AI Engineer to work with the founders on the core AI infrastructure powering their voice agent testing and monitoring platform. You will drive experimentation, build scalable systems, and interface directly with customers to incorporate feedback and improve performance.

This hands-on, high-impact role lets you help define product and technology direction from day one, shaping how the platform evaluates and ensures reliability of conversational AI in real-world

Qualifications

• 1+ years of experience as an AI or ML Engineer.
• Strong proficiency in Python.
• Experience working with LLMs or conversational AI systems.
• Familiarity with evaluation pipelines and production deployment.

Responsibilities

• Build AI tools for testing, benchmarking, and monitoring conversational agents
• Develop scalable pipelines and evaluation methods for LLMs
• Implement systems for real-time monitoring and feedback
• Optimize deployments for high availability and performance
• Engage with customers to understand needs and shape product direction
• Continuously experiment and improve AI reliability and accuracy

Skills

Python

LLMs / Conversational AI

Production deployment

Job description

About the Company

Our client is a Y Combinator–backed startup transforming how teams test and monitor AI voice agents. Founded by IIT Bombay alumni with deep research and trading expertise, they’re solving the pain of manual voice testing by building automation tools that simulate thousands of real-world conversations.

Their platform uses dynamic personas, AI-generated datasets, and detailed workflows to uncover edge cases, identify failure points, and ensure that conversational agents perform reliably — before going live. Their mission is simple: help teams ship voice agents that work every time.

About the Role

As a Founding AI Engineer, you’ll work closely with the founders to design and build the core AI infrastructure behind their voice agent testing and monitoring platform. You’ll drive experimentation, build scalable systems, and collaborate directly with customers to incorporate feedback and improve performance.

This is a hands-on, high-impact role where you'll help define both product and technology direction from day one.

Key Responsibilities

• Build AI tools for testing, benchmarking, and monitoring conversational agents
• Develop scalable pipelines and evaluation methods for LLMs
• Implement systems for real-time monitoring and feedback
• Optimize deployments for high availability and performance
• Engage with customers to understand needs and shape product direction
• Continuously experiment and improve AI reliability and accuracy

What We’re Looking For

• 1+ years of experience as an AI or ML Engineer
• Strong proficiency in Python
• Experience working with LLMs or conversational AI systems
• Familiarity with evaluation pipelines and production deployment
• Customer-focused mindset and startup experience a plus