Sr. AI / Machine Learning Platform Engineer - Voice Agents

Skyrocket Ventures; San Francisco (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

Skyrocket Ventures is looking for engineers who are proficient in Python and willing to learn new technologies. The role focuses on building and improving simulation and evaluation platforms for AI agents in San Francisco.

Successful candidates will have 5-10 years of experience in software engineering, particularly in backend systems. Responsibilities include designing and enhancing tools for AI agent evaluation and collaborating closely with other engineers to optimize performance.

Qualifications

• 5-10 years of engineering experience in reputable organizations.
• Strong software engineering experience in complex backend systems.
• Experience with evaluation and testing systems for AI products.

Responsibilities

• Program in Python and learn new technologies as needed.
• Build a simulation and evaluation platform for AI agents.
• Enhance frameworks used to develop AI agents.

Skills

Python programming

Software engineering

Backend systems development

AI/ML systems

Reliability analysis

Job description

Responsibilities

• Primarily programming in Python. The company seeks language agnostic engineers who are able and interested to learn new technologies as needed.
• Time spent will be roughly 50% building a simulation and evaluation platform for AI agents, and 50% working on the framework used to build the AI agents.
• Designing, building, and enhancing a simulation and evaluation platform used to rigorously test, benchmark, and improve AI agents across a wide range of tasks and environments.
• Developing scalable evaluation frameworks, tooling, and methodologies to measure agent performance, reliability, decision quality, and failure modes.
• Build core frameworks, abstractions, and infrastructure used to develop, orchestrate, and deploy AI agents.
• Collaborating closely with other engineers to rapidly iterate on agent architectures, capabilities, and evaluation strategies.
• Creating systems that enable reproducible experiments, large-scale simulations, and data-driven optimization of agent behavior.
• Improving developer workflows, testing capabilities, and platform reliability to accelerate AI agent development and deployment.
• Analyzing evaluation results and agent behaviors to identify weaknesses, guide improvements, and drive overall system performance.
• Contributing to technical architecture and engineering best practices across both the agent framework and evaluation platform stack.

Qualifications

• 5-10 years of engineering experience in good organizations.
• Strong software engineering experience building complex backend systems, platforms, frameworks, or developer tooling.
• Experience designing or building evaluation, testing, simulation, benchmarking, or experimentation systems for AI/ML products or distributed systems.
• Strong programming skills in Python and/or other modern backend languages.
• Experience working with LLMs, AI agents, reinforcement learning systems, or other modern AI application architectures.
• Strong understanding of software architecture, scalability, reliability, and performance optimization.
• Experience building internal platforms, SDKs, APIs, orchestration systems, or infrastructure that enable rapid product development.
• Ability to analyze complex system behavior, debug issues, and improve reliability through data-driven approaches.
• Comfortable working in fast-moving, ambiguous environments with a high degree of ownership and autonomy.