Senior AI Engineer

Wintermeyer Ventures; San Francisco (CA); Remote; USD 200,000 - 250,000; Full time

Job summary

Wintermeyer Ventures is building an early-stage AI company developing software for physical commodity trading and operations. You will architect agent systems to convert modern models into reliable workflows, from experiments to production.

Join a small, hands-on team, owning production systems, designing planning and handoffs, and improving latency and cost through evaluations and tracing. Remote within the United States, with SF as a primary location.

Qualifications

• Four to ten years of software engineering experience, including ownership of production systems through launch, operation, and debugging.
• Hands-on experience building LLM or agent systems used by real users, including orchestration or harnesses, tool calling, and evaluations.
• Experience measuring and improving production LLM quality, latency, or cost using evaluations, trace inspection, and performance analysis.
• Strong software engineering skills in addition to any data science background, and the ability to move from prototypes to reliable production features.
• Experience at an early-stage startup and good judgment about when agents should act autonomously or hand work to a person.
• Strong computer science fundamentals and experience building practical AI workflows or connected-system integrations.

Responsibilities

• Build agent harnesses that provide models with the tools, context, and execution environments needed to complete tasks across the product and connected systems.
• Design agent planning, action-taking, state management, failure recovery, and handoffs to people.
• Prototype model-powered features against real customer problems and develop promising experiments into production capabilities.
• Create evaluations for task completion and decision quality, and turn production failures into reproducible test cases.
• Inspect execution traces and debug prompts, context, tool interfaces, orchestration, and model behavior.
• Improve agent autonomy using appropriate permissions, checkpoints, and human intervention.
• Evaluate models and approaches for capability, reliability, latency, and cost in real workflows.
• Build reusable tools and harness components that make features easier to develop and operate.

Skills

LLM experience

Agent systems

Tool calling

Performance analysis

Production systems

Tools

Orchestration

Tool calling

Evaluations frameworks

Job description

About the Role

Join an early-stage AI company building software for physical commodity trading and operations. You will build reliable agent systems that turn modern models into useful workflows, from early experiments through production use.

What You'll Do

• Build agent harnesses that provide models with the tools, context, and execution environments needed to complete tasks across the product and connected systems.
• Design agent planning, action-taking, state management, failure recovery, and handoffs to people.
• Prototype model-powered features against real customer problems and develop promising experiments into production capabilities.
• Create evaluations for task completion and decision quality, and turn production failures into reproducible test cases.
• Inspect execution traces and debug prompts, context, tool interfaces, orchestration, and model behavior.
• Improve agent autonomy using appropriate permissions, checkpoints, and human intervention.
• Evaluate models and approaches for capability, reliability, latency, and cost in real workflows.
• Build reusable tools and harness components that make features easier to develop and operate.

What We're Looking For

• Four to ten years of software engineering experience, including ownership of production systems through launch, operation, and debugging.
• Hands-on experience building LLM or agent systems used by real users, including orchestration or harnesses, tool calling, and evaluations.
• Experience measuring and improving production LLM quality, latency, or cost using evaluations, trace inspection, and performance analysis.
• Strong software engineering skills in addition to any data science background, and the ability to move from prototypes to reliable production features.
• Experience at an early-stage startup and good judgment about when agents should act autonomously or hand work to a person.
• Strong computer science fundamentals and experience building practical AI workflows or connected-system integrations.

Compensation & Benefits

Base salary: $200,000 to $250,000 USD annually. Visa sponsorship is not available.

Location

Remote within the United States. Primary location: San Francisco, California.