Machine Learning Engineer

Nyotta AI Inc.; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Nyotta AI Inc. is seeking a software-focused ML engineer to advance agents that reason over information, use tools, and complete multi-step tasks. You will design experiments, evaluate outcomes, and improve product behavior with engineers.

You will work on decisions an agent makes—what to retrieve, which tool to use, and when to ask for help—combining ML, experimentation, and software engineering to ship robust AI workflows.

Qualifications

• Experience building ML/AI projects and testing results.
• Strong Python programming and debugging model behavior.
• Ability to define success metrics and iterate based on results.
• Experience collaborating with software engineers.

Responsibilities

• Experiment with models, prompts, retrieval, and context management.
• Build evaluations to test reasoning, tool selection, and output quality.
• Investigate failed runs and identify root causes in context, tools, or logic.
• Compare approaches using task success, latency, and cost; implement regression checks.
• Develop feedback loops turning user corrections into better data and evaluations.

Skills

Python programming

Experimentation

Debugging ML models

Cross-functional collaboration

Tools

LLM agents

Job description

Build and evaluate agents that reason over complex information, use tools, and complete multi-step tasks. Turn experiments into behavior we can measure and improve.

Nyotta builds AI agents for complex engineering and business workflows. Our work brings together reasoning over technical information, tool use, and human review so teams can move from source material to decisions and completed work. We apply these systems to engineering and commercial processes where accuracy, traceability, and control matter.

About the role

You will work on the decisions an agent makes: what information to retrieve, which tool to use, how to interpret a result, and when to ask a person for help. The work combines applied ML, experimentation, and software engineering. You will partner with software engineers to test new approaches, investigate failures, and bring useful improvements into the product.

In this role, you will

• Experiment with models, prompts, retrieval, and context management to improve agent task completion.
• Build evaluations that test reasoning, tool selection, and output quality across complete workflows.
• Investigate failed runs and distinguish problems in the model, available context, tools, and execution logic.
• Compare approaches using task success, latency, and cost; build regression checks before rolling out changes.
• Develop feedback loops that turn user corrections and observed failures into better datasets and evaluations.

Your background

• You have built an ML or AI project and can explain how you tested it, where it failed, and what you changed. Research, coursework, personal projects, and professional work all count.
• You write Python and are comfortable building experiments, working with data, and debugging model behavior.
• You can define a useful success measure and investigate results that disagree with your assumptions.
• You enjoy working with engineers to turn an experimental approach into maintainable software.

Level and scope

• Engineer: Work on a scoped agent capability or evaluation problem, with guidance and code review. Early-career applicants are welcome.
• Senior: Own an agent capability from experimental design through evaluation, release, and follow-up. Bring experience shipping and maintaining applied ML systems.

Helpful experience

Experience with LLM agents, tool use, retrieval, multimodal models, evaluation frameworks, or experimentation on production AI systems.

What makes this work hard

A run can look convincing while missing a necessary step. Model changes can fix one failure and introduce another. You will need evaluations that reveal those tradeoffs and the patience to understand individual failures.

A problem you might work on

An agent repeatedly chooses an unhelpful tool. Reproduce the behavior, test whether the problem is context or tool selection, and measure whether your fix improves complete tasks without increasing cost unnecessarily.