Member of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)

Aptiora; San Francisco (CA); Vor Ort; USD 180.000 - 260.000; Vollzeit

Zusammenfassung

Aptiora in San Francisco, CA is seeking a senior engineering leader to manage the engineering team and guide projects from prototype to production. You will own complex, scalable systems for enterprise evaluation environments and drive key architectural decisions.

You will shape evaluation infrastructure, encourage reusable tools and workflows, and mentor engineers to deliver reliable, cost-effective solutions that balance quality and speed.

Qualifikationen

• Strong system design and distributed systems skills.
• Experience owning complex production systems from design through deployment.
• Hands-on experience with backend development and AI/ML tooling.

Aufgaben

• Lead engineering team and drive from prototype to production.
• Analyze and improve evaluation infrastructure and workflows.
• Develop reusable skills and tools for complex experiments.

Tools

Python

Go

Rust

Jobbeschreibung

Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state. Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.

Manage engineering team

From prototype to production

Complex technical systems

Leadership scope High

AI / domain intensity Medium

What you would do

• Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state.
• Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.
• Analyze agent trajectories and production failures, identify root causes, and ship improvements across the agent harness, tools, and skills that measurably improve task success and reliability.
• Develop reusable skills and improve tool descriptions, schemas, and result handling so agents can select the right tools, execute workflows correctly, and recover from errors with fewer unnecessary calls.
• Build and evaluate model-selection and routing strategies that match each task to the right model and effort level, balancing quality, latency, reliability, and cost.
• Improve agent efficiency through context management, skill loading, caching, and execution strategies that reduce wasted tokens and cost per successful task without compromising quality or safety.
• Build regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes, and validate that evaluation gains translate into better production outcomes.

Why this role matters

• Work on a core product area for the next era of AI.
• Bridge product ambition with research and infrastructure reality.

What the employer asks for

• + years of software engineering or machine learning engineering experience, with a track record of owning complex production systems from design through deployment and operation.
• Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications.
• Strong proficiency in a backend language such as Python, Go, or Rust, and the ability to work effectively across a multi-language production stack.
• Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows.
• Deep expertise in at least one of the following:
• Evaluation infrastructure, benchmark design, and automated grading.
• Agent harnesses, tool interfaces, and reusable skills for complex workflows.
• Realistic test environments or simulations spanning multiple systems, datasets, permissions, and stateful workflows.
• Model selection, routing, and inference efficiency, including context management and caching.
• Strong experimental and statistical judgment, including choosing representative tasks, defining meaningful success criteria, diagnosing failures, and distinguishing real improvements from noise.

Where the uncertainty is

• Exact team size, reporting line, and near‑term mandate are not fully visible from the public posting.
• Compensation, equity, or leveling details are not explicit yet.
• Working model and location flexibility still need verification.
• The balance between research, product, and infrastructure responsibilities may still be broad.

Evidence this role rewards

• Experience in distributed systems and production engineering.
• Hands‑on comfort with modern AI or multimodal product stacks.
• Leadership experience in fast‑moving or high‑ownership teams.
• Track record of turning complex ideas into shipped products.

Why Aptiora keeps this on the radar

Aptiora is not asking whether the role sounds impressive. It asks whether the posting contains enough scope, evidence, leverage, and unresolved questions to deserve your attention. This public page organizes the opportunity; personalized fit begins inside My Aptiora.

• The role shows strong leverage rather than generic volume hiring.
• The posting contains enough substantive detail to justify deeper analysis inside Aptiora.
• Public evidence is still not the same as your personal fit or decision quality.
• Location and working model may be decisive filters before you invest too much energy.

What to verify before acting

• Team mission, size, and what success looks like in the first months.
• Reporting structure, level expectations, and decision authority.
• Whether the real scope is product, research, infrastructure, or a blend.
• Working model, compensation, equity, and interview process.

Stay inside Aptiora. Review the opportunity in your workspace, compare it against your direction, and decide with evidence before you spend energy on outreach or applications.

Get deeper company intelligence, fit analysis, and application strategy inside Aptiora.

Use Aptiora to turn this posting into a clearer plan: what matters, what is uncertain, what to ask, and whether the role deserves your attention at all.