Senior Software Engineer, Self-Improving AI Systems

Sycamore Labs, Inc.; Palo Alto (CA); Hybrid; USD 180,000 - 260,000; Full time

Job summary

Sycamore Labs, Inc. is seeking a seasoned engineer to build and optimize the runtime, evaluation, and learning systems powering our agent platform in Palo Alto.

You will own end-to-end slices from cloud services to React interfaces, safeguarding correctness across retries and failures while enabling scalable, secure multi-tenant deployments. Join a small, engineering-led team partnering with Fortune 500 enterprises.

Qualifications

• 5–12 years of software engineering experience.
• Strong backend and distributed-systems fundamentals.
• Depth in at least one area and willingness to work across others.
• Empirical approach to AI quality and evaluation design.
• Ability to reason about retries, partial failure, and long-running state.
• Security-minded for multi-tenant systems and privacy.
• Product judgment and ownership across cloud-native components.
• AI-native mindset with modern models as a multiplier.
• Comfort with React/TypeScript for UI when needed.
• Ability to thrive in startup ambiguity and fast feedback loops.

Responsibilities

• Develop the learning data plane around agents: structured trajectories, feedback and outcome signals, offline datasets, lineage, privacy controls, and reliable links between an agent version and its behavior.
• Create evaluation systems for task completion, tool use, long-horizon behavior, safety, latency, and cost, and own the credibility of the numbers they produce.
• Design experiment and versioning systems for comparing changes through replay, shadow traffic, canaries, or controlled rollouts, with clear promotion and rollback criteria.
• Build durable orchestration for long-running tasks, checkpoints, approvals, handoffs, and human-in-the-loop interactions, with typed tool interfaces, protocol-based execution, and memory retrieval that enforces tenant, user, and project visibility boundaries.
• Own the execution environments agents run in, including isolation, startup performance, resource limits, and cost, and the policy layer that gates self-modification so a proposed change is staged and tested rather than applied silently.
• Publish reliable APIs, event-driven interfaces, and reusable libraries that work across agent categories and enterprise deployments.

Skills

Backend Systems

Distributed Systems

API Design

Asynchronous Workflows

Production Debugging

Security for Multi-tenant

Product Judgment

AI/ML tooling familiarity

React

TypeScript

Tools

React

TypeScript

Cloud Services

Job description

Sycamore Labs, Inc. is seeking a seasoned engineer to build and optimize the runtime, evaluation, and learning systems powering our agent platform in Palo Alto.

You will own end-to-end slices from cloud services to React interfaces, safeguarding correctness across retries and failures while enabling scalable, secure multi-tenant deployments. Join a small, engineering-led team partnering with Fortune 500 enterprises.