Systems Engineer - AI Agent Harness & Evaluation

United States Digital Space LLC; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Lexsi Labs is seeking engineers to build the shared layer and evaluation system for autonomous AI agents. You will join teams across the harness, evals, and toolchain, focusing on planning, execution, observation, recovery, and stop conditions.

Your work enables reproducible, measurable improvements in agent behavior across production-like environments. You will work with Python, distributed systems, and containerized sandboxes, contributing to evaluation pipelines, traceability, and tooling

Qualifications

• Strong software engineering fundamentals with production experience in Python.
• Experience with concurrent and distributed systems and asynchronous execution.
• Familiarity with containerized environments and sandboxing concepts.
• Comfort with observability, tracing and structured logging.
• Proven ability to debug non-deterministic, long-running systems.
• Open source contributions are a plus.
• Experience with evaluation or benchmarking work for AI systems is a strong plus.
• Knowledge of LLM inference, serving and related tooling is beneficial.

Responsibilities

• Develop the core agent loop, execution model, and orchestration across sub-agents.
• Design context/state management for long-running tasks and retention.
• Define the shared tool protocol, versioning, and internal libraries.
• Build sandboxed, reproducible execution substrates with resource isolation.
• Implement robust failure semantics, retries, and idempotency.
• Develop trace schemas for debugging, training signals, and audit records.
• Create task suites and infrastructure to run evaluations at scale.
• Support integration with research for evaluation design and interpretability.

Skills

Advanced Python

Concurrency & distributed systems

Containers & sandboxing

Observability & tracing

Debugging systemic issues

Open source contributions

AI evaluation/benchmarking

LLM inference & serving

Developer tooling

Tools

Docker

OCI runtimes

Git

Job description

Lexsi Labs is seeking engineers to build the shared layer and evaluation system for autonomous AI agents. You will join teams across the harness, evals, and toolchain, focusing on planning, execution, observation, recovery, and stop conditions.

Your work enables reproducible, measurable improvements in agent behavior across production-like environments. You will work with Python, distributed systems, and containerized sandboxes, contributing to evaluation pipelines, traceability, and tooling