Backend Systems & Agentic Workflows Engineer Remote (US-Based) – SF Bay Area Preferred →

Coco Labs; San Francisco (CA); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Coco Labs is seeking an experienced backend engineer to own the Python/FastAPI stack, data extraction layers, and multi-modal workflow engines. You will build resilient services that process high-concurrency streams and manage token contexts at scale.

We prize simple, clean code, rigorous unit tests, and minimal abstractions. You’ll optimize latency, ensure data quality, and contribute to reliable ingestion pipelines that drive production-grade AI workflows.

Qualifications

• 3–5 years of Python backend experience in data-intensive web services.
• Advanced Python with FastAPI, asyncio, and uvicorn.
• Hands-on with multi-modal data and high-concurrency systems.
• Strong unit testing and code hygiene practices.
• Experience optimizing latency and reliability for production backends.
• Familiarity with data validation (Pydantic) and schema enforcement.

Responsibilities

• Own the FastAPI & Async Infrastructure: design, optimize, and maintain the core non-blocking service tier.
• Build Continuous LLM Evaluation Pipelines (Evals): create deterministic regression testing for model outputs.
• Orchestrate Multi-Modal Processing & Indexing: ingest and index unstructured text and visual media.
• Optimize Context Window & Token Management: design memory layers and context pruning strategies.
• Build Extraction, Quality & Filtering Pipelines: validate and cleanse data before it enters agent loops.
• Enforce Code Hygiene & Reliability: implement tests and maintain modular, well-documented code.
• Orchestrate High-Volume Ingestion: manage queues, rate limits, and backpressure with Redis.
• Engineer Stateful Agent Workflows: design deterministic control planes for multi-step loops.

Skills

Python

FastAPI

Asyncio

Uvicorn

Pydantic

Unit testing

Data quality

API design

Tools

Redis

Job description

We are looking for an engineer to drive our Python/FastAPI backend infrastructure, data extraction layers, and agentic workflow engines. We are building heavy backend ingestion pipelines that handle massive scale. Your focus will be on building resilient service layers, processing high-concurrency multi-modal streams, managing token context windows, and engineering strict data quality, filtering, and indexing pipelines.

Our Core Engineering Philosophy: We highly value simple solutions to complex problems. We do not build architecture astronauts or over-engineered abstractions. We want very clean, minimal code that is exceptionally easy to reason about, maintain, and debug. If you take pride in strong code hygiene, rigorous unit testing, and stripping away complexity rather than adding it, you will fit right in.

What You Will Do

Own the FastAPI & Async Infrastructure: Design, optimize, and maintain our core non-blocking service tier. Track down every millisecond of latency, eliminate thread-blocking operations, and manage high-volume API requests under heavy load with clean, minimal patterns.

Build Continuous LLM Evaluation Pipelines (Evals): Design and maintain deterministic regression testing frameworks to make non-deterministic LLM loops predictable. You will build the infrastructure to run statistical evals, validate model output consistency, check schema adherence via Pydantic, and ensure agent stability across robust datasets before code hits production.

Orchestrate Multi-Modal Processing & Indexing: Build ingestion layers that handle unstructured text alongside visual media (images and video). You will build simple, efficient processing loops to decode, validate, filter, and extract data from visual payloads before piping them into our indexing stores or vision-language models.

Optimize Context Window & Token Management: Design stateful memory layers for our agents. Build deterministic algorithms for dynamic context pruning, summary compression, and precise spatial/textual retrieval to keep multi-modal token usage lean and model attention sharp.

Build Extraction, Quality & Filtering Pipelines: Own the data post-processing layer. You will write automated validation, structural filtering, and data-cleansing pipelines to transform raw, noisy web payloads into high-signal data before it hits our agent loops.

Enforce Code Hygiene & Reliability: Write comprehensive unit and integration tests as a natural part of your development workflow. Ensure the codebase remains highly modular, well-documented, and trivial to debug when production edge cases arise.

Orchestrate High-Volume Ingestion: Build the simple, rugged systems that queue requests, handle API rate limits, catch downstream execution failures, and shield our infrastructure from backpressure via Redis.

Engineer Stateful Agent Workflows: Design deterministic control planes for autonomous multi-step loops using graph-based state machines, managing tool execution boundaries and state reconstruction without unnecessary abstraction.

What We Look For

3-5 Years in the Python Backend Trenches: You have built real, data-intensive web services and backends at a scaling or mid-sized tech startup using modern Python frameworks.

Obsession with Simplicity & Testing: A proven track record of refactoring complex problems into elegant, readable code. You write unit tests naturally and look at LLM predictability as a classic regression testing and statistical alignment challenge.

Advanced Asynchronous Python: Heavy hands-on experience with asyncio, uvicorn, and FastAPI. You know how to safely run non-blocking network calls, manage database connection pools under load, and structure async tasks safely.

Multi-Modal Data Familiarity: Experience handling files, media buffers, and unstructured payloads (images/videos). You know how to safely process and structure incoming visual elements without choking your server's memory or blocking the event loop.

Data Quality & Schema Pragmatism: Strong patterns around structured data validation (e.g., Pydantic), test-driven data extraction, and writing deterministic parsing/filtering logic to handle inconsistent incoming payloads.

Pragmatic AI Systems Experience: You know that "agents" aren't just chains of prompts. You have built production systems that deal with tool-calling errors, model hallucinations, infinite loops, and context-window constraints.

Work Authorization: Applicants must be currently authorized to work in the United States on a full-time basis. We are unable to sponsor or take over sponsorship of employment visas (e.g., H-1B) at this time.