Software Engineer, Developer Productivity & Testing

Precision Labs; San Francisco, Northern (CA, KY); Hybrid; USD 175,000 - 240,000; Full time

Benefits offered by this job

Medical, dental, and vision

401(k)

Meals on in-office days

Job summary

LangChain is hiring a Software Engineer to join the Infrastructure team and own developer productivity across LangGraph Cloud/Platform and LangSmith products. You will work with Infrastructure, Backend, and Frontend teams to ship confidently across Kubernetes-based services, APIs, and UI flows in person 5 days/week in San Francisco, CA or New York, NY.

You will help pioneer quality practices for LLM applications, including prompt regression testing and evaluation suites, with a focus on

Qualifications

• 3+ years of experience as a software engineer or infrastructure engineer.
• Strong hands-on experience with Python and testing frameworks such as pytest.
• Experience working with CI/CD systems (GitHub Actions preferred) and improving pipeline performance and reliability.
• Solid understanding of API testing, mocking/stubbing, and data setup/teardown.
• Comfort defining quality standards, writing test plans, and driving cross-team execution.

Responsibilities

• Own test strategy end-to-end across APIs, services, UI, data, and infrastructure (Kubernetes, Terraform, Helm).
• Stand up ephemerous test environments in Kubernetes for pull requests and release candidates; seed test data and run hermetic test suites.
• Shift quality earlier in CI/CD pipelines (GitHub Actions) through parallelization, caching, deterministic seeds, flake tracking, and quality gates.
• Build observability into testing workflows with rich failure artifacts such as logs, traces, and dashboards.
• Establish performance and reliability baselines for critical paths, including SLIs, SLOs, and regression detection.
• Partner on incident workflows by reproducing issues, adding targeted regression tests, and improving runbooks and postmortems.
• Write documentation including test plans, playbooks, and contributor guidelines for writing high-quality tests.
• Example projects you might own: PR ephemeral end-to-end testing harness, k6 performance scenario, flake-budget system.

Skills

Python

pytest

GitHub Actions

API testing

CI/CD

Data setup/ teardown

Tools

Kubernetes

Terraform

Helm

Datadog

OpenTelemetry

Job description

LangChain is hiring a Software Engineer to join the Infrastructure team and own developer productivity across LangGraph Cloud/Platform and LangSmith products. You will work with Infrastructure, Backend, and Frontend teams to ship confidently across Kubernetes-based services, APIs, and UI flows in person 5 days/week in San Francisco, CA or New York, NY.

You will help pioneer quality practices for LLM applications, including prompt regression testing and evaluation suites, with a focus on