Backend / Platform Engineer, AI Analytic Engines

ThirdLaw | Runtime AI Safety; United States; On-site; USD 120,000 - 150,000; Full time

Benefits offered by this job

Market cash compensation

Above-market equity

Generous benefits

Job summary

ThirdLaw | Runtime AI Safety is looking for a Backend Software Engineer in the United States to design and build scalable systems for AI safety. You will architect low-latency services and develop core evaluation engines to monitor AI behavior. The ideal candidate has 5+ years of experience in software engineering, proficiency in Python and Go, and expertise in streaming data pipelines. We offer competitive pay, above-market equity, and comprehensive benefits.

Qualifications

• 5+ years of Backend software engineering experience, including designing and shipping production software services.
• Strong coding proficiency in Python and/or Go.
• Deep experience with streaming data pipelines and batch processing systems.

Responsibilities

• Architect scalable, low-latency services for real-time evaluations.
• Design and build the core evaluation engine within ThirdLaw.
• Build a runtime intervention layer based on evaluation results.

Skills

Backend software engineering

Python

Go

Streaming data pipelines

Low-latency services

Cloud-native infrastructure

Observability patterns

Tools

Kafka

gRPC

Kubernetes

Job description

About The Company

ThirdLaw is building the control layer for AI in the enterprise. As companies rush to adopt LLMs and AI agents, they face new safety, compliance, and operational risks that traditional observability tools were never designed to detect. Metrics like latency or cost don’t capture when a model makes a bad decision, leaks sensitive data, or behaves unpredictably. We help IT and Security teams answer the foundational question: "Is this OK?"—and take real-time action when it’s not. Backed by top-tier venture firms and trusted by forward‑looking enterprise design partners, we’re building the infrastructure to monitor, evaluate, and control AI behavior in real‑world environments—at runtime, where it matters. If you’re excited to build systems that help AI work as intended—and stop it when it doesn’t—this is the place for you.

About The Role

You won’t just be piping logs or tuning models—you’ll build and scale systems that reconcile latency, correctness, and observability across distributed pipelines. You’ll be designing the nervous system of a new class of software—where AI Agents reason, act, and fail in unpredictable ways. If you’re excited by AI and want to shape its safe deployment—not just watch from the sidelines—this is your opportunity.

What You’ll Do

• Architect scalable, low‑latency services for running evaluations in real‑time and batch, integrating with streaming data pipelines and trace‑based event models.
• Design and build the core evaluation engine within ThirdLaw that applies heuristics, semantic models, and foundation model calls to detect violations across LLM inputs and outputs.
• Build a runtime intervention layer to determine and execute appropriate enforcement actions—such as block, redact, notify, and escalate—based on evaluation results and risk context.
• Create reusable frameworks for pluggable evaluators and intervention policies, supporting no‑code authoring and automated deployment pipelines.
• Configure and operationalize a vector database pipeline for RAG‑like use cases.
• Build for scale. Mitigate blocking gRPC threads, implement micro‑batching & streaming for LLM/embedding calls, and add reliability controls such as queue‑based back‑pressure and graceful degradation paths.

Required Experience

• 5+ years of Backend software engineering experience, including designing and shipping production software services
• Strong coding proficiency in Python and/or Go
• Deep experience with streaming data pipelines (e.g., Kafka, Pulsar, Redis Streams) and batch processing systems
• Proven track record scaling high‑QPS, low‑latency services (p95/p99 ownership a plus)
• Familiarity with vector databases (e.g., FAISS, Weaviate, Qdrant, pgvector) and embedding‑based matching
• Strong grasp of cloud‑native infrastructure: containers, Kubernetes, serverless functions, CI/CD pipelines
• Exposure to structured observability patterns, including OpenTelemetry (or similar tracing standards)
• Comfortable designing—then defending—trade‑offs around build vs buy vs OSS

Nice-to-Have

• Experience with modern Python APIs & concurrency, e.g. gRPC, FastAPI, asyncio, multithreading/processes
• Familiarity with ClickHouse, Apache Arrow, or fast analytical storage engines
• Prior work on agent frameworks (e.g. LangChain, CrewAI, AutoGen) or LLM orchestration
• Experience in trust & safety, compliance, or AI safety domains
• Hands‑on experience with secure enterprise integrations (authorization/authentication, webhooks, SIEM, IAM)

Why Apply?

Our team is small and focused, valuing autonomy and real impact over titles and management. We need strong technical skills, a proactive mindset, and clear written communication, as much of our work is asynchronous. If you’re organized, take initiative, and want to work closely with customers to shape our products, you’ll fit in well here. We pay market cash compensation and generally above‑market equity; we have well‑designed and generous benefits.