LLM Platform Engineer

allspice

United States

On-site

USD 180,000 - 240,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

AllSpice is seeking a senior software engineer at the intersection of AI platform development, data engineering, and full-stack product development. You will own the tooling, evaluation systems, and data pipelines that make AllSpice's Generative AI team more effective.

Your primary users will be internal engineers and product managers, with opportunities to build customer-facing experiences where they support our AI products.

Qualifications

  • Experience building AI platforms, data pipelines, and internal tooling.
  • Ability to drive projects from discovery to deployment.
  • Strong collaboration with cross-functional teams.
  • Experience with CI/CD, observability, and model evaluation workflows.
  • Excellent communication of complex findings to diverse audiences.
  • Growth mindset and willingness to contribute to core product features.

Responsibilities

  • Partner with engineers and product managers to discover workflow bottlenecks, prioritize the highest-impact problems, and own solutions end-to-end—from requirements and architecture to rollout and iteration.
  • Own the evaluation and observability layer for our AI products, including automated and human review workflows, regression detection, and comparisons across models, prompts, and pipelines. Make quality, cost, latency, usage, and customer outcomes visible and actionable.
  • Build usable internal applications and self-service tools that help engineers and product managers inspect AI results, investigate failures, and provide feedback. Where useful, extend these capabilities into customer-facing experiences.
  • Build and maintain scripts, reliable data pipelines, and integrations that connect internal systems, third‑party services, LLMs, and Model Context Protocol (MCP) tools, with attention to data quality, schema evolution, and customer data privacy.
  • Write clear documentation and communicate findings to both technical and non-technical audiences, including leadership and customers.
  • Champion code quality, testing, and reliability across the tooling stack through reviews, mentorship, and thoughtful design.

Job description

Help define the future of hardware development! At AllSpice , we're building the automation engine that powers the next generation of circuit design, enabling breakthroughs in smart vehicles, robotics, rockets, IoT, medical devices , and more.

We're creating the agile development environment for hardware engineers with a Git-friendly translation layer and automated CI/CD framework for native circuit designs. Think GitHub/GitLab + Copilot for electronics.

Our mission is to bring the same modern workflows that revolutionized software: version control, automation, and collaboration to the hardware world.

Learn more about our journey in TechCrunch and our recent Series A announcement .

If you're passionate about building tools that empower teams and delight customers, and you want to shape the foundation of modern hardware DevOps, we'd love to meet you!

Read more at https://allspice.io

What will you do?

This is a senior, high-impact software engineering role at the intersection of AI platform development, data engineering, and full-stack product development. You will own the tooling, evaluation systems, and data platforms that make AllSpice's Generative AI team more effective. Your primary users will be internal engineers and product managers, with opportunities to build customer-facing experiences where they support our AI products.

We're looking for someone who can identify the right problems, make thoughtful technical decisions, and take solutions from discovery through implementation, adoption, and ongoing improvement. As you build domain knowledge and establish the AI platform foundation, we'd also like you to grow into contributing directly to AllSpice's core product alongside the product engineering team. This is an aspirational growth opportunity, not a prerequisite or the role's initial primary focus.

You will:
  • Partner with engineers and product managers to discover workflow bottlenecks, prioritize the highest-impact problems, and own solutions end-to-end—from requirements and architecture to rollout and iteration.
  • Own the evaluation and observability layer for our AI products, including automated and human review workflows, regression detection, and comparisons across models, prompts, and pipelines. Make quality, cost, latency, usage, and customer outcomes visible and actionable.
  • Build usable internal applications and self-service tools that help engineers and product managers inspect AI results, investigate failures, and provide feedback. Where useful, extend these capabilities into customer-facing experiences.
  • Build and maintain scripts, reliable data pipelines, and integrations that connect internal systems, third‑party services, LLMs, and Model Context Protocol (MCP) tools, with attention to data quality, schema evolution, and customer data privacy.
  • Write clear documentation and communicate findings to both technical and non-technical audiences, including leadership and customers.
  • Champion code quality, testing, and reliability across the tooling stack through reviews, mentorship, and thoughtful design.
Example Projects
  • Build an internal evaluation workspace for DRCY, our AI design review product, that captures run configurations, inputs, outputs, and telemetry; compares model and prompt changes; and tracks quality, false positives, cost, and latency over time.
  • Develop review workflows that let engineers and product managers inspect AI-generated findings, apply scoring rubrics, override automated scores, and investigate regressions—with opportunities to surface relevant insights and feedback workflows to customers.
  • Create and maintain data pipelines that aggregate evaluation results, customer feedback, and telemetry into a unified analytics layer.
  • Build scripting tools and automations to streamline the Gen AI team's development and testing workflows (e.g., batch evaluation runs, report generation).
  • Design and implement RAG‑based information repositories and tooling to capture component data and domain knowledge for use by our suite of AI products.
  • Integrate with CI/CD systems (GitHub Actions) to surface AI insights directly in engineering workflows.
  • Partner with infrastructure to measure cache effectiveness, usage, and cost, and use those insights to improve the reliability and efficiency of AI workflows.
  • As an aspirational next step, contribute directly to core AllSpice product features—for example, integrating AI review findings and feedback into existing engineering workflows—in partnership with the product engineering team.
Our stack

Languages: Python, TypeScript/JavaScript, SQL

Analytics & Data: SQL databases (Postgres), Jupyter Notebooks, business intelligence/analytics tools

AI/LLM: Langchain, Pydantic AI, Langfuse

Web: Vue/TypeScript front-end, common web architectures and APIs

Infrastructure: GitHub Actions for CI/CD, Docker Swarm & Terraform, AWS

Our Hub application is a soft fork of Gitea

Go [server-side]

Vue/TypeScript Front-end

Expectations

Our ideal candidate sh

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior/Staff Software Engineer, AI Platform
Senior/Staff Software Engineer, AI Platform

Benchstrength • San Francisco (CA)

Hybrid
USD 180,000 - 260,000
Flexible work
Competitive salary & equity
Health, dental, vision insurance
+2
Senior/Staff Software Engineer, AI Platform
Senior/Staff Software Engineer, AI Platform

AllSpice.io • San Francisco (CA)

Hybrid
USD 160,000 - 190,000
Equity
Health, dental, vision benefits
Flexible work arrangements
+1
AI Engineer
AI Engineer

Valsoft Corporation • United States

Remote
USD 150,000 - 210,000
Lead Engineer - AI Developer Platform
Lead Engineer - AI Developer Platform

Raymond James & Associates • Saint Petersburg (FL)

Hybrid
USD 150,000 - 230,000
Full-Stack Engineer
Full-Stack Engineer

IP Author Inc • United States

On-site
USD 120,000 - 180,000
AI Platform Engineer - Hardware DevOps & Tools
AI Platform Engineer - Hardware DevOps & Tools

allspice • United States

On-site
USD 180,000 - 240,000
VP of Product — Hardware Workflow Platform
VP of Product — Hardware Workflow Platform

Benchstrength • San Francisco (CA)

Hybrid
USD 265,000 - 315,000
Health insurance
Vision insurance
Dental insurance
+3
Staff Software Engineer, Applied AI
Staff Software Engineer, Applied AI

Emergence Capital Partners • San Francisco (CA)

On-site
USD 160,000 - 230,000
VP of Product
VP of Product

Benchstrength • San Francisco (CA)

Hybrid
USD 265,000 - 315,000
Health insurance
Vision insurance
Dental insurance
+3
Lead Full Stack Engineer
Lead Full Stack Engineer

Winston Artory Group • New York (NY)

Remote
MXN 1,000,000 - 1,400,000
Meaningful equity
Competitive compensation
Remote-friendly culture
+1