AI & ML Infrastructure Architect

Scrim Technologies; United States; Remote; USD 90,000 - 150,000; Full time

Benefits offered by this job

Health coverage for employee anddepend

Hardware support

Global remote work

Job summary

Scrim Technologies seeks an experienced software engineer to design the infrastructure behind production AI features, including language-model workflows, semantic search, and document intelligence.

You will architect scalable inference and retrieval systems, build robust safeguards, and optimize costs while collaborating with product teams on document ingestion and conversational assistants. This global remote role offers equity and health coverage.

Qualifications

• 5+ years of software engineering experience.
• 2+ years focused on LLM or AI infrastructure.
• Strong Python, Rust, or Go skills with asynchronous execution.

Responsibilities

• Architect agent workflows using orchestration frameworks and custom Python or Rust workers.
• Build retrieval-augmented generation and vector search systems with Pinecone, Qdrant, or pgvector.
• Create evaluation tooling, structured-output validation, and safeguards for unreliable model responses.
• Reduce inference cost and latency through caching, model tiering, and related optimization techniques.
• Partner with product teams on document ingestion, automated triage, and conversational assistants.

Skills

Python

Rust

Go

Asynchronous execution

Tools

Pinecone

Qdrant

pgvector

Job description

Role overview

Design the infrastructure behind production AI features, including language-model workflows, semantic search, and document intelligence. This role combines system architecture and hands‑on engineering to make inference and retrieval scalable, efficient, observable, and controlled by reliable quality safeguards.

Responsibilities

• Architect agent workflows using orchestration frameworks and custom Python or Rust workers.
• Build retrieval‑augmented generation and vector search systems with technologies such as Pinecone, Qdrant, or pgvector.
• Create evaluation tooling, structured‑output validation, and safeguards to detect or limit unreliable model responses.
• Reduce inference cost and latency through caching, model tiering, and related optimization techniques.
• Partner with product teams on capabilities such as document ingestion, automated triage, and conversational assistants.

Requirements

• At least 5 years of software engineering experience, including 2 or more years focused on LLM or AI infrastructure.
• Strong Python, Rust, or Go skills and a solid grasp of asynchronous execution.
• Production experience with vector search and embedding systems.
• Familiarity with current hosted and self‑managed model APIs.
• Experience with distributed systems, rate limiting, token‑use optimization, and AI‑service observability.

Nice to have

• Fine‑tuning open‑weight models with LoRA or QLoRA.
• Contributions to open‑source AI tooling or published work on ML systems.
• Experience with edge AI runtimes or WebAssembly inference.

Benefits and work setup

• Full‑time, globally remote role with flexible, asynchronous work practices.
• Compensation is listed at $90,000–$150,000 USD annually plus equity.
• The role description mentions health coverage for the employee and direct dependents, and hardware support.