Senior AI & LLM Systems Engineer

Whizzly Lab; United States; Remote; USD 120,000 - 150,000; Full time

Job summary

Whizzly Lab is hiring an experienced software engineer to design and operate production AI systems that combine language models, agent workflows and multimodal retrieval. The work spans architecture, model deployment and evaluation, with a focus on responsive services that can handle substantial query volume.

You will build multi-agent workflows, deploy inference services with latency targets, and optimize reliability at scale.

Qualifications

• 4+ years of software engineering experience.
• Strong Python and FastAPI skills.
• Experience operating LLM applications in production.
• Knowledge of vector databases and semantic search methods.

Responsibilities

• Build stateful, multi-agent workflows using orchestration frameworks or custom components.
• Develop hybrid retrieval pipelines, including vector search, chunking, reranking and semantic caching.
• Fine-tune open-weight models and deploy inference services with strict latency targets.
• Create automated evaluations to detect hallucinations and catch prompt regressions.
• Collaborate with product leaders and customer teams to define and deliver AI features.

Skills

Python

FastAPI

Production LLMs

CI/CD

Orchestration

Tools

Vector databases

Semantic search

Docker

AWS

GCP

Orchestration frameworks

Job description

Role overview

Design and operate production AI systems that combine language models, agent workflows and multimodal retrieval. The work spans architecture, model deployment and evaluation, with a focus on responsive services that can handle substantial query volume.

Responsibilities

• Build stateful, multi-agent workflows using orchestration frameworks or custom components.
• Develop hybrid retrieval pipelines, including vector search, chunking, reranking and semantic caching.
• Fine-tune open-weight models and deploy inference services with strict latency targets.
• Create automated evaluations to detect hallucinations and catch prompt regressions.
• Collaborate with product leaders and customer teams to define and deliver AI features.

Requirements

• At least four years of software engineering experience, with strong Python and FastAPI skills.
• Experience operating reliable LLM applications in production.
• Knowledge of vector databases and semantic search methods.
• Understanding of token costs, streaming responses and prompt design.
• Experience with Docker, cloud deployments on AWS or GCP, and CI/CD.