Agentic Engineer

Perform Digital; United States; Remote; USD 140,000 - 200,000; Full time

Job summary

Perform Digital is seeking a hands-on software engineer to build, fine-tune, evaluate, and operate specialized AI agents for enterprise clients. The role blends applied research with production engineering, from choosing agent patterns to ensuring observability and cost efficiency at scale.

You will own agent design, curate datasets for fine-tuning, and optimize prompt architecture, retrieval, and latency. Strong Python, plus TypeScript/Go, and cloud experience are required.

Qualifications

• 4+ years of software engineering experience with 1 year shipping LLM-based products to production.
• Strong Python skills, plus TypeScript or Go for glue code and APIs.
• Hands-on production experience with major providers (Anthropic, OpenAI) or open-weight models on self-managed infra.
• Experience with fine-tuning workflows: LoRA, SFT, dataset curation, eval harnesses.
• Production RAG experience: vector stores, embedding choices, chunking, reranking, hybrid retrieval.
• Cost and latency aware: read dashboards and identify optimization levers.
• Comfort with AWS, GCP or Azure and one container orchestrator (ECS, Kubernetes).

Responsibilities

• Own agent design from problem framing to production, selecting patterns like tool-use orchestration, RAG, multi-step planning, and reflection loops.
• Curate datasets and run LoRA or full supervised fine-tunes when off-the-shelf models fall short, evaluating them against a harness.
• Engineer prompt architecture, retrieval strategy, chunking, reranking, and eval-driven prompt iteration.
• Optimize for cost and latency through quantization, batching, streaming, and fallback strategies.
• Instrument observability from day one, including tracing, regression suites, online evals, and drift alerts.

Skills

Python

TypeScript/Go

LLM production

Cost & latency awareness

Cloud platforms (AWS/GCP/Azure)

Kubernetes / ECS

Tools

lm-eval-harness

Vectors / embedding stores

Job description

Role overview

A hands-on engineering role focused on building, fine-tuning, evaluating, and operating specialized AI agents for enterprise clients, end to end. The work blends applied research with production engineering — from choosing the right agent pattern to ensuring the resulting system is observable, cost-efficient, and reliable at scale.

Responsibilities

• Own agent design from problem framing to production, selecting patterns such as tool-use orchestration, RAG, multi-step planning, and reflection loops
• Curate datasets and run LoRA or full supervised fine-tunes when off-the-shelf models fall short, evaluating them against a written harness
• Engineer prompt architecture, retrieval strategy, chunking, reranking, response caching, and eval-driven prompt iteration
• Optimize for cost and latency through quantization, batching, streaming, response budgets, and fallback strategies
• Instrument observability from day one, including tracing, regression suites, online evals, and drift alerts

Requirements

• 4+ years of software engineering experience, with at least 1 year shipping LLM-based products to production
• Strong Python skills, plus enough TypeScript or Go to ship glue and APIs
• Hands-on production experience with at least one major model provider (Anthropic, OpenAI, or open-weight Llama or Mistral on self-managed infrastructure)
• Practical experience with fine-tuning workflows — LoRA, full SFT, dataset curation, eval harnesses such as lm-eval-harness or equivalents, and basic reward modeling
• Production RAG experience covering vector stores, embedding choice, chunking strategies, reranking, and hybrid retrieval
• Cost and latency awareness, with the ability to read a billing dashboard or flamegraph and identify the right lever to pull
• Comfort with at least one major cloud (AWS, GCP, or Azure) and one container orchestrator (ECS, Kubernetes, or equivalent)

Nice to have

• Production experience with quantization techniques such as GPTQ, AWQ, or bitsandbytes
• Having built or contributed to an evaluation framework that runs in CI
• Experience working with on-prem or VPC-deployed models for compliance reasons