LLM Systems Engineer — Applied AI in Production

ICF Olson; United States; Remote; USD 99,000 - 168,000; Full time

Job summary

ICF Olson is seeking a skilled engineer to design and deploy LLM-powered survey tools. You will build end-to-end data-to-report pipelines, enable multilingual questionnaire translation, and support field staff with robust analytics.

You will collaborate with technical leads to define acceptance criteria, evaluate tool performance, and document results for public release. This remote US role may involve travel as needed.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• Minimum 4 years of professional software development experience, including LLM-powered systems.
• U.S. citizenship is required by federal government contract.
• Proficiency in Python or another modern language for backend and AI apps.
• Hands-on with frontier-model APIs and RAG techniques.
• Experience designing LLM evaluations and test sets.
• Experience with cloud platforms (AWS/Azure/GCP).
• Experience using agentic coding tools to produce and own production code.
• Experience with agent frameworks and AI evaluation tooling.

Responsibilities

• Design and build LLM pipelines and agents for survey workflows.
• Translate multilingual questionnaires and support field tools for interviewers.
• Build evaluation harnesses and run non-inferiority comparisons.
• Route work across models based on complexity, volume, cost, latency, and data requirements.
• Monitor costs, token usage, and batch processing; inform model-selection decisions.
• Connect models to data services and full-stack applications; produce and review code.
• Document methods and results for public release, including negative results.
• Transfer tools to partner-country institutions for local use.

Skills

Python

Frontier-model APIs

LLM evaluations

Cost & latency trade-offs

Cloud platforms

Agentic coding tools

Agent frameworks

RAG techniques

Education

Bachelor's degree in Computer Science or related field

Tools

Claude Code

OpenAI Codex

Cursor

Antigravity

GitHub Copilot

Job description

ICF Olson is seeking a skilled engineer to design and deploy LLM-powered survey tools. You will build end-to-end data-to-report pipelines, enable multilingual questionnaire translation, and support field staff with robust analytics.

You will collaborate with technical leads to define acceptance criteria, evaluate tool performance, and document results for public release. This remote US role may involve travel as needed.