Senior LLM Systems Engineer - Production Excellence

Risk Labs; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Risk Labs is seeking a Senior LLM Systems Engineer to strengthen its LLM components within an oracle automation stack. The role emphasizes improving accuracy, performance, and resilience of the systems that reason about prediction market rules and outcomes.

You will design validation processes, build evaluation datasets, and monitor system performance to ensure high-quality decisions and reliable operations in production environments.

Qualifications

• Strong background in LLMs and production systems.
• Experience improving prompts, model selection, and evaluation datasets.
• Ability to debug and optimize within uncertain environments.

Responsibilities

• Enhance the LLM-driven components of Risk Labs' oracle automation stack, focusing on system accuracy, performance, and resilience.
• Design validation processes, build evaluation datasets, and monitor system performance to maintain decision quality and operational reliability.
• Improve prompts, model selection, and operational quality to ensure systems can reason about prediction market rules and outcomes.

Skills

LLMs

System performance

Operational resilience

Prompt engineering

Model selection

Job description

Risk Labs is seeking a Senior LLM Systems Engineer to strengthen its LLM components within an oracle automation stack. The role emphasizes improving accuracy, performance, and resilience of the systems that reason about prediction market rules and outcomes.

You will design validation processes, build evaluation datasets, and monitor system performance to ensure high-quality decisions and reliable operations in production environments.