Senior LLM Systems Engineer

Risk Labs; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Risk Labs is seeking a Senior LLM Systems Engineer to strengthen its LLM components within an oracle automation stack. The role emphasizes improving accuracy, performance, and resilience of the systems that reason about prediction market rules and outcomes.

You will design validation processes, build evaluation datasets, and monitor system performance to ensure high-quality decisions and reliable operations in production environments.

Qualifications

• Strong background in LLMs and production systems.
• Experience improving prompts, model selection, and evaluation datasets.
• Ability to debug and optimize within uncertain environments.

Responsibilities

• Enhance the LLM-driven components of Risk Labs' oracle automation stack, focusing on system accuracy, performance, and resilience.
• Design validation processes, build evaluation datasets, and monitor system performance to maintain decision quality and operational reliability.
• Improve prompts, model selection, and operational quality to ensure systems can reason about prediction market rules and outcomes.

Skills

LLMs

System performance

Operational resilience

Prompt engineering

Model selection

Job description

Risk Labs seeks a Senior LLM Systems Engineer to enhance its LLM components within an oracle automation stack. Ideal candidates have experience with LLMs, system performance, and operational resilience.

In this role, you will enhance the LLM-driven components of Risk Labs' oracle automation stack, focusing on system accuracy, performance, and resilience. You will be responsible for improving prompts, model selection, and operational quality, ensuring that the systems can effectively reason about prediction market rules and outcomes.

You will also design validation processes, build evaluation datasets, and monitor system performance to maintain decision quality and operational reliability. The role requires a strong background in LLMs and production systems, along with an ability to debug and optimize within uncertain environments.