Applied AI Engineer: LLM Safety & Evaluation (Remote)

FICO; United States; Remote; USD 105,000 - 165,000; Full time

Benefits offered by this job

Work from Home

Competitive compensation

Benefits

Job summary

FICO is seeking an Applied AI Engineer to own prompts, evaluation prompts, and guardrails for AI-powered consumer features in the myFICO Scores product. The role blends Python backend work with Java (Spring Boot) microservices and ML tooling to deliver safe, reliable AI experiences.

You will design evaluation frameworks, implement prompt-caching and model routing, and collaborate with Compliance, Legal, and Analytics to ensure responsible AI while reducing latency and cost.

Qualifications

• Bachelor's or Master's in CS/Engineering and strong Python and Java skills.
• Proven experience building and operating LLM-powered apps in production.
• Experience with AWS services and data warehouses.

Responsibilities

• Own the system prompts, tool descriptions, and evaluation prompts for AI features.
• Build evaluation frameworks and regression suites before prompts or model changes.
• Lead model upgrades with evaluations and A/B testing, balancing cost and quality.
• Develop tooling for QA analysts, transcript labeling, and dashboards.

Skills

Python

Java/Spring

LLM systems

AWS Bedrock

SQL

Redshift

CI/CD

Agile/Scrum

Education

Bachelor's or Master's in CS/Engineering

Tools

Datadog

Tableau

GitHub Copilot

Job description

FICO is seeking an Applied AI Engineer to own prompts, evaluation prompts, and guardrails for AI-powered consumer features in the myFICO Scores product. The role blends Python backend work with Java (Spring Boot) microservices and ML tooling to deliver safe, reliable AI experiences.

You will design evaluation frameworks, implement prompt-caching and model routing, and collaborate with Compliance, Legal, and Analytics to ensure responsible AI while reducing latency and cost.