Frontline Language Model Engineer (Client-Facing)

Artificial Analysis, Inc.; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive compensation including **e

Job summary

Artificial Analysis, based in San Francisco, is seeking a Forward Deployed Engineer to own the day‑to‑day operation of our language model benchmarking stack. You’ll onboard new models to our evaluation pipeline, run benchmarks, and serve as the primary technical contact for AI lab customers: explaining results, answering methodology questions, and resolving endpoint issues in real time.

This deployed role sits at the interface between our platform and sophisticated customers, requiring strong

Qualifications

• 3+ years in a client-facing technical role such as solutions engineering or consulting.
• Strong Python proficiency with large, complex codebases.
• Hands-on experience with AI/ML model APIs (OpenAI, Anthropic, Google, Meta).
• Excellent debugging skills across APIs, data pipelines, and code.
• Strong English written and verbal communication; ability to explain concepts clearly.
• Reliable, proactive, and takes ownership of customer issues.

Responsibilities

• Operate and maintain the Python-based language model benchmarking pipeline end-to-end.
• Onboard new models, configure evaluations, run benchmark runs, and validate results.
• Debug issues across the stack and resolve them quickly.
• Serve as the primary technical contact for AI lab customers; explain results and address questions.
• Monitor runs for anomalies and ensure accuracy of published results.
• Maintain documentation of processes and model-specific configurations.
• Collaborate with engineering to propose pipeline improvements and refinements.
• Stay current with new model releases, API changes, and ecosystem developments.

Skills

Python

Client-facing experience

AI/ML model API experience

Debugging skills

English communication

Ownership mindset

Job description

Artificial Analysis, based in San Francisco, is seeking a Forward Deployed Engineer to own the day‑to‑day operation of our language model benchmarking stack. You’ll onboard new models to our evaluation pipeline, run benchmarks, and serve as the primary technical contact for AI lab customers: explaining results, answering methodology questions, and resolving endpoint issues in real time.

This deployed role sits at the interface between our platform and sophisticated customers, requiring strong