Client-Facing Language Model Engineer

Artificial Analysis Inc

Melbourne

On-site

AUD 140,000 - 210,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity

Job summary

Artificial Analysis is seeking a Forward Deployed Engineer to own the language model benchmarking stack, onboard new models, run evaluations, and serve as the technical face for AI labs and enterprise customers.

You’ll operate a Python-based pipeline, debug issues across the stack, and collaborate with engineering to refine processes while staying current with model releases and industry changes. Melbourne on-site with SF collaboration opportunities.

Qualifications

  • 3+ years in a client-facing technical role (solutions engineering, support engineering, technical consulting, or similar).
  • Strong Python proficiency and ability to work with large codebases you didn’t write.
  • Hands-on experience with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.).
  • Excellent debugging skills; trace issues across APIs, data pipelines and code.

Responsibilities

  • Operate and maintain the Python-based language model benchmarking pipeline end-to-end: onboard new models, configure evaluations, run benchmarks and validate results.
  • Debug issues across the stack: API timeouts, errors and unexpected outputs; resolve quickly.
  • Serve as the primary technical contact for AI lab customers: explain results, answer methodology questions and troubleshoot integration via Slack and video conferencing.
  • Monitor benchmark runs for anomalies and ensure accuracy and integrity of published results.
  • Maintain documentation of processes, known issues, and model-specific configurations.
  • Collaborate with the engineering team to flag pipeline improvements and contribute to refinements.
  • Stay current with new model releases, API changes, and developments across the language model ecosystem.

Skills

Python proficiency
Client-facing experience
AI/ML model APIs
Debugging skills
English communication
Ownership of customer issues
Attention to detail

Tools

Slack
Video conferencing

Job description

Artificial Analysis is seeking a Forward Deployed Engineer to own the language model benchmarking stack, onboard new models, run evaluations, and serve as the technical face for AI labs and enterprise customers.

You’ll operate a Python-based pipeline, debug issues across the stack, and collaborate with engineering to refine processes while staying current with model releases and industry changes. Melbourne on-site with SF collaboration opportunities.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Forward Deployed Engineer - Language Models
Forward Deployed Engineer - Language Models

Artificial Analysis Inc • Melbourne

On-site
AUD 140,000 - 210,000
Equity
Senior AI/ML Engineer - Benchmarking (Melbourne On-site)
Senior AI/ML Engineer - Benchmarking (Melbourne On-site)

FDE Times • Melbourne

Hybrid
AUD 150,000 - 190,000
Equity
Competitive compensation
Forward Deployed Engineer - Sydney
Forward Deployed Engineer - Sydney

FDE Times • Sydney

Hybrid
AUD 180,000 - 240,000
Relocation assistance
Up to 50% travel
Forward Deployed Engineer - Sydney
Forward Deployed Engineer - Sydney

FDE Times • Sydney

Hybrid
AUD 180,000 - 280,000
Hybrid work model
Relocation assistance
Travel up to 50%
Senior AI / ML Engineer
Senior AI / ML Engineer

FDE Times • Melbourne

Hybrid
AUD 150,000 - 190,000
Equity
Competitive compensation
Senior AI / ML Engineer
Senior AI / ML Engineer

Artificial Analysis Inc • Melbourne

On-site
AUD 180,000 - 240,000
Equity
On-site work in SF & Melbourne
Senior Forward-Deployed AI Engineer
Senior Forward-Deployed AI Engineer

Accenture Global • Brisbane

On-site
AUD 180,000 - 240,000
18 weeks parental leave
Career development program
Flexible Work Arrangements
+2
Forward Deployed Engineer - AUS
Forward Deployed Engineer - AUS

OpenAI, Inc. • Sydney

Hybrid
AUD 180,000 - 230,000
Senior Forward Deployed Engineer (Ai)
Senior Forward Deployed Engineer (Ai)

Nuage Tech Group • Melbourne

Hybrid
AUD 180,000 - 280,000
Competitive Salary
Well Established Enterprise
Flexible Hybrid Model
Sydney-Based Frontier Model Deployment Engineer
Sydney-Based Frontier Model Deployment Engineer

FDE Times • Sydney

Hybrid
AUD 180,000 - 280,000
Hybrid work model
Relocation assistance
Travel up to 50%
Top locations