Lead LLM Engineer

Fully Powered

Los Angeles (CA)

On-site

USD 120,000 - 190,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Fully Powered AI, based in Los Angeles, seeks a leader to drive experiments and engineering for dependable AI-generated workflows. You will shape how workflows are extracted from interviews and docs, build robust evaluations, and compare prompting, retrieval, and fine-tuning against baselines.

You’ll design datasets with clear provenance and help translate results into maintainable features. We welcome candidates who can trace failures to data, context, or code, and who can run model training

Qualifications

  • You have built LLM applications and can trace an output failure back to the data, context, instructions, model or surrounding code.
  • You have run model training or fine-tuning experiments and can explain the baseline, data choices and measured result.
  • You write strong Python and can build reproducible evaluation and data pipelines.
  • You can distinguish a persuasive demonstration from a repeatable improvement, and make that distinction clear to the team.

Responsibilities

  • Lead the experiments, evaluations and engineering that make generated workflows more useful and dependable.
  • Improve how we extract workflow structure from interviews, documents and examples, including missing information and conflicting instructions.
  • Build evaluations for generated workflows: completeness, factual grounding, exception handling and whether the work meets the agreed standard.
  • Compare prompting, retrieval, structured generation and fine-tuning against explicit baselines, including quality, latency and cost.
  • Design datasets and experiments for specialized models, with clear provenance and separation of training and evaluation material.
  • Turn promising results into maintainable product features, document failure modes and work with implementation teams to test improvements.

Skills

Python
LLM applications
Evaluation pipelines
Experiment design

Tools

Experiment tracking tools

Job description

Help AI understand how work should be done. Lead the experiments, evaluations and engineering that make generated workflows more useful and dependable.

  • Improve how we extract workflow structure from interviews, documents and examples, including missing information and conflicting instructions.
  • Build evaluations for generated workflows: completeness, factual grounding, exception handling and whether the work meets the agreed standard.
  • Compare prompting, retrieval, structured generation and fine-tuning against explicit baselines, including quality, latency and cost.
  • Design datasets and experiments for specialized models, with clear provenance, appropriate permissions and separation of training and evaluation material.
  • Turn promising results into maintainable product features, document failure modes and work with implementation teams to test improvements.
Show us how you think.

Bring examples that show how you learn from model behavior and turn that learning into better software.

  • You have built LLM applications and can trace an output failure back to the data, context, instructions, model or surrounding code.
  • You have run model training or fine-tuning experiments and can explain the baseline, data choices and measured result.
  • You write strong Python and can build reproducible evaluation and data pipelines.
  • You can distinguish a persuasive demonstration from a repeatable improvement, and make that distinction clear to the team.
What early progress looks like.

A useful evaluation set for workflow building, a reproducible baseline and a small set of experiments that show which changes improve results. That evidence should guide both the next product release and the direction of model research.

The company you're helping build.

Fully Powered AI is an AI native services startup based in Los Angeles. We build our own technology to help companies capture the way their best people work and establish functions they don't yet have. Fully Powered Studio and reusable workflow foundations turn that knowledge into AI workflows teams can use, check and improve.

Our next ambition is to develop models specialized for building business workflows. You'll lead that research direction, including fine-tuning experiments, while improving what we can deliver with existing models today. The question is practical: what helps a model capture the judgment in a job and turn it into a workflow someone can use?

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead LLM Engineer: Workflow Architect and Evaluator
Lead LLM Engineer: Workflow Architect and Evaluator

Fully Powered • Los Angeles (CA)

On-site
USD 120,000 - 190,000
Forward Deployed Engineer
Forward Deployed Engineer

Fully Powered • Los Angeles (CA)

On-site
USD 120,000 - 180,000
Application Engineer – LLM
Application Engineer – LLM

Salt • United States

On-site
USD 120,000 - 180,000
Senior Machine Learning Engineer
Senior Machine Learning Engineer

Arionkoder • United States

Remote
USD 150,000 - 230,000
Monthly USD Service Fee
Paid Agreed Time Off
Caregiver Time Off
+2
Lead/Staff Engineer - Applied AI
Lead/Staff Engineer - Applied AI

US based SaaS company that offers AI-powered business operating system • United States

Remote
USD 180,000 - 280,000
AI Engineer - FL
AI Engineer - FL

LawPro.ai • Florida

On-site
USD 140,000 - 210,000
AI Engineer - VA
AI Engineer - VA

LawPro.ai • Virginia

On-site
USD 150,000 - 190,000
AI Engineer
AI Engineer

Data • Union City (NJ)

On-site
USD 120,000 - 150,000
AI Engineer
AI Engineer

Tryscope • San Francisco (CA)

Hybrid
USD 180,000 - 210,000
Senior AI Engineer
Senior AI Engineer

Weblink Solutions • United States

Remote
USD 140,000 - 230,000