LLM Engineering Expert (Freelancing)

Zettamine Labs; United States; Remote; USD 165,000 - 276,000; Full time

Job summary

Zettamine Labs is seeking an experienced LLM Engineering Expert for freelance AI evaluation and engineering simulation work. The project focuses on creating complex simulation problems, validating outputs, and analyzing AI trajectories using open-source tools.

Strong background in engineering and Python is required for remote collaboration and rapid deployment. Ideal candidates have 10+ years in engineering design, advanced degrees, and hands-on experience with LLMs and AI evaluation.

Qualifications

• 10+ years of hands-on engineering design experience.
• Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering or closely related field.
• Strong Python programming/scripting skills.
• Experience with engineering simulation/tooling platforms listed.
• Experience with LLMs, AI coding agents, or AI evaluation.

Responsibilities

• Create complex, self-contained engineering design and simulation problems for AI evaluation.
• Define technical constraints, objectives, reference solutions, and evaluation criteria.
• Build and validate engineering simulation environments using open-source tools.
• Develop Python-based scripts, test benches, and automated graders.
• Analyze AI agent outputs, trajectories, and execution logs to identify issues.
• Identify failure modes such as simulator interpretation errors, premature convergence, and tool-use mistakes.
• Refine task difficulty based on model performance and evaluation results.
• Ensure physical plausibility, unit consistency, boundary conditions, convergence, and accuracy.
• Collaborate with AI researchers and engineering teams to improve benchmarks.

Skills

Engineering design

Python programming

AI evaluation

Engineering simulation

LLMs / AI agents

Education

Master's or PhD in Electrical, Mechanical, or Aerospace Engineering

Tools

ngspice

PySpice

OpenFOAM

FEniCSx

CalculiX

python-control

CadQuery

build123d

OpenModelica

Cantera

Gmsh

Job description

Job Title: LLM Engineering Expert AI Evaluation / Engineering Simulation (Freelancing)

Job Type: Freelance
Contract Duration: Up to 24 weeks
Experience: 10+ years
Work Mode: Remote
Availability: 40 hours/week with 4 hours overlap with PST
Joining: Immediate

Job Overview

We are looking for experienced Engineering Experts with strong LLM/AI evaluation and engineering simulation experience to work on advanced AI evaluation and benchmarking projects.

The role involves creating and validating complex engineering design problems for AI agents, working with open-source simulation tools, analyzing AI-generated solutions and execution logs, and identifying reasoning, coding, and tool-use failures.

Candidates should have a strong engineering background combined with hands-on experience in Python, engineering simulation, and modern LLM/coding-agent evaluation.

Key Responsibilities

• Create complex, self-contained engineering design and simulation problems for AI evaluation.
• Define technical constraints, optimization objectives, reference solutions, and objective evaluation criteria.
• Build and validate engineering simulation environments using open-source simulation tools.
• Develop Python-based scripts, test benches, and automated graders.
• Analyze AI agent outputs, coding trajectories, and execution logs.
• Identify failure modes such as incorrect simulator interpretation, premature convergence, invalid designs, and tool-use errors.
• Refine task difficulty based on model performance and evaluation results.
• Ensure physical plausibility, unit consistency, boundary conditions, convergence, and technical accuracy.
• Collaborate with AI researchers, domain experts, and engineering teams to improve AI evaluation benchmarks.

Mandatory Skills

• 10+ years of hands-on engineering design experience.
• Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering, or a closely related engineering discipline.
• Strong Python programming/scripting skills.
• Hands-on experience with at least one engineering simulation/tooling platform such as:
• ngspice
• PySpice
• OpenFOAM
• FEniCSx
• CalculiX
• python-control
• CadQuery
• build123d
• OpenModelica
• Cantera
• Gmsh
• Experience working with LLMs, AI coding agents, or AI evaluation.
• Understanding of AI evaluation concepts such as pass@k, failure-mode analysis, and nondeterministic behavior.
• Ability to analyze AI agent trajectories/logs and identify reasoning or tool-use failures.
• Strong understanding of engineering fundamentals, physical plausibility, units, boundary conditions, and convergence criteria.

Preferred Background :

• Masters degree or PhD in Electrical Engineering, Mechanical Engineering, Aerospace Engineering, or a closely related engineering discipline.
• Strong academic or professional background in engineering design, simulation, computational engineering, or related technical domains.
• Candidates with relevant practical engineering expertise and equivalent experience may also be considered, subject to the role requirements.

Important Note

This is a short-term contract/freelance opportunity focused on engineering simulation and AI evaluation rather than a conventional permanent engineering position.