Lead Model Routing & Cost-Governed Inference

Dipp AI Technologies; New York, Northern (NY, KY); Hybrid; USD 180,000 - 230,000; Full time

Benefits offered by this job

Equity

Health benefits

401(k) match

Hardware budget

Compute credits

Publish methodology

Job summary

Dipp AI Technologies seeks a senior ML/engineering leader to own the dynamic model routing layer. You will classify tasks, govern evaluation across frontier, open-weight, and private models, and ensure budgets and hard stops are enforced before inference.

You will also govern model entry, shadow-test new models, and document decisions for auditability. The role demands hands-on production experience with LLM systems, strong ML/systems background, and fluency in Python with a systems language for

Qualifications

• Hands-on production experience with LLM systems beyond prototypes.
• Experience building evaluation infrastructure that teams trust.
• Strong applied ML or systems background with rigorous measurement habits.

Responsibilities

• Build task classification and routing policy across frontier, open-weight, and private models.
• Own the evaluation harness that scores models per task class on quality, latency, and cost.
• Implement pre-inference cost governance: budgets, ceilings, degradation paths, and hard stops.
• Enforce enterprise data-boundary rules in routing so no restricted payload reaches an ineligible provider.
• Build shadow evaluation and staged promotion for new models entering the fleet.
• Instrument routing decisions so each one is explainable in the audit ledger.

Skills

LLM systems

ML engineering

Evaluation infrastructure

Python proficiency

Systems programming

Inference economics

Benchmark skepticism

Education

BS/MS/PhD in CS/ML/Statistics

Equivalent applied experience

Tools

LLM serving platforms

Evaluation tooling

Job description

Dipp AI Technologies seeks a senior ML/engineering leader to own the dynamic model routing layer. You will classify tasks, govern evaluation across frontier, open-weight, and private models, and ensure budgets and hard stops are enforced before inference.

You will also govern model entry, shadow-test new models, and document decisions for auditability. The role demands hands-on production experience with LLM systems, strong ML/systems background, and fluency in Python with a systems language for