Technical Solutions Architect, Evals & Fine-Tuning

Innodata Inc.; United States; On-site; USD 140,000 - 160,000; Full time

Job summary

Innodata Inc. seeks a Senior Technical Solutions Architect for Evals & Fine-Tuning to serve as the technical face to demanding AI customers in the United States. You will translate client goals into concrete engagements spanning data curation, SFT, RLHF, DPO, and eval harnesses.

This role requires deep ML experience, hands-on fine-tuning, and strong communication to align stakeholders with technical delivery across platforms and global SME teams.

Qualifications

• 7+ years of experience in applied ML or ML engineering, research, or technical solutions roles.
• Hands-on experience fine-tuning LLMs and designing data pipelines for SFT/RLHF/DPO.
• Deep familiarity with LLM evaluation methodologies and benchmarks.
• Strong Python skills and modern LLM toolchain proficiency (Hugging Face, PyTorch).
• Excellent technical communication; ability to brief researchers and executives.
• Bachelor’s or advanced degree in CS/ML/linguistics or equivalent.
• Willingness to own engagements from discovery to delivery.

Responsibilities

• Lead technical discovery with customers to understand objectives, gaps, and constraints.
• Design end-to-end post-training solutions: SFT data, RLHF/DPO data, and benchmarks.
• Architect engagements that combine Innodata platforms with global SME workforce.
• Author proposals, SOWs, diagrams, and pricing in partnership with sales.
• Run workshops, POCs, and pilot designs to de-risk programs.
• Serve as ongoing technical advisor during delivery with cross-functional teams.
• Feed customer signals back into R&D and product roadmap.

Skills

Applied ML

Fine-tuning LLMs

LLM evaluation

Python

Hugging Face

Technical communication

Stakeholder engagement

Education

Bachelor’s or higher in CS/ML/Computational linguistics

Tools

Hugging Face

PyTorch

vLLM

lm-evaluation-harness

lighteval

Job description

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope of the Role

Innodata partners with leading foundation model labs, hyperscalers, and enterprise AI teams to build the data, evaluation, and post-training systems that make modern LLMs trustworthy and production‑ready.

As a Technical Solutions Architect for Evals & Fine‑Tuning, you are the technical face of Innodata to our most demanding customers. You sit at the intersection of client AI/ML teams, our research scientists and ML engineers, our subject‑matter expert workforce, and our platform teams. You translate ambiguous customer goals – "improve factuality on long‑context legal QA," "build a safety eval suite for our next model release," "design a DPO pipeline for our coding assistant" – into concrete, scoped, deliverable engagements.

This is a senior individual‑contributor role for someone who has done the work: built fine‑tuning pipelines, designed eval harnesses, argued with stakeholders about benchmark validity, and earned credibility with sophisticated ML buyers.

What You’ll Own

• Lead technical discovery with prospective and existing customers – foundation model labs, frontier AI teams, and large enterprises – to understand model objectives, gaps, and constraints.
• Design end‑to‑end solutions across the post‑training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM‑as‑judge pipelines, human‑in‑the‑loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long‑context).
• Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains.
• Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
• Run technical workshops, POCs, and pilot designs that de‑risk larger programs and prove value quickly.
• Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent.
• Feed customer signal back into Innodata’s R&D and product roadmap – what benchmarks customers actually want, where eval methodology is breaking, what new fine‑tuning paradigms are gaining traction.
• Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long‑context and tool‑use evaluation) and post‑training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation).
• Represent Innodata externally – at customer reviews, conferences, and in technical content.

You’ll Thrive in This Role If You Have

• 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post‑training.
• Hands‑on experience fine‑tuning LLMs (SFT at minimum; preference optimization methods such as RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
• Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM‑as‑judge design and its failure modes, inter‑annotator agreement, and human eval workflow design.
• Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm‑evaluation‑harness, lighteval, or equivalents).
• Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non‑technical executive on the same engagement.
• A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation.
• Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field – or equivalent demonstrated experience.

The expected salary range for this position is $140,000 - $160,000 USD per year, based on experience, skills, and qualifications.