Technical Solutions Architect, Evals & Fine-Tuning

Innodata Inc.; United States; On-site; USD 140,000 - 160,000; Full time

Job summary

Innodata in the United States is seeking a Senior Technical Solutions Architect for Evals & Fine-Tuning to be the technical face to demanding customers. You will translate objectives into scoped engagements and lead technical discovery with foundation model teams and enterprise AI groups.

The role requires hands-on experience in fine-tuning LLMs, designing eval harnesses, and credible communication with researchers and executives.

Qualifications

• 7+ years in applied ML, ML engineering, ML research, or technical solutions.
• 2+ years focused on LLM evaluation and/or post-training.
• Hands-on fine-tuning of LLMs (SFT; RLHF/DPO preferred).
• Deep familiarity with LLM evaluation, benchmarks, and human eval workflows.
• Strong Python and modern LLM toolchain knowledge.
• Excellent technical communication and stakeholder management.
• Bachelor’s or advanced degree in CS/ML or equivalent experience.

Responsibilities

• Lead technical discovery with foundation model labs and large enterprises to understand model objectives and constraints.
• Design end-to-end post-training solutions: SFT data curation, RLHF/DPO data, golden datasets, benchmarks, LLM‑as‑judge pipelines, and evaluation harnesses.
• Architect engagements combining GenAI platforms with a global SME workforce across 85+ languages and domains.
• Author proposals, SOWs, diagrams, and pricing in collaboration with sales, delivery, and finance.
• Run workshops, POCs, and pilot designs to de-risk larger programs and prove value.
• Act as ongoing technical advisor during delivery with researchers, ML engineers, and program managers.
• Feed customer signal back into R&D and product roadmap; monitor eval methodologies.
• Stay current on state‑of‑the‑art evals and post-training approaches.

Skills

LLM evaluation

Python

ML engineering

Technical communication

Consultative mindset

SFT data curation

RLHF/DPO

Education

Bachelor’s or advanced degree in computer science / ML / computational linguistics

Tools

Hugging Face

PyTorch

vLLM

lm-evaluation-harness

Job description

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope Of The Role

Innodata partners with leading foundation model labs, hyperscalers, and enterprise AI teams to build the data, evaluation, and post‑training systems that make modern LLMs trustworthy and production‑ready.

As a Technical Solutions Architect for Evals & Fine‑Tuning, you are the technical face of Innodata to our most demanding customers. You sit at the intersection of client AI/ML teams, our research scientists and ML engineers, our subject‑matter expert workforce, and our platform teams. You translate ambiguous customer goals — “improve factuality on long‑context legal QA,” “build a safety eval suite for our next model release,” “design a DPO pipeline for our coding assistant” — into concrete, scoped, deliverable engagements.

This is a senior individual‑contributor role for someone who has done the work: built fine‑tuning pipelines, designed eval harnesses, argued with stakeholders about benchmark validity, and earned credibility with sophisticated ML buyers.

What You’ll Own

• Lead technical discovery with prospective and existing customers — foundation model labs, frontier AI teams, and large enterprises — to understand model objectives, gaps, and constraints.
• Design end‑to‑end solutions across the post‑training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM‑as‑judge pipelines, human‑in‑the‑loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long‑context).
• Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains.
• Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
• Run technical workshops, POCs, and pilot designs that de‑risk larger programs and prove value quickly.
• Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent.
• Feed customer signal back into Innodata’s R&D and product roadmap — what benchmarks customers actually want, where eval methodology is breaking, what new fine‑tuning paradigms are gaining traction.
• Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long‑context and tool‑use evaluation) and post‑training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation).
• Represent Innodata externally — at customer reviews, conferences, and in technical content.

You’ll Thrive In This Role If You Have

• 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post‑training.
• Hands‑on experience fine‑tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
• Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM‑as‑judge design and its failure modes, inter‑annotator agreement, and human eval workflow design.
• Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm‑evaluation‑harness, lighteval, or equivalents).
• Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non‑technical executive on the same engagement.
• A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation.
• Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field — or equivalent demonstrated experience.

The expected salary range for this position is $140,000 – $160,000 USD per year, based on experience, skills, and qualifications.

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at https://consumer.ftc.gov/articles/job-scams.

If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at [e-mail] and consider reporting it to the FTC at ReportFraud.ftc.gov.