Finance AI Evaluation & Datasets Scientist

Innodata Inc.; United States; On-site; USD 150,000 - 175,000; Full time

Job summary

Innodata Inc. in the United States seeks an Applied Data Scientist specializing in Financial AI Evaluation & Datasets. You’ll own the design, measurement quality, and domain validity of datasets used to train and evaluate financial-domain LLMs and multimodal models.

Focus on unstructured data, PDFs, statements, and transcripts to ensure compliance, auditable quality, and robust evaluation. You will collaborate with a cross-functional team to define rubrics, sampling strategies, and annotation

Qualifications

• 5 years of data science experience, with at least 2 years in financial services or regulated environments.
• Hands-on experience with unstructured and multimodal financial data (PDFs, transcripts, tables).
• Familiarity with financial standards such as GAAP/IFRS and privacy regimes (GLBA, MNPI).
• Experience designing ML datasets, annotation guidelines, and evaluation rubrics.

Responsibilities

• Translate customer goals into dataset specs, taxonomies, rubrics, and acceptance criteria.
• Design training and evaluation datasets across financial AI surfaces (QA, filings, earnings analysis).
• Foreground unstructured and multimodal data (PDFs, statements, charts) in dataset design.
• Develop evaluation methodologies beyond surface accuracy, including numerical consistency and fairness across segments.
• Instrument datasets into training/evaluation pipelines with rubric-grounded prompts and monitoring.

Skills

Python

SQL

Pandas

Data science

Education

Degree in statistics/data science/economics

Tools

Hugging Face

PyTorch

model APIs

Job description

Innodata Inc. in the United States seeks an Applied Data Scientist specializing in Financial AI Evaluation & Datasets. You’ll own the design, measurement quality, and domain validity of datasets used to train and evaluate financial-domain LLMs and multimodal models.

Focus on unstructured data, PDFs, statements, and transcripts to ensure compliance, auditable quality, and robust evaluation. You will collaborate with a cross-functional team to define rubrics, sampling strategies, and annotation