Lead NLP Data Scientist, Production-Scale AI

Wiley; United States; Remote; USD 140,000 - 201,000; Full time

Job summary

Wiley is seeking a Principal Data Scientist to own domain-specific content modeling end to end, from eval sets through shipping pipelines. You will design NLP enrichment pipelines, compare traditional NLP with LLM-based approaches, and own evaluations and metrics in production for high-volume scholarly text.

You will collaborate with editors, product managers, and engineers, writing scalable Python and building robust data pipelines using Airflow/Dagster to support agentic AI applications while

Qualifications

• Deep Python expertise with production-grade, scalable code.
• Strong NLP background across modern (LLMs, transformers, embeddings, retrieval) and classical approaches.

Responsibilities

• Design and build NLP enrichment pipelines that extract entities, classifications, claims, and summaries from scientific full-text at scale.
• Compare NLP approaches to extraction and enrichment against LLM-based approaches, and pick the right tool for each task with evaluation, cost, and operational tradeoffs.
• Own evaluation. Build golden sets with SMEs and vendors, choose metrics, and balance speed, quality, and cost.
• Write production-quality Python. Manage concurrency and cost for high-volume LLM workloads; craft extendable code.
• Collaborate with data engineers to orchestrate work in data pipelines and tools like Airflow and Dagster; design idempotent, retryable stages.
• Contribute to agentic AI applications: tool use, multi-step reasoning, guardrails, evaluation of trajectories.
• Work with editors, product managers, and engineers; translate stakeholder input into modeling work.

Skills

Deep Python

LLMs

Transformers

Embeddings

Retrieval

NER

Classification

Sequence labeling

LLMs in production

Agentic AI concepts

Scientific text

Tools

AWS S3

AWS Batch

AWS Lambda

AWS SageMaker

Parquet

Iceberg

Job description

Wiley is seeking a Principal Data Scientist to own domain-specific content modeling end to end, from eval sets through shipping pipelines. You will design NLP enrichment pipelines, compare traditional NLP with LLM-based approaches, and own evaluations and metrics in production for high-volume scholarly text.

You will collaborate with editors, product managers, and engineers, writing scalable Python and building robust data pipelines using Airflow/Dagster to support agentic AI applications while