Senior Data Engineer – AI-Grade Data Platforms (Remote)

NCBioImpact; Durham (NC); Hybrid; USD 150,000 - 190,000; Full time

Job summary

GSK is seeking a Senior Data Engineer to design production data systems within the Enterprise AI (EAI) team. You will define architectural standards, governance, and AI-grade data infrastructure, mentoring junior engineers and leading data lifecycle efforts for AI systems.

You will lead cross-functional work on data pipelines, orchestrations, and observability, ensuring high-quality data and scalable infrastructure across domains. Hybrid US-based role with on-site collaboration.

Qualifications

• 8+ years of data engineering experience, including significant time leading the design and operation of complex production data systems.
• Expertise with modern warehouses or lakehouses, orchestration and transformation frameworks, and batch and streaming architectures, such as Databricks, BigQuery, Airflow, dbt or Spark.
• Experience leading the design of AI-grade data infrastructure for production AI systems, including evaluation datasets, retrieval architectures and real-time data flows.
• Experience leading data lifecycle management for AI systems, including versioning, drift monitoring and governance of AI artefacts.
• A track record of defining data quality and governance standards for a team or domain that others adopted, with measurable outcomes.
• Experience leading cloud data infrastructure design with infrastructure as code.
• Experience designing and operating AI agents or substantial LLM-powered tools inside data engineering workflows, including tool design, evaluation, observability and human-in-the-loop controls, and AI-assisted development patterns that a team adopted.

Responsibilities

• Lead the design of production pipelines that serve multiple downstream consumers, and define the architectural standard the team and its digital workers apply.
• Define the team's patterns for agent-generated work within the operating model: what digital workers build and remediate on their own in the team's domains, which checks their work must pass, and when it needs human review.
• Design how digital workers integrate into the team's orchestration, observability and incident workflows, with clear tool design, evaluations, guardrails and human-in-the-loop controls.
• Lead the response to significant data quality incidents, from detection to permanent resolution, and drive improvements to team-wide pipeline quality.
• Set the context standards for the team's domains and contribute to the governed context layer across EAI.
• Define data quality and governance standards for the team's domains and drive adoption across the organization.
• Lead data lifecycle management for AI systems, including versioning and drift monitoring, in partnership with ML engineers.

Skills

Computer Programming

Data Analysis

Data Architecture Development

Databricks

Data Engineering

Data Governance

Data Ingestion

Data Integration

Data Migrations

Data Modeling

Data Transformation

ETL Development

Performance Optimization

Performance Tuning

SQL

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

BigQuery

Airflow

dbt

Spark

Job description

GSK is seeking a Senior Data Engineer to design production data systems within the Enterprise AI (EAI) team. You will define architectural standards, governance, and AI-grade data infrastructure, mentoring junior engineers and leading data lifecycle efforts for AI systems.

You will lead cross-functional work on data pipelines, orchestrations, and observability, ensuring high-quality data and scalable infrastructure across domains. Hybrid US-based role with on-site collaboration.