Data Engineer with ETL/ELT pipelines and API integrations

Dhanu Global; Indianapolis (IN); On-site; USD 120,000 - 170,000; Full time

Job summary

Dhanu Global is seeking a Data Engineer to design, build, and maintain production ETL/ELT pipelines and API integrations onsite in Indianapolis, IN. The role focuses on delivering a governed Azure-based data platform that ingests structured and unstructured data from laboratory, manufacturing, and partner systems, enabling analytics, AI, and regulatory reporting.

You will work across AI document ingestion and structured data tracks, ensuring data quality, lineage, and compliance in a regulated

Qualifications

• 8+ years of hands-on data engineering experience.
• Experience building production ETL/ELT pipelines and API integrations.
• Expert-level Python and/or PySpark for data ingestion, transformation, orchestration, testing, and CI/CD.
• Experience with Microsoft Azure Fabric (Lakehouse, Data Factory, Fabric Pipelines, Delta Lake) delivering end-to-end production solutions.
• Hands-on AWS experience, including S3, Glue (or equivalent), and RDS/Aurora.
• Strong SQL and PostgreSQL experience, including normalized schema design, query optimization, indexing, and performance tuning.
• Experience with data modeling, medallion architecture, and Lakehouse design patterns.
• Experience implementing data lineage, quality controls, schema validation, error handling, and monitoring in regulated environments.
• Knowledge of GxP, GMP, GCP within pharmaceutical, biotechnology, or medical device environments
• SCM, WMS, PLM, MM, QA, any validated applications with in a FDA Regulated systems

Responsibilities

• Design, build, and maintain production ETL/ELT pipelines integrating laboratory and operational systems (e.g., Darwin, Teamcenter/PLM, LabVantage LIMS, Jama, TurboAC, and Qdocs/Veeva) into Microsoft AzureFabric Lakehouse and PostgreSQL.
• Develop and optimize Bronze, Silver, and Gold medallion architecture, including schema mapping, data modeling, referential integrity, and performance optimization.
• Build scalable API integrations and cross-cloud data pipelines across Azure and AWS to support enterprise data integration.
• Implement automated data quality controls, controlled vocabulary normalization, schema validation, Q-gate/specification checks, data lineage, and audit trails to ensure GxP compliance.
• Monitor, troubleshoot, and optimize pipeline performance, reliability, error handling, and operational monitoring in production environments.
• Collaborate with business, engineering, and IT teams to integrate data sources and establish a governed, scalable digital thread supporting analytics, AI, and regulatory reporting.

Skills

Data engineering

ETL/ELT pipelines

API integrations

Python

PySpark

Azure Fabric

AWS

S3

Glue

RDS/Aurora

SQL

PostgreSQL

Data modeling

Medallion architecture

Lakehouse design

Data lineage

Quality controls

Schema validation

Error handling

Monitoring

GxP

GMP

GCP

FDA Regulated systems

SCM

WMS

PLM

MM

QA

Job description

Position Title: Data Engineer with ETL/ELT pipelines and API integrations Location: Indianapolis,IN (onsite 5 days per week) Duration: T hrough December 2026, possible extension into 2027

Job Description Pharmaceutical's client is building a governed data and AI platform, integrating device, laboratory, partner, and document data into a unified foundation that supportsregulatoryreporting, advanced analytics, and AI-driven scientificinsights.The Data Engineer is a hands-on builder responsible for developing data pipelines, API integrations, and AI infrastructure that bring structured and unstructured data into a governedAzure- basedarchitecture. This delivery-focused rolerequires designing, coding, testing, and maintaining production-ready solutions across both AI document ingestion and structured ETL/ELT data engineering tracks. This role integrates structured and unstructureddata fromlaboratory systems, product lifecycleapplications, and manufacturing partners into a governed Azure-based data platform, creating a scalable, audit-ready digital thread that supports analytics, AI, and regulatory compliance.

Key Responsibilities:

• Design, build, and maintain production ETL/ELT pipelines integrating laboratory and operational systems (e.g., Darwin, Teamcenter/PLM, LabVantage LIMS, Jama, TurboAC, and Qdocs/Veeva) into Microsoft AzureFabric Lakehouse and PostgreSQL.
• Develop and optimize Bronze, Silver, and Gold medallion architecture, including schema mapping, data modeling, referential integrity, and performance optimization.
• Build scalable API integrations and cross-cloud data pipelines across Azure and AWS to support enterprise data integration.
• Implement automated data quality controls, controlled vocabulary normalization, schema validation, Q-gate/specification checks, data lineage, and audit trails to ensure GxP compliance.
• Monitor, troubleshoot, and optimize pipeline performance, reliability, error handling, and operational monitoring in production environments.
• Collaborate with business, engineering, and IT teams to integrate data sources and establish a governed, scalable digital thread supporting analytics, AI, and regulatory reporting.

Required Skills:

• 8+ years of hands-on Data Engineering experience building production ETL/ELT pipelines and API integrations.
• Expert-level Python and/or PySpark for data ingestion, transformation, orchestration, testing, and CI/CD.
• Experience with Microsoft Azure Fabric (Lakehouse, Data Factory, Fabric Pipelines, Delta Lake) delivering end-to-end production solutions.
• Hands-on AWS experience, including S3, Glue (or equivalent), and RDS/Aurora.
• Strong SQL and PostgreSQL experience, including normalized schema design, query optimization, indexing, and performance tuning.
• Experience with data modeling, medallion architecture, and Lakehouse design patterns.
• Experience implementing data lineage, quality controls, schema validation, error handling, and monitoring in regulated environments.
• Knowledge of GxP, GMP, GCP within pharmaceutical, biotechnology, or medical device environments
• SCM, WMS, PLM, MM, QA, any validated applications with in a FDA Regulated systems