Clinical Data Engineer - CDISC/SDTM

QAAgility Technologies; Bengaluru; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

QAAgility Technologies in Bengaluru seeks a highly skilled clinical Data Engineer to design, build and maintain production ETL pipelines in Databricks / Delta Lake, ingesting RWD and transforming into CDISC SDTM and OMOP models.

You will work with clinical SMEs, data scientists and infrastructure teams, extending Medallion architecture, integrating AI components, validating data, and ensuring governance and reproducibility across the data lifecycle.

Responsibilities

• Design, build and maintain production ETL pipelines in Databricks / Delta Lake.
• Implement harmonisation to map RWD to OMOP and CDISC SDTM.
• Extend medallion architecture with validation, lineage and performance tuning.
• Develop configurable, input-driven transformation frameworks via config files.
• Integrate AI components with human-in-the-loop review and scoring.
• Establish testing, CI/CD, monitoring and alerting for ETL jobs.
• Collaborate with clinical data scientists and data stewards to define requirements.

Job description

Role Summary

We are seeking a highly skilled clinical Data Engineer. You will combine clinical data expertise with strong data engineering and technical skills to generate well documented pipelines from source to curated data sets in common data models like CDISC SDTM.

You will collaborate closely with clinical SMEs, data scientists, infrastructure, and other skilled data engineers. We are looking to expand this functionality to include Real World Data (from a broad range of registries). You will help extend our medallion Databricks pipelines (CDISC SDTM) to incorporate Real - World Data (RWD) from registries and other sources, working with clinical experts and AI teams to combine rule - based and automated mapping approaches (including OMOP interoperability).

Key Responsibilities

• Design, build and maintain production ETL pipelines in Databricks / Delta Lake to ingest RWD (registries, claims, EHR extracts) and transform into standard models.
• Implement harmonisation workflows to map incoming RWD to OMOP and to the internal CDISC SDTM canonical model; handle vocabulary mapping, units normalization and provenance.
• Extend the medallion architecture (bronze / silver / gold) patterns with robust validation, lineage, partitioning and performance tuning.
• Develop configurable, input - driven transformation frameworks so clinical experts can drive mapping rules via config files and catalogs.
• Integrate AI / automation components (e.g., model - assisted mapping, NLP for free text) with human - in - the - loop review and confidence scoring.
• Establish testing, CI / CD, monitoring and alerting for ETL jobs and automations; ensure reproducibility, versioning and governance.
• Collaborate with clinical data scientists, data stewards and stakeholders to define requirements, data contracts and success metrics.

Collaborate With

• Person has to collaborate with scientific team, balance data structure, support data team, and process data pipeline efficiently.

(ref:hirist.tech)