Principal Data Engineer RWE RWD

Veramed; India; On-site; INR 700,000 - 1,100,000; Full time

Job summary

Veramed is seeking a Data Engineer to develop and maintain data processes for automated generation of patient-level data used in dashboards, reports, and studies. You will transform raw datasets into analytics-ready structures, including OMOP conversions where feasible, and ensure data quality through validation and testing.

Collaboration with epidemiologists and statisticians is essential. The role emphasizes documenting data flows and pipelines, supporting study outputs, and working with IT

Qualifications

• Experience designing data processes to generate patient-level data.
• Ability to transform raw datasets into usable data structures for studies and dashboards.
• Familiarity with OMOP format conversions where applicable.
• Experience in data validation, testing and troubleshooting ETL pipelines.

Responsibilities

• Develop data processes for automated generation of patient-level data used in dashboards, reports, and studies.
• Manipulate downstream datasets after onboarding from data vendors/partners into usable structures for RWEs and outputs.
• Transform raw datasets into analytics-ready formats; convert bespoke datasets to OMOP where feasible.
• Engage with epidemiologists, statisticians, and health economists to document end-user requirements and translate into data structures.
• Maintain documentation of data flows, schemas, pipelines, and processes for onboarding and auditing.
• Collaborate with Data Engineering team members and IT for data quality, loading, extraction, and transformation issues; support study outputs.

Skills

Data processes

Data structures

ETL

OMOP concepts

Testing & validation

Documentation

Stakeholder communication

Databricks

Tools

Databricks

Job description

• Development of data processes for the automated ongoing generation of patient level data (the “data product”) to be used by various business stakeholders for a variety of purposes (e.g. dashboards, reports, studies).
• Downstream manipulation of datasets after their onboarding from data vendors/partners (an activity predominantly carried out by the vendors and Client IT department) from “raw” format as provided into useable data structures that will be used to carry out RWE studies, dashboards & other data outputs
• Transformation of raw datasets into usable datasets
• Occasional conversion of bespoke / one-off datasets (e.g. biomarkers, mutations) to OMOP format (including an understanding of what can and cannot be converted to OMOP format, e.g. to allow analysis to be carried out on residual data that cannot be converted to OMOP).

Communication

• Technical engagement with key stakeholders (e.g. epidemiologist, statisticians, market access/health economists) from outside the RWE programming team to ensure a full and detailed understanding of end-user requirement is created and carefully documented. This includes scoping discussions, business analysis and translation of verbalised end-user needs into actionable data structures
• Detailed technical engagement with colleagues from within the RWE programming team to build data structures required for the generation of RWE study outputs and data products; also support those team members in creating the study outputs where the data engineer’s skillset can add incremental value
• Liaison & ongoing interaction with IT department to ensure that raw datasets inbound from data partners are fit for the agreed purposes (as per bullet point 1 above)
• Liaise, where required, with technical staff employed by analysis software vendors (Databricks etc)

Documentation

• Maintain clear documentation of data flows, schemas, pipelines, and processes to facilitate onboarding, troubleshooting and auditing.

Quality, Validation & Support

• Design and carry out detailed testing (data validation and monitoring) approaches for data structures built by self or other members of team to ensure the accuracy and reliability of the data within the data product
• Troubleshoot any issues encountered with data loading, extraction and transformation (ETL)
• Work in collaboration with three other members of the Data Engineering team, taking on workload from others as and when required