Senior Azure Data Engineer

Agilisium Technologies LLC.; Chennai; On-site; INR 800,000 - 1,200,000; Full time

Job summary

A leading technology solutions provider in Chennai is seeking an experienced Data Engineer specializing in the Life Sciences/Biomedical domain. The role involves building scalable ETL/ELT data pipelines, optimizing data flows, and collaborating with domain experts on innovative data solutions. The ideal candidate has strong expertise in PySpark, Databricks, SQL, and cloud platforms like AWS and GCP, with a solid understanding of data governance and biomedical data integration.

Qualifications

• Strong experience with PySpark, Databricks, and SQL.
• Experience with cloud platforms (AWS, GCP, Azure).
• Solid knowledge of data modelling, schemas, and governance.
• Hands-on experience with data lake/warehouse technologies (Snowflake, Redshift, Synapse).
• Exposure to workflow orchestration tools (Airflow, Prefect, dbt).
• Understanding of research informatics and healthcare compliance standards.
• Experience working with clinical, genomics, or biomedical data.

Responsibilities

• Design and build ETL/ELT pipelines for biomedical data.
• Optimize data flows using Apache Spark and Databricks.
• Collaborate with domain experts in genomics and lab science.
• Develop and maintain data models and governance practices.
• Ensure data quality with checks, lineage, and alerts.
• Deploy pipelines on AWS, GCP, or Azure.
• Contribute to data lake/warehouse solutions.

Job description

Overview

We are looking for a Data Engineer (Life Sciences/Biomedical domain) with strong expertise in PySpark, Databricks, and SQL. The role involves building scalable data pipelines, integrating complex biomedical datasets, and enabling advanced analytics for genomics, clinical trials, and research.

Accountabilities

• Design and build ETL/ELT pipelines to ingest, transform, and load data from clinical, omics, research, and operational sources.
• Optimize performance and scalability of data flows using Apache Spark, Databricks, or AWS Glue.
• Collaborate with domain experts in genomics, clinical trials, and lab science to implement robust data solutions.
• Develop and maintain data models, schemas, and governance practices for structured and unstructured biomedical data.
• Implement data quality checks, lineage, logging, and alerts to ensure reliability and reproducibility.
• Work with cloud infrastructure teams to deploy pipelines on AWS, GCP, or Azure.
• Contribute to data lake/warehouse solutions (Snowflake, Redshift, Synapse).
• Strong experience with PySpark, Databricks, and SQL.
• Experience with cloud platforms (AWS, GCP, Azure).
• Solid knowledge of data modelling, schemas, and governance.
• Hands-on experience with data lake/warehouse technologies (Snowflake, Redshift, Synapse).
• Strong problem-solving and collaboration skills.
• Experience working with clinical, genomics, or biomedical data.
• Familiarity with data quality frameworks and reproducibility practices.
• Exposure to workflow orchestration tools (Airflow, Prefect, dbt).
• Understanding of research informatics and healthcare compliance standards.

Qualifications

• Strong experience with PySpark, Databricks, and SQL.
• Experience with cloud platforms (AWS, GCP, Azure).
• Solid knowledge of data modelling, schemas, and governance.
• Hands-on experience with data lake/warehouse technologies (Snowflake, Redshift, Synapse).
• Exposure to workflow orchestration tools (Airflow, Prefect, dbt).
• Understanding of research informatics and healthcare compliance standards.
• Experience working with clinical, genomics, or biomedical data.