Technical Lead-Data Engg

Birlasoft; Pune District; On-site; INR 2,800,000 - 4,200,000; Full time

Job summary

Birlasoft seeks a Technical Lead-Data Engg in Pune, India, to architect and deliver scalable data pipelines across clinical, regulatory, safety, and commercial domains. You will leverage Databricks, Spark, PySpark, Python, and SQL to move data through Bronze-Silver-Gold layers, build data products, and enable enterprise data harmonization.

You will design modern data models, ensure data quality, and implement reusable processing utilities while aligning with GxP, 21 CFR Part 11, and ALCOA+

Qualifications

• 10–15 years in Data Engineering and enterprise data platforms.
• Strong hands-on Databricks, Apache Spark, PySpark, Python, and SQL.
• 4+ years with Databricks/Spark-based data engineering preferred.
• Experience in data migration, ETL/ELT development, and data modelling.
• Cloud platforms AWS/Azure/GCP; CI/CD, Git, Jenkins, and DevOps practices.
• Experience in Life Sciences/Pharma environments preferred.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Databricks.
• Develop complex transformation logic across Bronze, Silver, and Gold layers.
• Implement reusable transformation frameworks and common data processing utilities.
• Optimize transformation workloads for performance, scalability, and cost.
• Apply data quality, validation, and business-rule checks throughout the pipeline.
• Design scalable data models using Dimensional modelling, Star/Snowflake schemas, Data Vault 2.0, and canonical models.
• Develop curated data products and datasets using Databricks SQL and Delta Lake.
• Support enterprise-wide data harmonization across clinical, regulatory, safety, and commercial domains.
• Ensure adherence to regulatory and data governance requirements including 21 CFR Part 11, HIPAA/GDPR, and ALCOA+.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

CI/CD

Git

Jenkins

DevOps

Data modelling

Data quality

Data migration

Modernization

Tools

Delta Lake

Unity Catalog

Databricks Workflows

Job description

Select how often (in days) to receive an alert:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Technical Lead-Data Engg

Description:

Area(s) of responsibility

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Databricks.
• Develop complex transformation logic across Bronze, Silver, and Gold layers.
• Implement reusable transformation frameworks and common data processing utilities.
• Optimize transformation workloads for performance, scalability, and cost.
• Apply data quality, validation, and business-rule checks throughout the pipeline.

Enterprise Data Modelling

• Design scalable data models using:
  • Dimensional modelling
  • Star/Snowflake schemas
  • Data Vault 2.0
  • Canonical data models
• Develop curated data products and analytical datasets using Databricks SQL and Delta Lake.
• Support enterprise-wide data harmonization across clinical, regulatory, safety, and commercial domains.

Experience supporting data platforms across:

• Clinical trials – EDC, CDMS, CTMS
• Regulatory and submission systems
• Pharmacovigilance and safety
• Commercial analytics
• Real-World Evidence (RWE)

Ensure adherence to relevant regulatory and data governance requirements including GxP, 21 CFR Part 11, HIPAA/GDPR, and ALCOA+ principles.

Required Qualifications

• 10–15 years of experience in Data Engineering and enterprise data platforms.
• Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.
• 4+ years of experience with Databricks/Spark-based data engineering preferred.
• Strong experience in:
  • Data migration and modernization
  • ETL/ELT development
  • Batch and streaming data pipelines
  • Delta Lake
  • Databricks Workflows
  • Unity Catalog
  • Data modelling
  • Data quality and reconciliation
• Experience with cloud platforms – AWS/Azure/GCP.
• Strong understanding of CI/CD, Git, Jenkins, and DevOps practices.
• Experience working in Life Sciences/Pharma environments preferred.