Principal Data Engineer — Hybrid Azure & Databricks for Healthcare AI

Persistent Systems; Dublin; Hybrid; EUR 110,000 - 160,000; Full time

Benefits offered by this job

Competitive salary

Hybrid work model

Career growth & certifications

Long service awards

Insurance coverage

Job summary

Persistent Systems is seeking a Principal Data Engineer to design, build, and evolve data solutions for U.S. healthcare.

This hybrid, permanent role sits at the intersection of engineering, analytics, and business strategy, delivering scalable data products and AI-enabled capabilities. You will mentor engineers, drive cloud-based data pipelines, and partner with stakeholders to translate complex needs into practical technical strategies, including a migration to enterprise-grade Databricks

Qualifications

• Bachelor's degree in Computer Science or related field or equivalent experience.
• 8+ years of hands-on experience delivering complex data products or platforms.
• 8+ years designing scalable data solutions with pipeline infra and analytics.
• Experience with high-volume batch data pipelines in Azure environment.
• 5+ years with Databricks and Apache Spark.
• 2+ years with DBT.
• Strong SQL and programming (Python or Scala).
• Knowledge of data modelling, data quality, lineage and observability.
• Experience with data pipeline orchestration tools (Airflow or Databricks).
• Security, reliability, performance, cost, and maintainability considerations in design decisions.

Responsibilities

• Design and build scalable data products and platform components using modern cloud, lakehouse, and software practices.
• Partner with product, business, architecture, security, governance, and engineering teams to translate needs into technical solutions.
• Lead large-scale migration from Databricks clusters to enterprise-managed Databricks clusters.
• Provide technical mentorship and design guidance to junior engineers.
• Contribute to data capability enhancements without direct people-management.

Skills

Azure

Databricks

Apache Spark

SQL

Python

Scala

Data modeling

Airflow

CI/CD

Cloud security

Education

Bachelor's degree in CS or related

Tools

Snowflake

Terraform

GitHub Actions

Job description

Persistent Systems is seeking a Principal Data Engineer to design, build, and evolve data solutions for U.S. healthcare.

This hybrid, permanent role sits at the intersection of engineering, analytics, and business strategy, delivering scalable data products and AI-enabled capabilities. You will mentor engineers, drive cloud-based data pipelines, and partner with stakeholders to translate complex needs into practical technical strategies, including a migration to enterprise-grade Databricks