Senior Azure Data Engineer - Databricks & Synapse

CloudBigD

Pennsylvania

On-site

USD 91,000 - 124,000

Full time

39 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

CloudBigD in Malvern, PA seeks an experienced Azure Data Engineer for a 12-month engagement to design and optimize large-scale data pipelines. The role includes migrating pipelines to Databricks, building real-time and batch ETL with PySpark, and ensuring data quality and governance.

You will collaborate with global teams, mentor juniors, and implement scalable cloud architectures using Azure data services. On-site position with opportunities to extend the contract.

Qualifications

  • 7+ years in data engineering, 3+ in a lead/senior role.
  • Strong expertise in Azure Databricks, Synapse, PySpark.
  • Proven experience with real-time streaming (Event Hubs, Kafka).
  • Deep knowledge of Azure ecosystem (Data Lake, Data Factory, Key Vault).
  • Strong SQL, performance tuning, and CI/CD pipeline skills.
  • Excellent leadership and communication skills.

Responsibilities

  • Collect, store, process and analyze large datasets to build ETL processes.
  • Lead migration of data pipelines from Synapse to Databricks.
  • Architect and implement batch + real-time data pipelines using PySpark, Databricks, and Event Hubs.
  • Establish data engineering best practices and mentor junior engineers.
  • Drive CI/CD automation with AzDO and Git.
  • Ensure data quality, governance, and compliance (Collibra).
  • Collaborate with business/tech teams; leverage insurance domain knowledge where applicable.
  • Develop reusable frameworks to reduce development effort and costs.
  • Develop quality code with performance optimizations at development stage.
  • Learn new technologies and work on cutting-edge cloud technologies.

Skills

Azure Databricks
Synapse
PySpark
SQL
CI/CD
Leadership
Communication
Event Hubs
Kafka
Delta Lake
Hive
Spark
Python
Scala

Tools

Azure Data Factory
Cosmos DB
ADLS
Data Bricks
Delta Lake
Hive
HDFS
Sqoop
Kafka
Kerberos
Impala

Job description

CloudBigD in Malvern, PA seeks an experienced Azure Data Engineer for a 12-month engagement to design and optimize large-scale data pipelines. The role includes migrating pipelines to Databricks, building real-time and batch ETL with PySpark, and ensuring data quality and governance.

You will collaborate with global teams, mentor juniors, and implement scalable cloud architectures using Azure data services. On-site position with opportunities to extend the contract.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Engineer Lead - Databricks, Synapse, PySpark
Azure Data Engineer Lead - Databricks, Synapse, PySpark

Ethereum Technologies LLC • Pennsylvania

On-site
USD 89,000 - 126,000
Senior Azure Data Engineer | Databricks & Synapse Lead
Senior Azure Data Engineer | Databricks & Synapse Lead

Valiant Technologies LLC • Pennsylvania

On-site
USD 89,000 - 126,000
Azure Data Engineer Lead | Databricks & PySpark
Azure Data Engineer Lead | Databricks & PySpark

Robotics Technologies • Malvern

On-site
USD 89,000 - 126,000
Azure Data Engineer — Lead Data Pipelines & Databricks
Azure Data Engineer — Lead Data Pipelines & Databricks

Quantum Technologies LLC • Pennsylvania

On-site
USD 91,000 - 124,000
Databricks Data Engineer - Azure, CDC Pipelines, PySpark
Databricks Data Engineer - Azure, CDC Pipelines, PySpark

CloudBigD • Phoenix (AZ)

On-site
USD 81,000 - 115,000
Lead Azure Data Engineer - Databricks & ETL Specialist
Lead Azure Data Engineer - Databricks & ETL Specialist

Robotic Process Automation LLC • Pennsylvania

On-site
USD 89,000 - 126,000
Azure Databricks Engineer
Azure Databricks Engineer

IQuest Solutions Corporation • Newtown Square

On-site
USD 120,000 - 170,000
Azure Data Engineer
Azure Data Engineer

Signature IT World • Pennsylvania

Hybrid
USD 120,000 - 150,000
Azure Data Engineer
Azure Data Engineer

Signature IT World • New Jersey

Hybrid
USD 110,000 - 150,000
Senior Databricks Engineer — Cloud Data Platform Lead
Senior Databricks Engineer — Cloud Data Platform Lead

Ethereum Technologies LLC • Pennsylvania

On-site
USD 112,000 - 128,000