Senior Azure Databricks Data Engineer (Onsite - Cary, NC)

Tech Mirrors; Cary (NC); On-site; USD 140,000 - 180,000; Full time

Job summary

Tech Mirrors in Cary, NC seeks an experienced Azure Data Engineer to design and build batch and streaming data pipelines using Azure, Databricks, and Kafka. You will develop ETL/ELT solutions with Spark, SQL and Python/PySpark, and implement Data Lake/Lakehouse architectures while migrating data from legacy systems.

Ideal candidates have 8+ years of data engineering on Azure, strong SQL/Python skills, and hands-on experience with batch and real-time ingestion.

Qualifications

• 8+ years of experience in data engineering in Azure using Azure Databricks.
• Strong experience with Azure Data Lake, Databricks, ADF, Postgres.
• Hands-on experience with SQL, Python and PySpark or Scala.
• Proven experience with batch & real-time ingestion.

Responsibilities

• Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
• Develop ETL/ELT solutions using Spark, SQL and Python/PySpark, or Scala.
• Implement and support Data Lake / Lakehouse architectures and migrate data to Azure Cloud.
• Collaborate with cross-functional teams and stakeholders.

Skills

SQL

Python

PySpark

Scala

Kafka

Education

Bachelor's degree in Computer Science or related

Tools

Microsoft Azure (ADB/ADF)

Azure Databricks

Azure Data Lake

Postgres

Kafka

Job description

Tech Mirrors in Cary, NC seeks an experienced Azure Data Engineer to design and build batch and streaming data pipelines using Azure, Databricks, and Kafka. You will develop ETL/ELT solutions with Spark, SQL and Python/PySpark, and implement Data Lake/Lakehouse architectures while migrating data from legacy systems.

Ideal candidates have 8+ years of data engineering on Azure, strong SQL/Python skills, and hands-on experience with batch and real-time ingestion.