Senior Data Engineer — Azure & Databricks Pipelines

Sansar Tec, LLC; Haddon Heights (NJ); On-site; USD 110,000 - 140,000; Full time

Job summary

A leading data solutions company is seeking an experienced Senior Data Engineer specializing in Azure and Databricks to drive data engineering initiatives. The ideal candidate will create scalable data pipelines and support the Enterprise Data Lake and Data Warehouse. Responsibilities include implementing complex ETL processes, orchestrating data flows, and providing production support. Candidates should have at least 5 years of Azure-focused data engineering experience, with strong skills in SQL, Python, and PySpark.

Qualifications

• 5+ years of experience in data engineering with Azure-focused solutions.
• Proven track record in building and orchestrating data pipelines in a cloud environment.
• Strong skills in SQL, Python, and PySpark for data transformations.

Responsibilities

• Build new data pipelines and enhance existing ones using Databricks and Azure Data Factory.
• Implement complex data transformations according to business requirements.
• Utilize Azure Data Factory to orchestrate data flows and automate processes.
• Conduct unit testing and manage User Acceptance Testing (UAT).
• Develop and maintain design documents and runbooks.
• Configure and optimize compute resources.
• Identify and implement performance improvements for data pipelines.
• Provide ongoing support for production environments.

Skills

Azure Data Factory

Databricks

PySpark

SQL

Python

Education

Bachelor's or Master's degree in Computer Science or related field

Job description

A leading data solutions company is seeking an experienced Senior Data Engineer specializing in Azure and Databricks to drive data engineering initiatives. The ideal candidate will create scalable data pipelines and support the Enterprise Data Lake and Data Warehouse. Responsibilities include implementing complex ETL processes, orchestrating data flows, and providing production support. Candidates should have at least 5 years of Azure-focused data engineering experience, with strong skills in SQL, Python, and PySpark.