Azure Data Engineer: Build Scalable Pipelines & Lakehouse

CloudBigD

Raleigh (NC)

On-site

USD 110,000 - 160,000

Full time

3 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

CloudBigD is seeking an experienced Azure Data Engineer to design, build, and optimize scalable data pipelines on Microsoft Azure. You will work with Azure Data Factory, Databricks, and Synapse Analytics to deliver robust data solutions for analytics and reporting.

The role requires strong SQL and PySpark skills, experience with lakehouse architectures, and a focus on performance, security, and cost efficiency. Collaboration with stakeholders is essential for successful outcomes.

Qualifications

  • Experience designing scalable data pipelines on Azure.
  • Proficiency in PySpark and SQL for data processing.
  • Strong knowledge of data lakehouse architectures and governance.

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure data services.
  • Build and optimize ETL/ELT workflows using Azure Data Factory.
  • Develop data processing solutions using Azure Databricks and PySpark.
  • Work with Azure Synapse Analytics for data warehousing and analytics.
  • Develop complex SQL queries, stored procedures, and performance optimization strategies.
  • Implement data lake and lakehouse architectures using Azure Data Lake Storage Gen2.
  • Integrate data from multiple sources, including relational databases, APIs, and cloud platforms.
  • Implement data quality, validation, transformation, and governance processes.
  • Develop and maintain batch and real-time data processing solutions.
  • Optimize data pipelines for performance, scalability, and cost efficiency.
  • Implement monitoring, logging, and error-handling mechanisms.
  • Collaborate with data architects, analysts, and business stakeholders to deliver data solutions.
  • Follow security best practices for data access, encryption, and compliance.

Skills

Python
PySpark
SQL

Tools

Azure Data Factory
Azure Databricks
Azure Synapse Analytics
Delta Lake
Microsoft Purview

Job description

CloudBigD is seeking an experienced Azure Data Engineer to design, build, and optimize scalable data pipelines on Microsoft Azure. You will work with Azure Data Factory, Databricks, and Synapse Analytics to deliver robust data solutions for analytics and reporting.

The role requires strong SQL and PySpark skills, experience with lakehouse architectures, and a focus on performance, security, and cost efficiency. Collaboration with stakeholders is essential for successful outcomes.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Engineer: Scalable Pipelines & Lakehouse
Azure Data Engineer: Scalable Pipelines & Lakehouse

Robotics Technologies • Raleigh (NC)

On-site
USD 110,000 - 150,000
Azure Data Engineer: Scale Data Pipelines & Lakehouse
Azure Data Engineer: Scale Data Pipelines & Lakehouse

Robotic Process Automation LLC • Raleigh (NC)

On-site
USD 110,000 - 160,000
Azure Databricks Engineer - Scalable Data Pipelines & Lakehouse
Azure Databricks Engineer - Scalable Data Pipelines & Lakehouse

Valiant Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 170,000
Azure Data Engineer: Scale Data Pipelines & Lakehouses
Azure Data Engineer: Scale Data Pipelines & Lakehouses

Trinity Health Group • Spokane (WA)

Hybrid
USD 110,000 - 150,000
Azure Databricks Data Engineer - Lakehouse & Pipelines
Azure Databricks Data Engineer - Lakehouse & Pipelines

Ethereum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 160,000
Azure Data Engineer: Lakehouse & Real-Time Data Pipelines
Azure Data Engineer: Lakehouse & Real-Time Data Pipelines

Robotics Technologies • Raleigh (NC)

On-site
USD 89,000 - 126,000
Azure Data Engineer: Pipelines, Lakehouse & Synapse
Azure Data Engineer: Pipelines, Lakehouse & Synapse

Ethereum Technologies LLC • Raleigh (NC)

On-site
USD 120,000 - 150,000
None
Senior Azure Data Engineer: Pipelines, Databricks & Big Data
Senior Azure Data Engineer: Pipelines, Databricks & Big Data

CloudBigD • Houston (TX)

On-site
USD 120,000 - 160,000
Azure Data Engineer: Pipelines, Lakes & Synapse
Azure Data Engineer: Pipelines, Lakes & Synapse

Experis Technology Group • Jersey City (NJ)

On-site
USD 110,000 - 140,000
Azure Data Engineer: Pipelines, PySpark & Power BI
Azure Data Engineer: Pipelines, PySpark & Power BI

Diverse Lynx, LLC • Cleveland (OH)

On-site
USD 100,000 - 150,000