Azure Data Engineer: Scalable Pipelines & Lakehouse

Signature IT World Inc; New Jersey; On-site; USD 120,000 - 150,000; Full time

Job summary

Signature IT World Inc. is seeking an experienced Azure Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data solutions using Microsoft Azure.

The ideal candidate will have strong expertise in Azure Data Factory, Azure Databricks, PySpark, SQL, and data warehousing concepts. The role involves building ETL/ELT pipelines, optimizing data workflows, and ensuring data quality and security across environments.

Qualifications

• Experience designing and implementing large-scale ETL/ELT pipelines on Azure.
• Strong SQL skills and data warehousing concepts.
• Knowledge of Delta Lake and lakehouse architectures.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory.
• Build and optimize data processing workflows with Azure Databricks and PySpark.
• Develop complex SQL queries, stored procedures, and data transformation logic.
• Integrate data from databases, APIs, files, and cloud storage.
• Implement data lake and lakehouse solutions with Azure Data Lake Storage Gen2.
• Work with Delta Lake and efficient data processing strategies.
• Ensure data quality, security, and reliability across pipelines.
• Optimize pipelines and SQL queries for performance and cost.
• Implement monitoring, logging, and error-handling mechanisms.
• Collaborate with data architects, analysts, and stakeholders to deliver data solutions.
• Support CI/CD pipelines and deployment automation for data workloads.
• Follow data governance, access control, and cloud security best practices.

Skills

ETL/ELT design

SQL proficiency

Data modeling

Cloud data engineering

Tools

Azure Data Factory

Azure Databricks

PySpark

Azure Data Lake Storage Gen2

Delta Lake

Lakehouse architecture

Terraform

Job description

Signature IT World Inc. is seeking an experienced Azure Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data solutions using Microsoft Azure.

The ideal candidate will have strong expertise in Azure Data Factory, Azure Databricks, PySpark, SQL, and data warehousing concepts. The role involves building ETL/ELT pipelines, optimizing data workflows, and ensuring data quality and security across environments.