Azure Data Engineering Lead | Spark, Synapse & Data Lake

Veriipro; Atlanta (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Veriipro in Atlanta seeks an experienced Technology Lead / Azure Data Engineer to drive data engineering initiatives with hands-on Azure Synapse Analytics and PySpark expertise.

You will design and optimize data pipelines, apply Spark architectures to large-scale workloads, and collaborate across teams to deliver scalable data solutions. Microsoft Fabric familiarity is a nice-to-have, supporting modern cloud data platforms.

Qualifications

• Must have 8+ years in data engineering/cloud platforms or related roles.
• Strong hands-on experience with Azure Synapse Analytics and PySpark/Spark SQL.
• Deep understanding of Apache Spark and distributed data processing.

Responsibilities

• Design, develop, and optimize data solutions using Azure Synapse Analytics.
• Develop scalable datapipelines and transformations using PySpark and Spark SQL.
• Apply Spark architecture and distributed processing concepts to large-scale data workloads.
• Design and implement Data Lake and Medallion Architecture solutions.
• Troubleshoot performance, data quality, and pipeline-related issues.
• Collaborate with cross-functional teams to understand requirements and deliver scalable solutions.
• Evaluate and leverage Fabric capabilities where appropriate.
• Provide technical leadership and contribute to solution architecture and development standards.

Skills

Azure Synapse Analytics

PySpark

Apache Spark architecture

Spark SQL

Data Lake architecture

Medallion Architecture

Data transformation and processing

Strong troubleshooting and problem-sol

Strong communication and collaboration

Tools

Microsoft Fabric

Job description

Veriipro in Atlanta seeks an experienced Technology Lead / Azure Data Engineer to drive data engineering initiatives with hands-on Azure Synapse Analytics and PySpark expertise.

You will design and optimize data pipelines, apply Spark architectures to large-scale workloads, and collaborate across teams to deliver scalable data solutions. Microsoft Fabric familiarity is a nice-to-have, supporting modern cloud data platforms.