Technology Lead | Azure Data Engineering

Veriipro; Atlanta (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Veriipro in Atlanta seeks an experienced Technology Lead / Azure Data Engineer to drive data engineering initiatives with hands-on Azure Synapse Analytics and PySpark expertise.

You will design and optimize data pipelines, apply Spark architectures to large-scale workloads, and collaborate across teams to deliver scalable data solutions. Microsoft Fabric familiarity is a nice-to-have, supporting modern cloud data platforms.

Qualifications

• Must have 8+ years in data engineering/cloud platforms or related roles.
• Strong hands-on experience with Azure Synapse Analytics and PySpark/Spark SQL.
• Deep understanding of Apache Spark and distributed data processing.

Responsibilities

• Design, develop, and optimize data solutions using Azure Synapse Analytics.
• Develop scalable datapipelines and transformations using PySpark and Spark SQL.
• Apply Spark architecture and distributed processing concepts to large-scale data workloads.
• Design and implement Data Lake and Medallion Architecture solutions.
• Troubleshoot performance, data quality, and pipeline-related issues.
• Collaborate with cross-functional teams to understand requirements and deliver scalable solutions.
• Evaluate and leverage Fabric capabilities where appropriate.
• Provide technical leadership and contribute to solution architecture and development standards.

Skills

Azure Synapse Analytics

PySpark

Apache Spark architecture

Spark SQL

Data Lake architecture

Medallion Architecture

Data transformation and processing

Strong troubleshooting and problem-sol

Strong communication and collaboration

Tools

Microsoft Fabric

Job description

We are seekingan experiencedTechnology Lead / AzureData Engineerwith strong hands-on expertiseinAzure Synapse Analyticsand PySpark. The ideal candidatewill have adeep understanding ofApache Spark, distributeddata processing, DataLake architecture, andmodern data engineeringpractices.

Must-Have Skills

• Azure Synapse Analytics
• PySpark
• Apache Spark architecture
• Spark SQL
• Data Lake architecture
• Medallion Architecture
• Data transformationand processing
• Strong troubleshooting andproblem-solving skills

Nice-to-Have

• Microsoft Fabric
• Experience with moderncloud data platforms

Responsibilities

• Design, develop, and optimizedata solutions usingAzure Synapse Analytics.
• Develop scalable datapipelines and transformationsusingPySpark and SparkSQL.
• ApplyApache Spark architectureand distributed processingconcepts to large-scale data workloads.
• Design and implementData Lake andMedallion Architecturesolutions.
• Troubleshoot performance,data quality, andpipeline-related issues.
• Collaborate with cross-functional teams tounderstand requirementsand deliver scalablesolutions.
• Evaluate and leverageMicrosoft Fabriccapabilities where appropriate.
• Provide technical leadershipand contribute tosolution architecture anddevelopment standards.

Required Qualifications

• 8+ years ofexperiencein data engineering, cloud platforms, orrelated technology roles.
• Strong hands-onexperience withAzure Synapse Analytics.
• Strong programming experiencewithPySparkandSpark SQL.
• Deep understanding ofApache Sparkand distributed dataprocessing.
• Experience designing andimplementingData Lake architectures.
• Excellent communicationand collaboration skills.
• Strong analytical, troubleshooting, and problem-solving abilities.