Azure Data Engineer

Cpctechno; Kennesaw (GA); On-site; USD 110,000 - 140,000; Full time

Job summary

Cpctechno in Georgia seeks a data engineer with 6-8 years of experience in Python and Spark to design and scale robust data solutions. You will build and optimize data pipelines, perform root cause analysis, and ensure high performance across distributed Spark jobs.

The role requires strong SQL skills and cloud experience with Azure, including Synapse and Data Factory, and collaboration with IT, analytics, and stakeholders to deliver reliable data platforms.

Qualifications

• 6-8 years of experience developing data solutions in Python with Spark.
• Strong SQL proficiency and data modeling skills.
• Experience with PySpark in distributed environments.
• Cloud knowledge focusing on Azure.

Responsibilities

• Designs, modifies, and builds scalable data processes.
• Identify performance bottlenecks in Spark jobs and perform root cause analysis.
• Develop parallel data processing platforms in PySpark.
• Collaborate with stakeholders and IT to resolve data issues.

Skills

Python

Spark

SQL

Data engineering

Distributed computing

PySpark

Azure

Tools

Azure Synapse

Azure Data Factory

Spark Jobs

Job description

6 - 8 years’ of experience on developing data solutions in Python using Spark framework

Job Description

• Designs, modifies, and builds new and scalable data processes.
• Ability to perform root cause analysis and identify performance bottlenecks in Spark Jobs.
• Expert in Data Engineering and building data pipelines, implementing Algorithms in a distributed environment.
• Ability to design and develop parallel processing data platform in PySpark.
• Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
• Strong Proficiency in SQL.
• Cloud knowledge especially Azure
• Collaborates with stakeholders, IT, database engineers and other scientists.
• Hands-on knowledge in Azure Synapse and Azure Data Factory is a plus.