Senior Data Engineer: Azure Databricks & Spark Expert

Vibotek LLC; Alpharetta (GA); Hybrid; USD 120,000 - 150,000; Full time

Job summary

A leading technology firm located in Alpharetta is looking for an experienced Data Engineer with over 8 years of hands-on experience in enterprise environments. The ideal candidate should have strong expertise in Azure services and proficiency in Apache Spark with Python. Responsibilities include designing end-to-end data solutions and working with cross-functional teams. This position offers a hybrid work model with the possibility for immediate engagement.

Qualifications

• 8+ years of hands-on data engineering experience in enterprise environments.
• Strong expertise in Azure services, especially Azure Databricks.
• Advanced proficiency in Apache Spark with Python (PySpark).

Skills

Data engineering experience

Azure services (Databricks, Functions, Data Factory)

Apache Spark with Python (PySpark)

SQL and query optimization

ETL/ELT methodologies

Delta Lake

Data modelling

Batch and real-time processing

Data architecture principles

Problem-solving skills

Collaboration with cross-functional teams

Tools

Azure DevOps

Git

Terraform

Azure Purview

Job description

A leading technology firm located in Alpharetta is looking for an experienced Data Engineer with over 8 years of hands-on experience in enterprise environments. The ideal candidate should have strong expertise in Azure services and proficiency in Apache Spark with Python. Responsibilities include designing end-to-end data solutions and working with cross-functional teams. This position offers a hybrid work model with the possibility for immediate engagement.