Azure Databricks Data Engineer: Spark & Delta Lake Expert

Prodapt; Irving (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Prodapt is seeking an experienced Azure Databricks Engineer to join our Dallas, TX team. The candidate will design and build scalable data pipelines using Azure Databricks, PySpark, Python, and SQL, and modernize existing Python apps into distributed Spark processing.

The role requires strong expertise in Delta Lake, Azure Data Factory, and ADLS Gen2, along with a track record of delivering large-scale data solutions and data quality controls.

Qualifications

• 10+ years of Data Engineering experience with Azure data services.
• Bachelor’s degree in CS/Engineering or related field, or equivalent.
• 4+ years of Azure Databricks experience across Azure services.
• 7+ years of Python development experience.
• 4+ years of Snowflake or SQL experience.
• Expert PySpark and Spark SQL knowledge.
• Experience with Delta Lake and Lakehouse architecture.
• Git, CI/CD, and DevOps practices.

Responsibilities

• Design, develop, and implement scalable data pipelines using Azure Databricks.
• Build and optimize ETL/ELT pipelines with PySpark, Python, and SQL.
• Develop data solutions using the Medallion Architecture (Bronze, Silver, Gold).
• Work with Delta Lake and advanced Databricks optimization techniques.
• Integrate Databricks with Azure services (ADLS Gen2, ADF, Synapse, Key Vault).
• Lead technical discussions and mentor junior engineers.

Skills

Azure Databricks

Python

Spark SQL

ETL/ELT design

Data pipelines

Education

Bachelor’s degree in Computer Science or related field

Equivalent education/experience

Tools

Azure Data Factory

ADLS Gen2

Azure Synapse

Delta Lake

Git/CI-CD

Job description

Prodapt is seeking an experienced Azure Databricks Engineer to join our Dallas, TX team. The candidate will design and build scalable data pipelines using Azure Databricks, PySpark, Python, and SQL, and modernize existing Python apps into distributed Spark processing.

The role requires strong expertise in Delta Lake, Azure Data Factory, and ADLS Gen2, along with a track record of delivering large-scale data solutions and data quality controls.