Azure Data Platform Engineer: Delta Lake & Spark Expert

TechWish; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

TechWish in New York is seeking a senior Data Engineer to design and deliver scalable data platforms and pipelines across the SDLC. You will build robust data solutions with a focus on performance, reliability, and governance.

Leveraging Azure Databricks, Delta Lake, and Delta Live Tables, you will implement lakehouse architectures, ensure data quality, and optimize costs for enterprise-scale workloads. The role requires strong SQL, Python, and Spark skills and collaboration across distributed

Qualifications

• 6–10 years of experience in Data Engineering, Application Development, System Integration, or EDW projects.
• Proven expertise building scalable data pipelines on Azure Databricks with Delta Lake and DLT.
• Experience developing batch and streaming pipelines with PySpark, Spark SQL, and Python.
• Strong SQL programming and data modeling for relational, dimensional, and lakehouse architectures.
• Knowledge of Delta Lake features: ACID, schema evolution, time travel, CDC, and data quality.
• Experience with data quality, monitoring, and orchestration using Delta Live Tables, Databricks Workflows, and Azure services.

Skills

Data engineering

Python

PySpark

SQL

Azure Databricks

Delta Lake

Delta Live Tables

Data modeling

SDLC & CI/CD

Cost optimization

Education

Bachelor's degree in CS/IT/Engineering or related

Tools

Azure Databricks

Delta Lake

Delta Live Tables

Medallion Architecture

Spark SQL

Job description

TechWish in New York is seeking a senior Data Engineer to design and deliver scalable data platforms and pipelines across the SDLC. You will build robust data solutions with a focus on performance, reliability, and governance.

Leveraging Azure Databricks, Delta Lake, and Delta Live Tables, you will implement lakehouse architectures, ensure data quality, and optimize costs for enterprise-scale workloads. The role requires strong SQL, Python, and Spark skills and collaboration across distributed