Senior Data Engineer (Azure Databricks) - On-site in Plano

Ziosk; Dallas (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Ziosk is seeking a Senior Data Engineer to design and optimize our modern data platform from Plano, TX. This on-site role focuses on building scalable ETL pipelines with Azure Databricks, Delta Lake, and governance practices for reliable analytics.

You will collaborate with data scientists, analysts, and cloud teams to deliver trusted datasets, ensure security, and maintain performance in a high-volume environment. Strong PySpark/SQL, Delta Lake, and CI/CD experience are essential.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, with 3+ years focused on Azure Databricks and Spark (PySpark).
• Expertise in Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing.
• Hands-on experience with Azure cloud services: ADLS Gen2, Event Hub, ADF, Key Vault, Synapse.
• Strong programming skills in Python, SQL, and familiarity with Scala.
• Solid understanding of data modeling (OLTP/OLAP), star/snowflake schemas, and performance tuning.
• Experience with CI/CD pipelines in Databricks using Databricks CLI, Terraform, or Azure DevOps.

Responsibilities

• Design and implement scalable ETL/ELT pipelines using Azure Databricks (PySpark, Spark SQL).
• Build robust data ingestion workflows from diverse sources into Bronze/Silver/Gold layers of medallion architecture.
• Define and implement best practices for Delta Lake and Unity Catalog for governance.
• Optimize performance and cost of large-scale data pipelines through partitioning, caching, clustering, and tuning Spark.
• Integrate Databricks with ADLS Gen2, ADF, Synapse, Event Hub, Key Vault, and Azure DevOps.
• Implement secure and governed access to data using managed identities, RBAC, and data masking techniques.
• Partner with data analysts, data scientists, and business users to deliver trusted datasets and KPIs.
• Provide technical leadership and mentorship to junior engineers, conducting code reviews and guiding architecture.
• Collaborate with enterprise architects and InfoSec to ensure data security and privacy policies.
• Implement data quality checks, alerts, and lineage tracking using tools such as Great Expectations, Unity Catalog, and Data Observability platforms.
• Set up monitoring dashboards, alerts, and job recovery mechanisms for mission-critical pipelines.

Skills

Python

SQL

Scala

Spark

Databricks

Delta Lake

Unity Catalog

CI/CD

ETL/ELT

Data Modeling

Education

Bachelor's or Master's degree in Computer Science, Engineering, or related field

Tools

Azure Databricks

Azure Data Factory

ADLS Gen2

Azure Synapse

Event Hub

Azure DevOps

Job description

Ziosk is seeking a Senior Data Engineer to design and optimize our modern data platform from Plano, TX. This on-site role focuses on building scalable ETL pipelines with Azure Databricks, Delta Lake, and governance practices for reliable analytics.

You will collaborate with data scientists, analysts, and cloud teams to deliver trusted datasets, ensure security, and maintain performance in a high-volume environment. Strong PySpark/SQL, Delta Lake, and CI/CD experience are essential.