Senior Data Engineer on-site)

Ziosk; Dallas (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Ziosk is seeking a Senior Data Engineer to design and optimize our modern data platform from Plano, TX. This on-site role focuses on building scalable ETL pipelines with Azure Databricks, Delta Lake, and governance practices for reliable analytics.

You will collaborate with data scientists, analysts, and cloud teams to deliver trusted datasets, ensure security, and maintain performance in a high-volume environment. Strong PySpark/SQL, Delta Lake, and CI/CD experience are essential.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, with 3+ years focused on Azure Databricks and Spark (PySpark).
• Expertise in Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing.
• Hands-on experience with Azure cloud services: ADLS Gen2, Event Hub, ADF, Key Vault, Synapse.
• Strong programming skills in Python, SQL, and familiarity with Scala.
• Solid understanding of data modeling (OLTP/OLAP), star/snowflake schemas, and performance tuning.
• Experience with CI/CD pipelines in Databricks using Databricks CLI, Terraform, or Azure DevOps.

Responsibilities

• Design and implement scalable ETL/ELT pipelines using Azure Databricks (PySpark, Spark SQL).
• Build robust data ingestion workflows from diverse sources into Bronze/Silver/Gold layers of medallion architecture.
• Define and implement best practices for Delta Lake and Unity Catalog for governance.
• Optimize performance and cost of large-scale data pipelines through partitioning, caching, clustering, and tuning Spark.
• Integrate Databricks with ADLS Gen2, ADF, Synapse, Event Hub, Key Vault, and Azure DevOps.
• Implement secure and governed access to data using managed identities, RBAC, and data masking techniques.
• Partner with data analysts, data scientists, and business users to deliver trusted datasets and KPIs.
• Provide technical leadership and mentorship to junior engineers, conducting code reviews and guiding architecture.
• Collaborate with enterprise architects and InfoSec to ensure data security and privacy policies.
• Implement data quality checks, alerts, and lineage tracking using tools such as Great Expectations, Unity Catalog, and Data Observability platforms.
• Set up monitoring dashboards, alerts, and job recovery mechanisms for mission-critical pipelines.

Skills

Python

SQL

Scala

Spark

Databricks

Delta Lake

Unity Catalog

CI/CD

ETL/ELT

Data Modeling

Education

Bachelor's or Master's degree in Computer Science, Engineering, or related field

Tools

Azure Databricks

Azure Data Factory

ADLS Gen2

Azure Synapse

Event Hub

Azure DevOps

Job description

Senior Data Engineer (on-site) – Plano, TX

We are looking for a highly skilled and experienced Senior Data Engineer with deep expertise in Azure Databricks to contribute to the design, development, and optimization of our modern data platform. In this role, you will be responsible for building scalable ETL pipelines, implementing data lakehouse architectures, and supporting advanced analytics use cases in a high-volume data environment. You will work closely with data architects, data scientists, business stakeholders, and cloud infrastructure teams to deliver reliable, secure, and high-performance data solutions.

Responsibilities

• Design and implement scalable ETL/ELT pipelines using Azure Databricks (PySpark, Spark SQL).
• Build robust data ingestion workflows from diverse sources (Event Hub, APIs, Azure SQL, Blob Storage, etc.) into the Bronze/Silver/Gold layers of the medallion architecture.
• Define and implement best practices for Delta Lake and Unity Catalog for schema evolution, versioning, and governance.
• Optimize performance and cost efficiency of large-scale data pipelines through intelligent partitioning, caching, clustering, and tuning Spark configurations.
• Integrate Databricks with Azure Data Lake Storage (ADLS Gen2), Azure Data Factory (ADF), Azure Synapse, Event Hub, Key Vault, and Azure DevOps.
• Implement secure and governed access to data using managed identities, RBAC, and data masking techniques.
• Partner with data analysts, data scientists, and business users to understand data needs and deliver trusted datasets and KPIs.
• Provide technical leadership and mentorship to junior engineers, conducting code reviews and guiding architectural decisions.
• Collaborate with enterprise architects and InfoSec to ensure compliance with data security and privacy policies.
• Implement data quality checks, alerts, and lineage tracking using tools such as Great Expectations, Unity Catalog, and Data Observability platforms.
• Set up monitoring dashboards, alerts, and job recovery mechanisms for mission-critical pipelines.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, with 3+ years focused on Azure Databricks and Spark (PySpark).
• Expertise in Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing.
• Hands‑on experience with Azure cloud services: ADLS Gen2, Event Hub, ADF, Key Vault, Synapse.
• Strong programming skills in Python, SQL, and familiarity with Scala.
• Solid understanding of data modeling (OLTP/OLAP), star/snowflake schemas, and performance tuning.
• Experience with CI/CD pipelines in Databricks using Databricks CLI, Terraform, or Azure DevOps.

Preferred Qualifications

• Databricks certification (e.g., Databricks Certified Data Engineer Professional).
• Experience with real‑time streaming (e.g., Structured Streaming, Event Hub, Kafka).
• Familiarity with MLflow or machine learning pipeline support in Databricks.
• Experience with Power BI, Azure Purview, or Microsoft Fabric for downstream reporting.
• Knowledge of data governance, lineage, and compliance standards (GDPR, HIPAA, SOC2).

Ziosk is an Equal Opportunity employer offering competitive benefits and compensation. Candidates must be eligible to work in the U.S. and be able to commute to Plano, TX daily. Applicants must be authorized to work for any employer in the U.S. No agencies or third-party recruiters, please.