Senior Azure Data Engineer

Yoh, A Day & Zimmermann Company

Plano (TX)

On-site

USD 105,000 - 150,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical, Prescription, Dental & Vision
Health Savings Account (HSA)
Life & Disability Insurance
MetLife Voluntary Benefits
Employee Assistance Program (EAP)
401K Retirement Savings Plan
Direct Deposit & weekly epayroll
Referral Bonus Programs
Certification and training

Job summary

Yoh, A Day & Zimmermann Company is seeking a Senior Data Engineer to own enterprise-grade lakehouse data pipelines on Azure Databricks. You will design and optimize ETL/ELT pipelines, implement governance and data quality, and collaborate with analytics teams to deliver trusted datasets and KPIs.

Hybrid/onsite in Plano, TX with relocation not supported; require 5+ years in data engineering and strong PySpark/Delta Lake experience.

Qualifications

  • Bachelor’s degree in CS, Engineering, or related field (or equivalent experience).
  • 5+ years of data engineering experience.
  • 3+ years with Azure Databricks / Apache Spark using PySpark.
  • Strong experience with Delta Lake, Unity Catalog, Databricks notebooks, and SQL-based warehousing patterns.
  • Hands-on experience with ADLS Gen2, Event Hub, ADF, Key Vault, and Synapse.
  • Strong coding skills in Python and SQL; Scala is a plus.
  • Understanding of data modeling for OLTP/OLAP, star/snowflake schemas, and performance tuning.
  • Experience implementing CI/CD for Databricks using Databricks CLI, Terraform, and/or Azure DevOps.
  • Databricks certification (e.g., Databricks Certified Data Engineer Professional).
  • Experience with streaming/real-time pipelines using Structured Streaming and Event Hub.
  • ML platform exposure (MLflow).
  • Reporting/consumption layers such as Power BI, Purview, or Microsoft Fabric.
  • Knowledge of GDPR, HIPAA, SOC2.
  • Must reside in the Dallas–Fort Worth (DFW) area and hybrid schedule in Plano, TX.
  • Relocation is not available.
  • Authorized to work in the United States without sponsorship.

Responsibilities

  • Design and build scalable ETL/ELT pipelines using Azure Databricks with PySpark and Spark SQL.
  • Develop ingestion workflows from multiple source systems into Bronze/Silver/Gold layers (Event Hub, APIs, Azure SQL, Blob Storage).
  • Apply best practices for Delta Lake and Unity Catalog, including schema evolution, versioning, and governance.
  • Optimize pipeline performance and cost using partitioning, caching, clustering, and Spark tuning.
  • Integrate Databricks with Azure services including ADLS Gen2, ADF, Synapse, Event Hub, Key Vault, and Azure DevOps.
  • Implement secure access controls using managed identities, RBAC, and data protection practices such as masking.
  • Collaborate with analytics and business teams to deliver trusted datasets, metrics, and KPIs.
  • Provide technical leadership through mentoring, code reviews, and architectural guidance.
  • Work with enterprise architecture and security teams to align data solutions with data privacy/security policies.
  • Implement data quality checks, alerting, and lineage tracking using tools such as Great Expectations, Unity Catalog, and data observability platforms.
  • Build monitoring dashboards, alerts, and recovery mechanisms for critical pipelines.

Skills

Python
SQL
PySpark
Spark SQL
Delta Lake
Unity Catalog
Databricks
Azure Databricks
Structured Streaming
MLflow
Power BI
Purview
Microsoft Fabric
Kafka
Terraform
Azure DevOps
RBAC
Great Expectations

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

Databricks CLI
Terraform
Azure DevOps
Azure Data Factory
Azure Synapse
ADLS Gen2
Key Vault
Event Hub
Power BI
Purview

Job description

Own enterprise-grade lakehouse data engineering on Azure, building and optimizing scalable ETL/ELT pipelines with Azure Databricks.

Responsibilities
  • Design and build scalable ETL/ELT pipelines using Azure Databricks with PySpark and Spark SQL
  • Develop ingestion workflows from multiple source systems into Bronze/Silver/Gold layers (e.g., Event Hub, APIs, Azure SQL, Blob Storage)
  • Apply best practices for Delta Lake and Unity Catalog, including schema evolution, versioning, and governance
  • Optimize pipeline performance and cost using partitioning, caching, clustering, and Spark tuning
  • Integrate Databricks with key Azure services including ADLS Gen2, Azure Data Factory (ADF), Azure Synapse, Event Hub, Key Vault, and Azure DevOps
  • Implement secure access controls using managed identities, RBAC, and data protection practices such as masking
  • Collaborate with analytics and business teams to deliver trusted datasets, metrics, and KPIs
  • Provide technical leadership through mentoring, code reviews, and architectural guidance
  • Work with enterprise architecture and security teams to align data solutions with data privacy/security policies
  • Implement data quality checks, alerting, and lineage tracking using tools such as Great Expectations, Unity Catalog, and data observability platforms
  • Build monitoring dashboards, alerts, and recovery mechanisms for critical pipelines
Requirements
  • Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience)
  • 5+ years of data engineering experience
  • 3+ years with Azure Databricks / Apache Spark using PySpark
  • Strong experience with Delta Lake, Unity Catalog, Databricks notebooks, and SQL-based warehousing patterns
  • Hands‑on experience with ADLS Gen2, Event Hub, ADF, Key Vault, and Synapse
  • Strong coding skills in Python and SQL; Scala is a plus
  • Understanding of data modeling for OLTP/OLAP, star/snowflake schemas, and performance tuning
  • Experience implementing CI/CD for Databricks using Databricks CLI, Terraform, and/or Azure DevOps
  • Databricks certification (e.g., Databricks Certified Data Engineer Professional)
  • Experience with streaming/real‑time pipelines using Structured Streaming and Event Hub (Kafka experience is referenced)
  • ML platform exposure (e.g., MLflow or supporting ML pipelines)
  • Reporting/consumption layer experience such as Power BI, Purview, or Microsoft Fabric
  • Knowledge of governance/compliance standards including GDPR, HIPAA, and SOC2
  • Must currently reside in the Dallas–Fort Worth (DFW) area and be able to work a hybrid schedule in Plano, TX
  • Relocation is not available
  • Candidates must be authorized to work in the United States without sponsorship now or in the future
Technologies
  • Azure Databricks
  • PySpark, Spark SQL
  • Delta Lake, Unity Catalog
  • Python, SQL, Scala
  • ADLS Gen2
  • Azure Data Factory (ADF)
  • Azure Synapse
  • Event Hub
  • Key Vault, Azure DevOps
  • Managed identities, RBAC, masking
  • Great Expectations
  • Data observability platforms
  • Databricks notebooks, Databricks CLI
  • Terraform
  • Structured Streaming, Kafka
  • MLflow
  • Power BI, Purview, Microsoft Fabric
Benefits
  • Medical, Prescription, Dental & Vision Benefits (for employees working 20+ hours per week)
  • Health Savings Account (HSA) (for employees working 20+ hours per week)
  • Life & Disability Insurance (for employees working 20+ hours per week)
  • MetLife Voluntary Benefits
  • Employee Assistance Program (EAP)
  • 401K Retirement Savings Plan
  • Direct Deposit & weekly epayroll
  • Referral Bonus Programs
  • Certification and training opportunities

Location: Plano, TX (onsite)
Compensation: USD 105,000 - 150,000 per year

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Databricks Data Engineer
Azure Databricks Data Engineer

Tata Consultancy Services Limited • St. Louis (MO)

On-site
USD 110,000 - 135,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Maternal & Parental Leaves
+10
Senior Data Engineer on-site)
Senior Data Engineer on-site)

Ziosk • Dallas (TX)

On-site
USD 140,000 - 190,000
Senior Data Engineer
Senior Data Engineer

Hillwood Development Company LLC • Dallas (TX)

On-site
USD 140,000 - 180,000
Medical Insurance
Dental Insurance
Vision Insurance
+12
Senior Databricks Data Engineer
Senior Databricks Data Engineer

CGI Group, Inc. • Miami (FL)

Hybrid
USD 92,000 - 204,000
Competitive compensation
Comprehensive insurance options
401(k) match and share purchase plan
+3
Databricks Engineer
Databricks Engineer

Dataflix • United States

Remote
USD 130,000 - 170,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

Capgemini • New York (NY)

On-site
USD 89,000 - 166,000
Paid time off
Health benefits
401(k) plan
+1
Senior Data Engineer
Senior Data Engineer

PharmaLink, LLC • Largo (FL)

On-site
USD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

HumCap • Plano (TX)

Hybrid
USD 120,000 - 170,000
Azure Databricks Engineer (Dallas, TX)
Azure Databricks Engineer (Dallas, TX)

Cedent Consulting Inc. • Dallas (TX)

On-site
USD 120,000 - 150,000
Azure Databricks Engineer
Azure Databricks Engineer

ZEUS SOLUTIONS INC • Houston (TX)

On-site
USD 140,000 - 180,000