Senior Data Engineer - Azure Databricks Specialist (Plano)

Stelvio Inc.; Plano (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Stelvio Inc. in Plano, TX is seeking a Senior Data Engineer to design and optimize a modern Azure data platform handling high-volume datasets and analytics-driven needs.

You will build scalable ETL/ELT pipelines, implement medallion architecture, and collaborate with analysts, scientists, and security teams. The role emphasizes hands-on development with Databricks, Delta Lake, Unity Catalog, and cloud data services, with a focus on performance, cost optimization, and data governance.

Qualifications

• :Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• :5+ years of data engineering experience.
• :3+ years hands-on experience with Azure Databricks and Spark/PySpark.
• :Strong expertise in Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing.
• :Hands-on experience with ADLS Gen2, Event Hub, Azure Data Factory, Key Vault, and Azure Synapse.
• :Strong programming skills in Python and SQL, with familiarity with Scala.
• :Solid understanding of OLTP/OLAP data modelling, star and snowflake schemas, and performance tuning.
• :Experience implementing CI/CD for Databricks using Databricks CLI, Terraform, or Azure DevOps.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Spark SQL.
• Build data ingestion workflows across sources including Event Hub, APIs, Azure SQL, and Blob Storage.
• Develop and maintain Bronze, Silver, and Gold layers within a medallion architecture.
• Implement best practices across Delta Lake and Unity Catalog, including schema evolution, versioning, and governance.
• Optimize large-scale data pipelines for performance and cost through partitioning, caching, clustering, and Spark configuration tuning.
• Integrate Databricks with ADLS Gen2, Azure Data Factory, Azure Synapse, Event Hub, Key Vault, and Azure DevOps.
• Implement secure data access using managed identities, RBAC, and data masking.
• Work with data analysts, data scientists, business stakeholders, architects, and cloud teams to deliver trusted datasets and KPIs.
• Provide technical guidance and mentorship to junior engineers, including code reviews and architectural input.
• Partner with enterprise architecture and information security teams to support data security and privacy requirements.
• Implement data quality checks, alerts, and lineage tracking using tools such as Great Expectations, Unity Catalog, and data observability platforms.
• Build monitoring, alerting, and recovery processes for mission-critical data pipelines.

Skills

Python

SQL

Scala familiarity

CI/CD for Databricks

Performance tuning

OLTP/OLAP modelling

Data governance

Data masking

Mentorship

Code reviews

Education

Bachelor’s or Master’s degree in Computer Science / Engineering or related field

Tools

Azure Databricks

Spark/PySpark

Delta Lake

Unity Catalog

ADLS Gen2

Azure Data Factory

Azure Synapse

Event Hub

Databricks CLI

Terraform

Azure DevOps

Job description

Stelvio Inc. in Plano, TX is seeking a Senior Data Engineer to design and optimize a modern Azure data platform handling high-volume datasets and analytics-driven needs.

You will build scalable ETL/ELT pipelines, implement medallion architecture, and collaborate with analysts, scientists, and security teams. The role emphasizes hands-on development with Databricks, Delta Lake, Unity Catalog, and cloud data services, with a focus on performance, cost optimization, and data governance.