Senior Data Engineer

Addison Group; Austin (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Addison Group in Austin, TX is seeking a Senior Data Platform Engineer to design and deliver scalable cloud data platform capabilities. You will lead cross-functional initiatives, own critical production readiness, and mentor engineers while balancing reliability, security, and cost.

This role emphasizes platform modernization, automation, and observability. The candidate will focus ~70% on new capability development and ~30% on operations, support, and platform reliability, working with Azure

Qualifications

• 10+ years in data engineering, data warehousing, or data platform engineering.
• Strong Azure Databricks, Delta Lake, and lakehouse concepts.
• Advanced SQL, Python, PySpark, and Spark performance optimization.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products using Azure Databricks.
• Lead and mentor engineers; drive solution design and production readiness.
• Build batch, streaming, and near-real-time integration with governance and security.
• Develop CI/CD, testing, deployment, and observability processes.
• Collaborate with architecture, governance, and business teams to translate requirements.

Skills

Azure Databricks

PySpark

SQL

Python

Delta Lake

Data modeling

CI/CD

Security controls

Education

Bachelor's degree in CS/IS/Engineering

Master's degree preferred

Tools

Azure DevOps

Git

Delta Live Tables

Databricks Workflows

Unity Catalog

Power BI

Job description

The Senior Data Platform Engineer is a senior individual contributor responsible for designing, developing, enhancing, and supporting enterprise cloud data platform capabilities. Reporting to the Sr. Director of Data Platform Transformations & Operations, this role combines hands-on engineering, solution design, and operational ownership to deliver scalable, secure, and reliable solutions. The engineer serves as a technical lead on major initiatives while remaining accountable for production support and operational excellence.

The role supports platform transformation and steady-state operations, with approximately 70% focused on development of new capabilities and 30% on operations, support, and platform reliability. Primary areas include Azure Databricks, cloud data engineering, platform modernization, automation, governance, and continuous improvement.

Responsibilities

• Design, develop, and maintain scalable data pipelines, ingestion frameworks, transformation processes, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake.
• Implement Bronze, Silver, and Gold architecture patterns supporting enterprise reporting, analytics, AI, and self-service data consumption.
• Build reusable frameworks, utilities, and platform components that improve engineering productivity, quality, consistency, and deployment speed.
• Develop and support batch, near-real-time, and streaming integration solutions, including modernization of legacy warehouse and ETL workloads.
• Serve as technical lead for complex initiatives; develop solution designs, lead technical reviews, recommend tools and approaches, and guide work through production implementation.
• Partner with data architects and platform leaders to ensure solutions are scalable, secure, governed, cost-conscious, and operationally supportable.
• Mentor engineers and promote standards for coding, testing, documentation, performance, and production readiness.
• Implement data quality, validation, reconciliation, monitoring, metadata, and lineage capabilities; support RBAC and enterprise security controls.
• Build and maintain CI/CD, automated testing, deployment, and release processes using Azure DevOps and Git-based practices.
• Contribute to platform observability, alerting, operational dashboards, health metrics, performance tuning, and cost optimization.
• Participate in production support, incident response, pager, and on-call rotations; troubleshoot issues, lead root cause analysis, and implement durable remediation.
• Create and maintain operational runbooks, support procedures, technical documentation, and knowledge-sharing assets.
• Collaborate with architecture, governance, security, analytics, application, and business teams to translate requirements into scalable platform solutions.
• Support technical discovery, estimation, roadmap planning, delivery execution, and evaluation of emerging cloud, data, and AI capabilities.

Job Requirements

Education

• Bachelor's degree in Computer Science, Information Systems, Engineering, Data Science, or a related field.
• Master's degree preferred.

Experience

• 10+ years of experience in data engineering, data warehousing, or data platform engineering.
• 5+ years of experience designing or implementing cloud-based data warehouse, data lake, or data platform solutions.
• 3+ years of hands-on Azure Databricks experience in enterprise production environments.
• Experience with ETL/ELT development, data modeling, large-scale integration, and cloud platform modernization.
• Experience leading technical implementations, solution design, design reviews, and production releases.
• Experience supporting production environments, incident response, root cause analysis, and operational support processes.
• Experience in regulated, financial services, banking, or audit-sensitive environments preferred.
• Advanced proficiency in SQL, Python, PySpark, and Spark performance optimization.
• Strong experience with Azure Databricks, Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables.
• Strong understanding of lakehouse, Medallion Architecture, dimensional modeling, data warehousing, and analytics-oriented data structures.
• Experience with Azure DevOps, Git, CI/CD, automated testing, and deployment practices; Databricks Asset Bundles experience preferred.
• Experience implementing data quality, validation, reconciliation, monitoring, metadata, lineage, governance, and security controls.
• Strong technical leadership, solution design, analytical, troubleshooting, and problem-solving skills.
• Ability to independently lead complex work from design through production support and balance innovation with reliability, security, cost, and supportability.
• Excellent collaboration, mentoring, documentation, and communication skills across technical and business teams.
• Experience with Power BI, MicroStrategy, MDM, streaming architectures, infrastructure automation, or AI-assisted development preferred.
• Databricks Data Engineer certification at the Associate or Professional level preferred.