Technical Lead - Production Support (Databricks & ADF)

Anblicks; Dallas (TX); On-site; USD 120,000 - 160,000; Full time

Job summary

Anblicks is seeking a strong Data Engineering Lead with support engagement experience to manage a Data Platform built on Azure and Databricks. This hands-on leader will stabilize, optimize, and mature platform operations through structured processes and automation.

Responsibilities include incident management, proactive monitoring, data governance, and streamlined release practices, driving KPI-based improvements and knowledge sharing across teams.

Qualifications

• 8+ years in data engineering & platform support, with expertise in Azure & Databricks.
• Strong knowledge of incident management, performance tuning, and data governance.
• Hands-on experience with SQL, Python, Spark, and CI/CD pipelines.
• Familiarity with monitoring tools (Azure Monitor etc.) and automation frameworks.
• Excellent communication and stakeholder management skills.

Responsibilities

• Incident & Communication Management – Ensure quick resolution, RCA, and transparent stakeholder updates.
• Proactive Monitoring & Optimization – Improve performance, cost, and reliability.
• Data Quality & Governance – Enforce data integrity, lineage, and compliance.
• Change & Release Management – Streamline deployments and minimize disruptions.
• Reporting & Continuous Improvement – Track KPIs, resolve recurring issues, and drive efficiency.
• Knowledge Management – Maintain documentation, SOPs, and training for seamless operations.
• Automation & Monitoring – Implement self-healing mechanisms and reduce manual overhead.

Skills

Incident management

Performance tuning

Data governance

Stakeholder management

Communication skills

Tools

Azure

Databricks

SQL

Python

Spark

CI/CD

Azure Monitor

Job description

Role Overview

We are seeking a strong Data Engineering Lead with support engagement experience to manage and enhance a Data Platform built on Azure and Databricks. This role requires a hands-on leader who can stabilize, optimize, and mature platform operations through structured processes and automation.

Key Responsibilities

• Incident & Communication Management – Ensure quick resolution, RCA, and transparent stakeholder updates.
• Proactive Monitoring & Optimization – Improve performance, cost, and reliability.
• Data Quality & Governance – Enforce data integrity, lineage, and compliance.
• Change & Release Management – Streamline deployments and minimize disruptions.
• Reporting & Continuous Improvement – Track KPIs, resolve recurring issues, and drive efficiency.
• Knowledge Management – Maintain documentation, SOPs, and training for seamless operations.
• Automation & Monitoring – Implement self-healing mechanisms and reduce manual overhead.

Qualifications

• 8+ years in data engineering & platform support, with expertise in Azure & Databricks.
• Strong knowledge of incident management, performance tuning, and data governance.
• Hands-on experience with SQL, Python, Spark, and CI/CD pipelines.
• Familiarity with monitoring tools (Azure Monitor etc.) and automation frameworks.
• Excellent communication and stakeholder management skills.