Azure Data Platform Lead: Lakehouse & ML Enablement

Confidentiel; Pittsburgh (Allegheny County); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Confidentiel in the United States seeks a Cloud Platform Manager to design, build, and manage scalable Azure-based data platforms. You will lead data engineers and analysts, architect Lakehouse environments with Delta Lake, and enable analytics, BI, and ML initiatives using PySpark, SQL, and Spark optimization.

The role blends hands-on coding with strategic governance, RBAC, and data quality controls, collaborating with stakeholders across the business.

Qualifications

• 5–7 years hands-on data engineering, cloud architecture, or enterprise data platform development.
• 2–4 years with Azure Databricks and Azure cloud data tech.

Responsibilities

• Lead data engineering teams and mentor analysts and engineers.
• Architect scalable Azure Databricks solutions and modern Lakehouse environments.
• Build and optimize batch and streaming ETL/ELT pipelines using PySpark, SQL, and Python.
• Implement data governance, RBAC, lineage, and security controls in Unity Catalog.
• Collaborate with stakeholders to translate requirements into scalable data solutions.
• Ensure reliability, performance, and cost efficiency of cloud data platforms.
• Support ML initiatives and model deployment with MLOps practices.

Skills

Leadership

Mentoring

SQL

Python

PySpark

Scala

PowerShell

Data modeling

Communication

Problem solving

Education

Bachelor's degree in a technical field

Tools

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

Azure Synapse

Delta Lake

MLflow

Airflow

Azure DevOps

Kafka

Job description

Confidentiel in the United States seeks a Cloud Platform Manager to design, build, and manage scalable Azure-based data platforms. You will lead data engineers and analysts, architect Lakehouse environments with Delta Lake, and enable analytics, BI, and ML initiatives using PySpark, SQL, and Spark optimization.

The role blends hands-on coding with strategic governance, RBAC, and data quality controls, collaborating with stakeholders across the business.