Azure Databricks Data Platform Leader (Hybrid)

Buchanan Ingersoll & Rooney PC; Pittsburgh (Allegheny County); Hybrid; USD 140,000 - 180,000; Full time

Benefits offered by this job

Hybrid Schedule

Medical, Dental, Vision insurance

401K Program

Generous Paid Time Off

Paid Holidays

Job summary

Buchanan Ingersoll & Rooney PC is seeking a Data Platform Manager for our Pittsburgh, PA location. You will lead the design, build, and optimization of scalable enterprise data platforms on Databricks in Azure, mentoring a team of data engineers and data scientists.

The role requires deep Databricks expertise, strong Azure data services knowledge, and a leadership mindset to deliver high-performance data engineering and analytics solutions. Hybrid schedule: 3 days in office, 2 remote.

Qualifications

• 5-7+ years hands-on data engineering or architecture, with at least 2-4 years on Azure Databricks
• 2-5 years experience managing a team of data engineers/scientists/analysts
• Bachelor's degree in Computer Science, Engineering, or related field
• Certifications preferred: Microsoft Azure Data Engineer, Databricks Certified, or Azure Solutions Architect
• Proficiency in Relational and NoSQL databases and data modeling
• Experience with Spark, Hadoop, Presto/Trino for big data
• Knowledge of Parquet/Avro/ORC formats and compression techniques
• Programming languages: Python (PySpark), SQL, PowerShell, Scala
• Azure cloud infra experience: Networking, Key Vault, Identity Management
• Big Data tools: Spark, MLflow, Airflow

Responsibilities

• Lead and mentor a team of data engineers and data scientists
• Collaborate with stakeholders to translate requirements into specs
• Design, develop and deploy scalable ETL/ELT pipelines on Databricks (PySpark/Scala/SQL)
• Drive data quality, governance, and security across the data platform
• Optimize performance, cost, and scalability in Azure cloud environment
• Integrate Databricks with ADF, ADLS, and other Azure services
• Operationalize ML models with data scientists and analysts
• Define end-to-end lakehouse architecture with Delta Lake and medallion layers

Skills

Team leadership

Stakeholder collaboration

Data modeling

PySpark

Education

Bachelor's degree in CS/Engineering

DP-203 / Databricks Certified / Architect Expert

Tools

Databricks

Azure Data Factory

Azure Data Lake

Azure Synapse

Apache Kafka

Airflow

MLflow

Job description

Buchanan Ingersoll & Rooney PC is seeking a Data Platform Manager for our Pittsburgh, PA location. You will lead the design, build, and optimization of scalable enterprise data platforms on Databricks in Azure, mentoring a team of data engineers and data scientists.

The role requires deep Databricks expertise, strong Azure data services knowledge, and a leadership mindset to deliver high-performance data engineering and analytics solutions. Hybrid schedule: 3 days in office, 2 remote.