Azure Databricks Data Engineering Lead

5 Star Global Recruitment Partners; United States; Remote; USD 150,000 - 180,000; Full time

Job summary

Starbucks is seeking a senior data engineer to drive the WFM Data Services platform, bringing 7+ years in Databricks, Azure, PySpark and Python. You will lead cross-functional teams, shape the technical roadmap, and deliver automated, observable data pipelines and dashboards for business insights.

You will collaborate with executives and product teams to translate strategic needs into scalable technical solutions and deliver reliable data infrastructure at scale.

Qualifications

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.
• Experience leading large-scale, cross-functional projects to build the technical roadmap for data platforms.
• Ability to review and critique design artifacts and drive technical decisions.
• Experience building automation and monitoring frameworks with KPI visibility.

Responsibilities

• Lead large-scale, cross-functional projects to define the technical roadmap for the WFM Data Services platform.
• Lead and review design artifacts to ensure quality and alignment with business needs.
• Build and own automation and monitoring frameworks for reliable data pipelines and KPIs.
• Execute PoC on new technologies to select best-fit tools and solutions.
• Collaborate with business partners to identify opportunities and drive resolution.
• Communicate status to senior leadership and stakeholders.

Skills

Databricks

Microsoft Azure

PySpark

Python

Job description

Job Description

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.

• Lead large-scale, complex, cross-functional projects to build the technical roadmap for the WFM Data Services platform.

• Lead and review design artifacts.

• Build and own the automation and monitoring frameworks that showcase reliable, accurate, easy-to-understand metrics and operational KPIs to stakeholders for data pipeline quality.

• Execute proof of concept on new technology and tools to pick the best tools and solutions.

• Support business objectives by collaborating with business partners to identify opportunities and drive resolution.

• Communicate status and issues to Senior Starbucks leadership and stakeholders.

• Direct project team and cross-functional teams on all technical aspects of the projects.

• Lead with engineering team to build and support real-time, highly available data, data pipeline and technology capabilities.

• Translate strategic requirements into business requirements to ensure solutions meet business needs.

• Define and implement data retention policies and procedures.

• Define and implement data governance policies and procedures.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.

• Enable team to pursue insights and applied breakthroughs, while also driving the solutions to Starbucks scale.

• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of structured and unstructured data sources using big data technologies.

• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.

• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

• Perform root cause analysis to identify permanent resolutions to software or business process issues.

MUST HAVE:

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.