Lead Data Platform Engineer

Forbes Technical Consulting; Chicago (IL); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Forbes Technical Consulting in Chicago, IL is seeking a Lead Data Platform Engineer to own and evolve a modern enterprise data platform built on Databricks Lakehouse. This role blends hands-on technical leadership with platform strategy, governance, and engineering to enable analytics, AI, and data engineering teams across the organization.

You'll drive design of scalable batch and streaming pipelines, establish CI/CD and governance practices, and collaborate with security, governance, and

Qualifications

• 7+ years of experience in data engineering, platform engineering, or data architecture.
• Experience building enterprise data platforms, preferably using Databricks Lakehouse.
• Strong knowledge of Databricks, Unity Catalog, SQL, and dbt.
• Experience developing data ingestion, transformation, and delivery pipelines.
• Understanding of platform reliability, observability, and cloud cost optimization.
• Exposure to AI/ML workflows, MLOps, or model lifecycle management is desirable.
• Experience working in regulated environments such as healthcare, financial services, or other highly governed industries is a plus.

Responsibilities

• Own the enterprise Databricks data platform, including architecture, engineering standards, and day-to-day operations.
• Lead the design and implementation of scalable batch and streaming data pipelines.
• Develop and maintain the technical roadmap, engineering backlog, and platform standards.
• Build reusable data products and self-service analytics capabilities.
• Partner with engineering, security, governance, and business teams to onboard new data sources and enable enterprise analytics.
• Drive platform reliability, performance tuning, monitoring, observability, and cloud cost optimization.
• Implement governance controls around data quality, metadata, lineage, access management, and security.
• Support MLOps capabilities and help enable responsible AI adoption across the organization.
• Establish best practices for CI/CD, code quality, testing, documentation, and platform operations.

Skills

Data Engineering

Data Platform Engineering

Data Architecture

Databricks Lakehouse

SQL

dbt

Data pipelines

Observability

MLOps

Tools

Databricks

Unity Catalog

Job description

Location: Chicago, IL (Hybrid – Monday, Tuesday & Wednesday onsite)

We're partnering with a well-established national organization to identify a Lead Data Platform Engineer to own and evolve a modern enterprise data platform built on Databricks Lakehouse.

This is an excellent opportunity for someone who enjoys combining hands‑on technical leadership with platform strategy, architecture, governance, and engineering. You'll help shape the future of the organization's enterprise data ecosystem while enabling analytics, AI, and data engineering teams to build secure, scalable, and reliable data solutions.

What You'll Do

• Own the enterprise Databricks data platform, including architecture, engineering standards, and day‑to‑day operations.
• Lead the design and implementation of scalable batch and streaming data pipelines.
• Develop and maintain the technical roadmap, engineering backlog, and platform standards.
• Build reusable data products and self‑service analytics capabilities.
• Partner with engineering, security, governance, and business teams to onboard new data sources and enable enterprise analytics.
• Drive platform reliability, performance tuning, monitoring, observability, and cloud cost optimization.
• Implement governance controls around data quality, metadata, lineage, access management, and security.
• Support MLOps capabilities and help enable responsible AI adoption across the organization.
• Establish best practices for CI/CD, code quality, testing, documentation, and platform operations.

What We're Looking For

• 7+ years of experience in Data Engineering, Data Platform Engineering, or Data Architecture
• Experience designing, building, and supporting enterprise data platforms, preferably using Databricks Lakehouse
• Strong experience with:
• Databricks
• Unity Catalog
• SQL
• dbt (or similar transformation frameworks)
• Experience building data ingestion, transformation, and delivery pipelines
• Strong understanding of platform reliability, observability, performance tuning, and cloud cost optimization
• Experience implementing data governance, metadata management, data quality, access controls, and lineage
• Exposure to AI/ML workflows, MLOps, or model lifecycle management is highly desirable
• Experience working in regulated environments such as healthcare, financial services, or other highly governed industries is a plus

Why Consider This Opportunity?

• Direct-hire position with a stable, mission‑driven organization
• Opportunity to own and influence a modern enterprise data platform
• High visibility across engineering, architecture, AI, security, and business teams
• Hybrid work environment in downtown Chicago
• Competitive compensation, bonus potential, and comprehensive benefits