AI-Driven Data Foundations Platform Lead

Chamberlain Group; Chicago (IL); On-site; USD 130,000 - 227,000; Full time

Benefits offered by this job

Comprehensive benefits package

401k contribution

Job summary

Chamberlain Group seeks a data engineering leader to manage the delivery and operation of data pipelines, analytics engineering, and Databricks platform capabilities within the Data and Analytics Foundations group.

You will lead a blended team of data and analytics engineers and external partners to publish audited data products, ensure data quality, and deliver AI-assisted development across the lakehouse platform.

Qualifications

• Bachelor's degree in CS or equivalent.
• 5+ years in data or analytics engineering.
• 2+ years hands‑on Databricks delivery.
• Experience leading technical teams (people or tech lead).
• Experience working with external delivery partners or offshore teams.
• Experience using AI development tools (Claude, Copilot, Databricks Genie Code).
• Ability to translate priorities with stakeholders into scoped deliverables.

Responsibilities

• Design, deliver, and operate scalable batch and streaming pipelines and data products.
• Lead a blended team of engineers and external partners to deliver quality results.
• Own Databricks platform operations, Unity Catalog, and cost optimization.
• Set data governance, masking, and access control standards with Data Governance and Security.
• Ensure quality, semantics, and dashboards across the data estate.

Skills

Leadership and people management

Data engineering

Analytics engineering

AI tooling awareness

Strong communication

Mentoring and coaching

Education

Bachelor's Degree in Computer Science or equivalent

Tools

Databricks

Unity Catalog

PySpark

Spark SQL

Delta Lake

Git / CI/CD tooling

Job description

Chamberlain Group seeks a data engineering leader to manage the delivery and operation of data pipelines, analytics engineering, and Databricks platform capabilities within the Data and Analytics Foundations group.

You will lead a blended team of data and analytics engineers and external partners to publish audited data products, ensure data quality, and deliver AI-assisted development across the lakehouse platform.