Databricks Tech Lead: Data Platform Architect

Ender-IT; Chicago (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

A leading staffing and recruiting firm seeks a Databricks Technical Lead in Chicago, IL. This role requires extensive experience with Databricks and Spark, guiding the design and implementation of data engineering solutions. Responsibilities include leading the technical direction for data pipelines, mentoring data engineers, and ensuring best practices in data transformation and architecture. Ideal candidates should have strong communication skills and a proven track record in managing complex data platforms, as well as deep knowledge of AWS data services.

Qualifications

• 5+ years Databricks experience at a senior/lead level.
• Strong Spark skills, not just SQL users.
• Deep understanding of Delta Lake and medallion architecture.

Responsibilities

• Lead the technical direction for Databricks-based data pipelines.
• Design and review data ingestion and transformation patterns.
• Provide technical oversight across engineering squads.

Skills

Hands-on Spark experience

Expertise with Databricks

Strong understanding of data engineering patterns

Strong communication skills

Tools

AWS data services

Delta Lake

Spark (PySpark + Spark SQL)

Job description

A leading staffing and recruiting firm seeks a Databricks Technical Lead in Chicago, IL. This role requires extensive experience with Databricks and Spark, guiding the design and implementation of data engineering solutions. Responsibilities include leading the technical direction for data pipelines, mentoring data engineers, and ensuring best practices in data transformation and architecture. Ideal candidates should have strong communication skills and a proven track record in managing complex data platforms, as well as deep knowledge of AWS data services.