Remote Lead Data Platform Architect

8th Light; United States; On-site; USD 155,000 - 221,000; Full time

Benefits offered by this job

Unlimited O’Reilly learning access

Annual learning stipend

Coworking support for global team

Flexible PTO

12 weeks new parent leave

Job summary

8th Light, a Chicago-based data and engineering consultancy, seeks a principal data platform engineer to lead architecture and delivery of modern data platforms. You’ll shape data strategy, drive scalable solutions, and mentor teams across client engagements.

You’ll work with Databricks, Delta Lake, Spark-based processing, and CI/CD practices to deliver high-impact data solutions for healthcare and enterprise clients, while partnering with stakeholders and guiding platform evolution.

Qualifications

• 10+ years in software with 7+ years in data engineering and leadership.
• Shaped data strategy and architecture for data platforms.
• Experience with Databricks and lakehouse architectures (Delta Lake, Spark).
• Strong SQL and Python for large-scale data pipelines.

Responsibilities

• Own architecture and end-to-end delivery of modern data platforms.
• Shape client data strategy and design scalable solutions.
• Lead implementation from design to production.
• Mentor team members and ensure high-quality delivery.
• Advise on architecture, scalability, and platform evolution.

Skills

Platform engineering

Real-time data streaming

Production agentic systems

MLOps

Infrastructure as Code

Tools

Databricks

Kafka

Kinesis

Pub/Sub

Job description

8th Light, a Chicago-based data and engineering consultancy, seeks a principal data platform engineer to lead architecture and delivery of modern data platforms. You’ll shape data strategy, drive scalable solutions, and mentor teams across client engagements.

You’ll work with Databricks, Delta Lake, Spark-based processing, and CI/CD practices to deliver high-impact data solutions for healthcare and enterprise clients, while partnering with stakeholders and guiding platform evolution.