Lead Data Engineer & Architect — Lakehouse & ML Pipelines

IO Datasphere, Inc.; Lansing (MI); On-site; USD 90,000 - 130,000; Full time

Job summary

IO Datasphere, Inc. is seeking a senior data engineer in Lansing, MI to lead data modeling, Lakehouse architecture, and ML product delivery. The role requires deep experience with Spark, Kafka, Kubernetes, SageMaker, DBT, Docker, and SQL, plus strong visualization skills with Power BI/Tableau.

You will collaborate with Michigan DHHS and cross-functional teams in an agile DevOps environment. You will design and run data pipelines, ensure governance and security, and mentor junior engineers while

Qualifications

• 4+ years hands-on experience hydrating a Data Lake on AWS, Azure or GCP with ETL/ELT pipelines.
• 4+ years transforming datasets in a Data Lake to deliver data products with traceable attributes.
• 2+ years enforcing Data Sharing Policies for data products at subscription time.
• Experience with Spark, Kafka, Kubernetes, SageMaker, DBT, Docker, SQL.
• Experience with Power BI/Tableau for visualization.
• BS in Computer Science / Data Science / Information Systems with 4+ years overall.

Responsibilities

• Lead data engineering efforts with Lakehouse and ML product delivery in an agile DevOps setup.
• Engage with the State of Michigan proof-of-concept team to promote best practices.
• Oversee design, development, and maintenance of data pipelines for analytics and operations.
• Architect and optimize data infrastructure including storage and ETL/ELT processes.
• Ensure data quality, reliability, and security with governance practices.
• Mentor data engineers and collaborate with data scientists and product teams.

Skills

Data modeling

Lakehouse architecture

Distributed data

Spark

Kafka

Kubernetes

SageMaker

DBT

Python

SQL

Power BI

Tableau

DevOps

Data governance

Data security

Education

BS in Computer Science / Data Science / Information Systems

Tools

DataBricks

SageMaker

GitHub

Jenkins

Terraform

Docker

SQL

Spark

Kubernetes

Job description

IO Datasphere, Inc. is seeking a senior data engineer in Lansing, MI to lead data modeling, Lakehouse architecture, and ML product delivery. The role requires deep experience with Spark, Kafka, Kubernetes, SageMaker, DBT, Docker, and SQL, plus strong visualization skills with Power BI/Tableau.

You will collaborate with Michigan DHHS and cross-functional teams in an agile DevOps environment. You will design and run data pipelines, ensure governance and security, and mentor junior engineers while