Senior Manufacturing Data Engineer - Delta Lake & Databricks

WorkGenius Group; San Mateo (CA); On-site; USD 96,000 - 152,000; Full time

Job summary

WorkGenius Group is seeking a Manufacturing Data Engineer in Foster City, CA for a ~12-month engagement. The role focuses on building and maintaining batch and streaming pipelines into Delta Lake, with Databricks, PySpark, and dbt as core technologies.

You will design medallion data models to support KPIs, dashboards, analytics, and AI use cases. The ideal candidate has 6+ years in data engineering, strong SQL/Python skills, and hands-on experience with Databricks, Unity Catalog, and CI/CD in

Qualifications

• 6+ years of data engineering experience required.
• Bachelor's degree or equivalent practical experience.
• Strong SQL and Python with production experience using PySpark.

Responsibilities

• Build and maintain batch and streaming ingestion pipelines into Delta Lake from various sources such as manufacturing databases, SAP MES/HANA, test systems, and AWS CDC sources.
• Design medallion architecture layers and trusted manufacturing data models for KPIs, dashboards, analytics, and AI use cases.
• Develop incremental dbt models using PySpark, Databricks SQL, MERGE/upsert, SCD Type 2, and late-arriving data handling.
• Improve pipeline reliability, performance, quality, and cost through orchestration, monitoring, testing, alerting, and Spark/Databricks optimization.
• Implement CI/CD, governance, documentation, security, and operational processes across Databricks, Unity Catalog, AWS, GitHub Actions, and Terraform.

Skills

Databricks

Delta Lake

PySpark

dbt

Unity Catalog

AWS DMS

Structured Streaming

Data Modeling

CI/CD

Terraform

Education

Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field

Tools

Databricks

Delta Lake

Unity Catalog

Workflows/Jobs

Spark

Job description

WorkGenius Group is seeking a Manufacturing Data Engineer in Foster City, CA for a ~12-month engagement. The role focuses on building and maintaining batch and streaming pipelines into Delta Lake, with Databricks, PySpark, and dbt as core technologies.

You will design medallion data models to support KPIs, dashboards, analytics, and AI use cases. The ideal candidate has 6+ years in data engineering, strong SQL/Python skills, and hands-on experience with Databricks, Unity Catalog, and CI/CD in