Manufacturing Data Engineer

WorkGenius Group; San Mateo (CA); On-site; USD 96,000 - 152,000; Full time

Job summary

WorkGenius Group is seeking a Manufacturing Data Engineer in Foster City, CA for a ~12-month engagement. The role focuses on building and maintaining batch and streaming pipelines into Delta Lake, with Databricks, PySpark, and dbt as core technologies.

You will design medallion data models to support KPIs, dashboards, analytics, and AI use cases. The ideal candidate has 6+ years in data engineering, strong SQL/Python skills, and hands-on experience with Databricks, Unity Catalog, and CI/CD in

Qualifications

• 6+ years of data engineering experience required.
• Bachelor's degree or equivalent practical experience.
• Strong SQL and Python with production experience using PySpark.

Responsibilities

• Build and maintain batch and streaming ingestion pipelines into Delta Lake from various sources such as manufacturing databases, SAP MES/HANA, test systems, and AWS CDC sources.
• Design medallion architecture layers and trusted manufacturing data models for KPIs, dashboards, analytics, and AI use cases.
• Develop incremental dbt models using PySpark, Databricks SQL, MERGE/upsert, SCD Type 2, and late-arriving data handling.
• Improve pipeline reliability, performance, quality, and cost through orchestration, monitoring, testing, alerting, and Spark/Databricks optimization.
• Implement CI/CD, governance, documentation, security, and operational processes across Databricks, Unity Catalog, AWS, GitHub Actions, and Terraform.

Skills

Databricks

Delta Lake

PySpark

dbt

Unity Catalog

AWS DMS

Structured Streaming

Data Modeling

CI/CD

Terraform

Education

Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field

Tools

Databricks

Delta Lake

Unity Catalog

Workflows/Jobs

Spark

Job description

Title: Manufacturing Data Engineer
Industry: Autonomous Vehicle / Technology
Location: Foster City, CA
Duration: ~12 months

Responsibilities

• Build and maintain batch and streaming ingestion pipelines into Delta Lake from manufacturing databases, SAP MES/HANA, test systems, and AWS CDC sources.
• Design medallion architecture layers and trusted manufacturing data models supporting KPIs, dashboards, analytics, and AI use cases.
• Develop incremental dbt models using PySpark, Databricks SQL, MERGE/upsert patterns, SCD Type 2, and late-arriving data handling.
• Improve pipeline reliability, performance, quality, and cost through orchestration, monitoring, testing, alerting, and Spark/Databricks optimization.
• Implement CI/CD, governance, documentation, security, and operational processes across Databricks, Unity Catalog, AWS, GitHub Actions, and Terraform.

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field—or equivalent practical experience—with 6+ years of data engineering experience.
• Strong SQL and Python skills, including production experience with PySpark.
• Hands-on production experience with Databricks, Delta Lake, Unity Catalog, Workflows/jobs, and Spark performance tuning.
• Production experience with dbt, including incremental models and data-quality tests; strong understanding of dimensional modeling, medallion architecture, CDC, and incremental loading.
• Experience with Git-based CI/CD, AWS data services such as S3, IAM, DMS, or Kinesis, and infrastructure as code such as Terraform.

Skills

• Databricks
• Delta Lake
• PySpark
• dbt
• Unity Catalog
• AWS DMS
• Structured Streaming
• Data Modeling
• CI/CD
• Terraform

Hourly rate is commensurate with experience and is an estimated range provided by WorkGenius.