Manufacturing Data Engineer

Kforce Inc

San Mateo (CA)

Sur place

USD 150 000 - 190 000

Plein temps

Il y a 3 heures
Soyez parmi les premiers à postuler

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

Kforce is seeking a Manufacturing Data Engineer in Foster City, CA to design and maintain a medallion data architecture (bronze, silver, gold) in Delta Lake and Databricks. You will build batch and streaming ingestion, implement dbt models, and optimize queries for analytics across manufacturing data such as WOs, yield, and nonconformances.

Responsibilities include migrating Redshift models to Databricks, enforcing data quality, and collaborating with analysts and stakeholders to document

Qualifications

  • Bachelor's degree or equivalent practical experience + 6+ years in data engineering.

Responsabilités

  • Build batch and streaming ingestion into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect or Structured Streaming.
  • Design and maintain bronze, silver and gold (medallion) layers following MOPS naming conventions and data dictionary standards.
  • Build incremental dbt models on Databricks using MERGE and upsert patterns, SCD Type 2, handling late or out-of-order records.
  • Model manufacturing data for analytics (SFC, work orders, nonconformances, station results, yield) and feed the semantic layer for analysts.
  • Process large and semi-structured data (JSON payloads) using PySpark and Databricks SQL.
  • Tune pipeline performance and cost via partitioning, OPTIMIZE, VACUUM, and right-sized compute.
  • Own data quality with dbt tests, Lakeflow expectations and Lakehouse Monitoring; set freshness SLAs.
  • Orchestrate workloads with Databricks Workflows or Airflow.
  • Migrate Redshift models/pipelines to Databricks and cut over.
  • Document lineage, data contracts and table definitions in Unity Catalog with stakeholders.
  • Build and maintain CI/CD for MOPS jobs, pipelines and dbt models with Databricks Asset Bundles and GitHub Actions.

Connaissances

SQL
Python
Data modeling
ETL

Formation

Bachelor's degree in CS/IS/Engineering

Outils

Databricks
Delta Lake
Unity Catalog
Databricks Workflows
GitHub Actions
Terraform
AWS DMS

Description du poste

Responsibilities

Kforce has a client that is seeking a Manufacturing Data Engineer in Foster City, CA. Responsibilities:

  • Build batch and streaming ingestion into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect or Structured Streaming, including CDC feeds from AWS DMS and SAP SLT
  • Design and maintain bronze, silver and gold (medallion) layers that follow MOPS naming conventions and data dictionary standards
  • Build incremental dbt models on Databricks (dbt-databricks), using MERGE and upsert patterns, SCD Type 2, and handling for late or out-of-order records
  • Model manufacturing data for analytics, such as a single source of truth per unit (SFC), work orders, nonconformances, station results and first-pass yield, and feed the semantic layer used by analysts
  • Process large and semi-structured data, such as JSON test payloads, using PySpark and Databricks SQL
  • Tune pipeline performance and cost through liquid clustering or partitioning, OPTIMIZE, VACUUM, file sizing and right-sized job compute
  • Own data quality with dbt tests, Lakeflow expectations and Lakehouse Monitoring, including freshness SLAs and alerting
  • Orchestrate workloads with Databricks Workflows or Airflow
  • Migrate existing Redshift models and pipelines to Databricks: translate SQL, run in parallel, reconcile results and cut over
  • Partner with analysts and manufacturing stakeholders on requirements, and document lineage, data contracts and table definitions in Unity Catalog
  • Build and maintain CI/CD for MOPS jobs, pipelines and dbt models with Databricks Asset Bundles and GitHub Actions, across dev, staging and prod
  • Manage MOPS-owned catalogs, schemas, volumes and grants in Unity Catalog, following the central platform team's governance model
  • Configure job compute and SQL warehouses within central cluster policies; Manage MOPS service principals, secret scopes and OAuth machine-to-machine auth for Airflow and dbt
Responsibilities

Kforce has a client that is seeking a Manufacturing Data Engineer in Foster City, CA. Responsibilities:

  • Build batch and streaming ingestion into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect or Structured Streaming, including CDC feeds from AWS DMS and SAP SLT
  • Design and maintain bronze, silver and gold (medallion) layers that follow MOPS naming conventions and data dictionary standards
  • Build incremental dbt models on Databricks (dbt-databricks), using MERGE and upsert patterns, SCD Type 2, and handling for late or out-of-order records
  • Model manufacturing data for analytics, such as a single source of truth per unit (SFC), work orders, nonconformances, station results and first-pass yield, and feed the semantic layer used by analysts
  • Process large and semi-structured data, such as JSON test payloads, using PySpark and Databricks SQL
  • Tune pipeline performance and cost through liquid clustering or partitioning, OPTIMIZE, VACUUM, file sizing and right-sized job compute
  • Own data quality with dbt tests, Lakeflow expectations and Lakehouse Monitoring, including freshness SLAs and alerting
  • Orchestrate workloads with Databricks Workflows or Airflow
  • Migrate existing Redshift models and pipelines to Databricks: translate SQL, run in parallel, reconcile results and cut over
  • Partner with analysts and manufacturing stakeholders on requirements, and document lineage, data contracts and table definitions in Unity Catalog
  • Build and maintain CI/CD for MOPS jobs, pipelines and dbt models with Databricks Asset Bundles and GitHub Actions, across dev, staging and prod
  • Manage MOPS-owned catalogs, schemas, volumes and grants in Unity Catalog, following the central platform team's governance model
  • Configure job compute and SQL warehouses within central cluster policies; Manage MOPS service principals, secret scopes and OAuth machine-to-machine auth for Airflow and dbt
Requirements
  • Bachelor's degree in Computer Science, Information Systems, Engineering or a related field, or equivalent practical experience, plus 6+ years in data engineering
  • Strong SQL and Python, including PySpark
  • Hands-on production experience on Databricks: Delta Lake, Unity Catalog permissions, Workflows or jobs, and Spark performance tuning
  • Production experience with dbt, including incremental models and tests
  • Solid grasp of dimensional modeling, the medallion pattern, and CDC and incremental loading
  • Experience deploying data code through Git-based workflows and CI/CD (e.g., GitHub Actions, Databricks Asset Bundles or similar)
  • Working knowledge of AWS data services (S3, IAM, DMS or Kinesis) and infrastructure as code (e.g., Terraform)
Preferred Qualifications
  • Databricks Certified Data Engineer Associate or Professional; AWS certification (e.g., Solutions Architect Associate) a plus
  • Experience preparing data for Databricks AI/BI dashboards, Genie, or ML and LLM use cases
  • Background in manufacturing or industrial data, such as quality, yield, traceability or MES
  • Understanding of data governance and compliance as they relate to data pipelines

The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.

We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.

Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless and until paid and may be modified in its discretion consistent with the law.

This job is not eligible for bonuses, incentives or commissions.

Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Manufacturing Data Engineer
Manufacturing Data Engineer

Ursus, Inc. • Foster City (CA)

Sur place
USD 94 000 - 107 000
Medical benefits
Dental benefits
Vision benefits
+2
Data Engineer - Fleet Operations
Data Engineer - Fleet Operations

Hireart • Foster City (CA)

Sur place
USD 78 000 - 110 000
Pre‑tax commuter benefits
Healthcare benefits
FSA for healthcare costs
+2
Data Engineer - Fleet Operations
Data Engineer - Fleet Operations

HireArt, Inc. • Foster City (CA)

Sur place
USD 150 000 - 210 000
Employer subsidized healthcare
Flexible Spending Account
Disability and life insurance coverage
+1
Data Engineer [214448]
Data Engineer [214448]

Aquent Talent • Foster City (CA)

Sur place
USD 95 000 - 105 000
Data Engineer I
Data Engineer I

Kforce Inc • Whitehall Township (MI)

Sur place
USD 80 000 - 110 000
Manufacturing Data Engineer: CDC & Lakehouse
Manufacturing Data Engineer: CDC & Lakehouse

Aquent Talent • Foster City (CA)

Sur place
USD 95 000 - 105 000
Manufacturing Data Engineer
Manufacturing Data Engineer

DeWinter BH • Foster City (CA)

Sur place
USD 140 000 - 190 000
Data Engineer - Fleet Operations
Data Engineer - Fleet Operations

An Autonomous Mobility Company • Foster City (CA)

Sur place
USD 83 000 - 107 000
Pre‑tax commuter benefits
Healthcare benefits (HireArt)
Flexible Spending Account
+2
Data Engineer (Azure)
Data Engineer (Azure)

Kforce Inc • New York (NY)

Sur place
USD 120 000 - 180 000
Data Engineer
Data Engineer

Kforce Technology Staffing • Coppell (TX)

Sur place
USD 110 000 - 140 000
Medical/dental/vision insurance
HSA
FSA
+2