Data Engineer - Fleet Operations

HireArt, Inc.

Foster City (CA)

On-site

USD 150,000 - 210,000

Full time

3 hours ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Employer subsidized healthcare
Flexible Spending Account
Disability and life insurance coverage
401k package

Job summary

HireArt is helping an on-demand, autonomous ride-hailing company find a Data Engineer - Fleet Operations to design, build, and maintain data pipelines and models that power its Manufacturing Intelligence data lakehouse.

In this role, you'll integrate manufacturing data from on-premises databases, SAP MES, SAP HANA, and testing and diagnostic systems into the Manufacturing Operations (MOPS) lakehouse, transforming it into reliable datasets for reporting, dashboards, and AI applications.

Qualifications

  • Bachelor's degree or equivalent practical experience in CS/IS/Engineering.
  • 6+ years of data engineering experience.
  • Strong SQL, Python, and PySpark skills.
  • Production experience with Databricks, Delta Lake, and Unity Catalog.
  • Experience with dbt, ETL/ELT design, and CI/CD tooling.
  • Knowledge of AWS data services and Terraform is a plus.

Responsibilities

  • Build and maintain batch and streaming ingestion pipelines into Delta Lake.
  • Design bronze, silver, and gold data layers per MOPS standards.
  • Develop incremental dbt models on Databricks with SCD Type 2.
  • Model manufacturing data for analytics and dashboards.
  • Optimize large datasets with PySpark, clustering, and cost controls.
  • Maintain data quality with tests, monitoring, and alerts.
  • Collaborate with analysts and operations teams and document lineage.

Skills

SQL
Python
PySpark
Databricks
Delta Lake
Unity Catalog
Workflows
GitHub Actions
CI/CD
AWS
Kinesis
Terraform

Education

Bachelor's degree in computer science / information systems / engineering

Tools

Databricks
Delta Lake
Unity Catalog
dbt
AWS data services (S3, IAM, DMS, Kinesis)

Job description

HireArt is helping an on-demand, autonomous ride-hailing company find a Data Engineer - Fleet Operations to design, build, and maintain data pipelines and models that power its Manufacturing Intelligence data lakehouse.

In this role, you'll integrate manufacturing data from on-premises databases, SAP MES, SAP HANA, and testing and diagnostic systems into the Manufacturing Operations (MOPS) lakehouse, transforming it into reliable datasets for reporting, dashboards, and AI applications. Approximately 80% of your work will focus on hands‑on data engineering, with the remaining 20% dedicated to CI/CD, AWS ingestion infrastructure, Databricks configuration, and pipeline reliability. You'll also support the migration from Redshift to Databricks while collaborating with data engineering, analytics, manufacturing operations, and platform teams.

The ideal candidate has extensive experience building production‑grade data pipelines and models, with strong expertise in Databricks, SQL, Python, and cloud‑based data infrastructure. They're detail‑oriented, proactive, and skilled at optimizing data performance, maintaining data quality, and collaborating across technical and operational teams.

As a Data Engineer - Fleet Operations, you'll:
  • Build and maintain batch and streaming ingestion pipelines into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect, or Structured Streaming, including change data capture (CDC) feeds from AWS DMS and SAP SLT.
  • Design and maintain bronze, silver, and gold data layers following MOPS naming conventions and data dictionary standards.
  • Develop incremental dbt models on Databricks using MERGE, upsert, and Slowly Changing Dimension (SCD) Type 2 patterns, including handling late or out-of-order records.
  • Model manufacturing data, including unit‑level traceability (SFC), work orders, nonconformances, station results, and first‑pass yield, to support analytics, dashboards, and semantic reporting.
  • Process large‑scale and semi‑structured datasets using PySpark and Databricks SQL, optimizing performance and cost through clustering, partitioning, OPTIMIZE, VACUUM, file sizing, and compute configuration.
  • Maintain data quality and pipeline reliability using dbt tests, Lakeflow expectations, Lakehouse Monitoring, freshness SLAs, and automated alerts.
  • Orchestrate data workloads using Databricks Workflows or Airflow and support Redshift‑to‑Databricks migration through SQL conversion, parallel validation, reconciliation, and cutover.
  • Collaborate with analysts and manufacturing teams to define data requirements and document lineage, data contracts, and table definitions in Unity Catalog.
  • Build and maintain CI/CD pipelines using Databricks Asset Bundles and GitHub Actions across development, staging, and production environments.
  • Manage MOPS‑specific Databricks configurations, including Unity Catalog permissions, compute resources, SQL warehouses, service principals, secrets, and OAuth authentication, in alignment with central platform governance.
  • Configure and troubleshoot AWS ingestion infrastructure, including DMS, Kinesis, S3, and IAM, while coordinating Terraform‑based provisioning with infrastructure teams.
  • Monitor pipeline health, performance, and Databricks Unit (DBU) costs, maintain operational documentation, and support incident response and root cause analysis.
Requirements
  • A bachelor's degree in computer science, information systems, engineering, or a related discipline, or equivalent practical experience
  • 6+ years of data engineering experience
  • Strong proficiency in SQL, Python, and PySpark
  • Hands‑on production experience with Databricks, including Delta Lake, Unity Catalog permissions, Workflows, and Spark performance tuning
  • Production experience with dbt, including incremental modeling and testing
  • Strong understanding of dimensional modeling, medallion architecture, CDC, and incremental data loading
  • Experience deploying data code through Git‑based workflows and CI/CD tools such as GitHub Actions or Databricks Asset Bundles
  • Working knowledge of AWS data services, including S3, IAM, DMS, or Kinesis, and infrastructure‑as‑code tools such as Terraform
Preferred Qualifications
  • Databricks Certified Data Engineer Associate or Professional certification; AWS certification, such as Solutions Architect Associate, is a plus
  • Experience preparing data for Databricks AI/BI dashboards, Genie, machine learning, or large language model applications
  • Experience with manufacturing or industrial data, including quality, yield, traceability, or manufacturing execution systems (MES)
  • Understanding of data governance and compliance requirements related to data pipelines
  • Employer (HireArt) subsidized healthcare benefits (eligibility begins on the first of the month following 60 days of service)
  • Flexible Spending Account for healthcare-related costs
  • HireArt covers all costs for short- and long-term disability and life insurance
  • 401k package
Commitment

This is a full-time, 12-month contract position staffed via HireArt. It will be onsite and available to candidates who are local to the Foster City, CA area.

HireArt values diversity and is an Equal Opportunity Employer. We are interested in every qualified candidate who is eligible to work in the United States. Unfortunately, we are not able to sponsor visas or employ corp-to-corp.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer - Fleet Operations
Data Engineer - Fleet Operations

Hireart • Foster City (CA)

On-site
USD 78,000 - 110,000
Pre‑tax commuter benefits
Healthcare benefits
FSA for healthcare costs
+2
Data Engineer - Fleet Operations
Data Engineer - Fleet Operations

An Autonomous Mobility Company • Foster City (CA)

On-site
USD 83,000 - 107,000
Pre‑tax commuter benefits
Healthcare benefits (HireArt)
Flexible Spending Account
+2
Manufacturing Data Engineer
Manufacturing Data Engineer

DeWinter BH • Foster City (CA)

On-site
USD 140,000 - 190,000
Data Engineer [214448]
Data Engineer [214448]

Aquent Talent • Foster City (CA)

On-site
USD 95,000 - 105,000
Manufacturing Data Engineer
Manufacturing Data Engineer

Ursus, Inc. • Foster City (CA)

On-site
USD 94,000 - 107,000
Medical benefits
Dental benefits
Vision benefits
+2
Manufacturing Data Engineer
Manufacturing Data Engineer

Kforce Inc • San Mateo (CA)

On-site
USD 150,000 - 190,000
Data Engineer
Data Engineer

Catena Operations Ltd. • Malta (MT)

On-site
USD 90,000 - 140,000
Senior Data Engineer – Databricks & AWS
Senior Data Engineer – Databricks & AWS

Robotics Technologies • Sunnyvale (CA)

On-site
USD 76,000 - 108,000
Senior Data Engineer
Senior Data Engineer

Principal Mineral Holdings • Camden (SC)

On-site
USD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

Hillwood Development Company LLC • Dallas (TX)

On-site
USD 140,000 - 180,000
Medical Insurance
Dental Insurance
Vision Insurance
+12