Data Engineer III: Spark/Hive Data Pipelines for Workforce

itd

United States

Remote

USD 140,000 - 180,000

Full time

11 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Onsite six-month assignment in Menlo —
Medical benefits
Retirement plan
Paid holidays
Direct W-2 employment; no visa sponsor

Job summary

itd is seeking a data engineer to build and maintain analytics-ready datasets that help large operational teams measure workforce needs and productivity. You will develop ETL/ELT pipelines with SQL and Python, and maintain Hive and Spark-based workflows to support reporting and decision making.

You will collaborate with analysts, data scientists, and operations to translate requirements into scalable data solutions, ensure data quality, and implement improvements to legacy systems.

Qualifications

  • Degree in a technical field required.
  • Strong SQL skills with experience in Presto, Trino or Spark SQL.
  • Python for production data workflows.
  • Experience building large-scale pipelines in Hive or Spark.
  • Knowledge of data warehousing, partitioning, and query optimization.
  • Experience with orchestration tools like Airflow or Dataswarm.
  • Ability to troubleshoot pipeline failures and implement data quality checks.

Responsibilities

  • Develop and tune ETL/ELT pipelines using SQL and Python for analytics and reporting.
  • Maintain workflows in Hive and Spark data warehouse environments, including scheduling, backfills, and failure recovery.
  • Create datasets and data models for workforce and operational measurement.
  • Monitor pipeline health, add data quality checks, and support service-level targets.
  • Improve legacy workflows, automate manual tasks, and provide self-service data assets.
  • Work with analysts, data scientists, architects, and operational partners to translate needs into data solutions.

Skills

SQL
Python
Pipeline troubleshooting
Data modeling
Data warehousing
Airflow
Spark/Hive

Education

Bachelor's degree in CS/CE or related field

Tools

Presto/Trino
Spark SQL
Hive
Airflow
Dataswarm

Job description

itd is seeking a data engineer to build and maintain analytics-ready datasets that help large operational teams measure workforce needs and productivity. You will develop ETL/ELT pipelines with SQL and Python, and maintain Hive and Spark-based workflows to support reporting and decision making.

You will collaborate with analysts, data scientists, and operations to translate requirements into scalable data solutions, ensure data quality, and implement improvements to legacy systems.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer – ETL/ELT & Pipelines
Senior Data Engineer – ETL/ELT & Pipelines

itD Tech • Clayton (GA)

On-site
USD 100,000 - 167,000
Medical benefits
401k plan
Paid holidays
Data Engineer III - Scalable ETL Pipelines (6mo)
Data Engineer III - Scalable ETL Pipelines (6mo)

ITD Technology Co. • Menlo Park (CA)

On-site
USD 140,000 - 180,000
Senior Data Engineer — Scalable ETL/ELT Pipelines
Senior Data Engineer — Scalable ETL/ELT Pipelines

itD • Menlo Park (CA)

On-site
USD 140,000 - 170,000
Medical benefits
401k plan
Paid holidays
Data Engineer III-6377
Data Engineer III-6377

itd • United States

Remote
USD 140,000 - 180,000
Onsite six-month assignment in Menlo —
Medical benefits
Retirement plan
+2
Data Engineer III-6377
Data Engineer III-6377

itD Tech • Menlo Park (CA)

On-site
USD 140,000 - 180,000
Data Engineer: Scalable Pipelines with Spark & Hive
Data Engineer: Scalable Pipelines with Spark & Hive

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 125,000 - 140,000
Data Engineer: Cloudera, Spark & Hive Pipelines
Data Engineer: Cloudera, Spark & Hive Pipelines

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 125,000 - 145,000
Cloud Data Engineer: Spark & Hive Data Pipelines
Cloud Data Engineer: Spark & Hive Data Pipelines

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 110,000
Data Engineer: Build Scalable Pipelines & Insights
Data Engineer: Build Scalable Pipelines & Insights

Compunnel Inc. • Orlando (FL)

On-site
USD 85,000 - 110,000
Data Engineer - III
Data Engineer - III

Compunnel Inc. • San Francisco (CA)

On-site
USD 100,000 - 130,000