Senior Data Engineer – ETL/ELT & Pipelines

itD Tech

Clayton (GA)

On-site

USD 100,000 - 167,000

Full time

12 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical benefits
401k plan
Paid holidays

Job summary

itD is seeking a Data Engineer III to design, build, and optimize scalable ETL/ELT pipelines that power workforce analytics and reporting. You will work across Hive/Spark data warehouses and manage orchestration using Dataswarm or Airflow, ensuring data quality and reliable workflows.

The role emphasizes backfills, data modeling, and collaboration with data scientists and analysts to translate business needs into robust data solutions.

Qualifications

  • Strong proficiency in advanced SQL including Presto/Trino/Spark SQL.
  • Basic to intermediate Python programming for production ETL/ELT development.
  • Experience building, optimizing, maintaining large-scale data pipelines across Hive/Spark environments.
  • Strong understanding of data warehousing fundamentals: partitioning, incremental loads, data modeling, query performance.
  • Experience with pipeline orchestration tools such as Dataswarm, Airflow, or equivalent.
  • Experience troubleshooting pipeline failures, backfills, data quality checks, reliable workflows.
  • Understanding of data pipeline monitoring, reliability, and SLA requirements.

Responsibilities

  • Design, develop, and maintain scalable ETL/ELT data pipelines using SQL and Python.
  • Build, optimize, and maintain data workflows across Hive/Spark environments.
  • Develop and manage orchestration workflows using Dataswarm, Airflow, or equivalent.
  • Create analytics-ready datasets supporting workforce measurement and reporting.
  • Monitor pipeline health, implement data quality checks, and maintain SLAs.
  • Automate legacy pipelines and develop self-service datasets.

Skills

SQL
Python
Hive/Spark
Data Warehousing
Airflow
Dataswarm
Backfills

Education

Bachelors in CS/CE

Tools

Datacraft
Airflow

Job description

itD is seeking a Data Engineer III to design, build, and optimize scalable ETL/ELT pipelines that power workforce analytics and reporting. You will work across Hive/Spark data warehouses and manage orchestration using Dataswarm or Airflow, ensuring data quality and reliable workflows.

The role emphasizes backfills, data modeling, and collaboration with data scientists and analysts to translate business needs into robust data solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer III - Scalable ETL Pipelines (6mo)
Data Engineer III - Scalable ETL Pipelines (6mo)

ITD Technology Co. • Menlo Park (CA)

On-site
USD 140,000 - 180,000
Senior Data Engineer — Scalable ETL/ELT Pipelines
Senior Data Engineer — Scalable ETL/ELT Pipelines

itD • Menlo Park (CA)

On-site
USD 140,000 - 170,000
Medical benefits
401k plan
Paid holidays
Data Engineer III: Spark/Hive Data Pipelines for Workforce
Data Engineer III: Spark/Hive Data Pipelines for Workforce

itd • United States

Remote
USD 140,000 - 180,000
Onsite six-month assignment in Menlo —
Medical benefits
Retirement plan
+2
Data Engineer III-6377
Data Engineer III-6377

itD Tech • Menlo Park (CA)

On-site
USD 140,000 - 180,000
Data Engineer III-6377
Data Engineer III-6377

itd • United States

Remote
USD 140,000 - 180,000
Onsite six-month assignment in Menlo —
Medical benefits
Retirement plan
+2
Data Engineer III: ETL Pipelines, Cloud & Automation
Data Engineer III: ETL Pipelines, Cloud & Automation

Bow Wave LLC • Arlington (VA)

On-site
USD 90,000 - 140,000
Senior Data Engineer: Lead Scalable ETL & AI Pipelines
Senior Data Engineer: Lead Scalable ETL & AI Pipelines

Iscale Solutions • United States

Remote
USD 120,000 - 180,000
Salary package
Vacation and sick leave
Health coverage
+6
Senior Data Engineer - Scalable Pipelines & Spark Expert
Senior Data Engineer - Scalable Pipelines & Spark Expert

Ethereum Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 180,000
Remote Data Engineer — ETL, Data Warehousing, Pipelines
Remote Data Engineer — ETL, Data Warehousing, Pipelines

ITD Technology Co. • San Jose (CA)

Remote
USD 120,000 - 160,000
Medical benefits
401k plan
Paid holidays
Senior Data Engineer: Scalable Cloud Pipelines
Senior Data Engineer: Scalable Cloud Pipelines

Compunnel Inc. • San Francisco (CA)

On-site
USD 100,000 - 130,000