Data Pipeline Engineer II: ETL, Airflow & Spark

PTR Global

West Des Moines (IA)

On-site

USD 55,104 - 61,992

Part time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

A leading technology provider seeks a Software Engineer 2 in West Des Moines, Iowa. This contingent role involves designing and developing ETL/ELT workflows and optimizing data structures for performance. Candidates should have around 2 years of Software Engineering experience, strong skills in data pipelines, and proficiency in tools like Apache Airflow and PySpark. The role offers a pay range of $40 to $45 per hour.

Qualifications

  • 2 years of Software Engineering experience or equivalent.
  • Strong understanding of data structures and lifecycle management.
  • Hands-on experience designing and managing ETL/ELT data pipelines.
  • Advanced skills in distributed data processing with PySpark.
  • Knowledge of Hadoop Ecosystem components.
  • Programming expertise in Python, Spark, and SQL.

Responsibilities

  • Design and develop ETL/ELT workflows for data processing.
  • Build and maintain data pipelines using open-source frameworks.
  • Implement operational data stores leveraging Delta Lake.
  • Collaborate with teams to align with architecture.
  • Troubleshoot data pipeline issues and ensure reliability.

Skills

Software Engineering experience
Data structures understanding
ETL/ELT data pipelines
Distributed data processing with PySpark
Cloud platforms (GCP)
Apache Airflow
Python

Education

2 years of relevant experience

Tools

Apache Airflow
PySpark
SQL
NetApp Iceberg
Hadoop Ecosystem (HDFS, Hive)

Job description

A leading technology provider seeks a Software Engineer 2 in West Des Moines, Iowa. This contingent role involves designing and developing ETL/ELT workflows and optimizing data structures for performance. Candidates should have around 2 years of Software Engineering experience, strong skills in data pipelines, and proficiency in tools like Apache Airflow and PySpark. The role offers a pay range of $40 to $45 per hour.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Enterprise Data Engineering - Software Engineer II - P2
Enterprise Data Engineering - Software Engineer II - P2

Credit One Bank • Las Vegas (NV)

On-site
USD 120,000 - 180,000
Senior Data Engineer: Real-Time Pipelines & Spark
Senior Data Engineer: Real-Time Pipelines & Spark

InfoVision Inc. • Denver (CO)

On-site
USD 110,000 - 126,000
Senior Data Engineer - Scalable Pipelines & Spark Expert
Senior Data Engineer - Scalable Pipelines & Spark Expert

Ethereum Technologies LLC • Charlotte (NC)

On-site
USD 120,000 - 180,000
Cloud Data Engineer — ETL Pipelines, Python & Spark
Cloud Data Engineer — ETL Pipelines, Python & Spark

Dexian • Dearborn (MI)

On-site
USD 90,000 - 120,000
Senior Dataflow Engineer - Spark, Airflow, Big Data
Senior Dataflow Engineer - Spark, Airflow, Big Data

Erias Ventures, LLC. • Maryland

On-site
USD 229,000 - 251,000
Above Market Pay
401k Immediate Vesting
Spot Bonuses
+10
Senior Data Engineer: Pipelines, Lakes & Spark/Airflow
Senior Data Engineer: Pipelines, Lakes & Spark/Airflow

Ethereum Technologies • Charlotte (NC)

On-site
USD 89,000 - 126,000
Senior Data Engineer – ETL/ELT & Pipelines
Senior Data Engineer – ETL/ELT & Pipelines

itD Tech • Clayton (GA)

On-site
USD 100,000 - 167,000
Medical benefits
401k plan
Paid holidays
Data Engineer II: Build Pipelines & Data Solutions
Data Engineer II: Build Pipelines & Data Solutions

Fremont Economic Development Corporation • Colorado

Hybrid
USD 103,500 - 126,500
Non-Commissioned Bonus Plans or Variable Commission
401(k) plan with employer match
Employee Assistance Program
+2
Data Engineer: API-Driven Pipelines & Data Lakes
Data Engineer: API-Driven Pipelines & Data Lakes

Cloud7Works • United States

On-site
USD 100,000 - 130,000
Data Engineer - ETL, Spark & Airflow on AWS
Data Engineer - ETL, Spark & Airflow on AWS

TP-Link Systems Inc. • Irvine (CA)

On-site
USD 100,000 - 120,000
Medical insurance
401k contributions
PTO
+3