AWS PySpark Data Platform Engineer — Production ETL

The Briefed; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Mactores is seeking an Associate Data Engineer to consolidate and migrate client data platforms on AWS. You will work with senior engineers, data scientists, and business leads to build trusted production data products, robust pipelines, and high-performance analytical models.

Responsibilities include developing production ETL/ELT pipelines, optimizing SQL for Athena and Redshift, and ensuring data quality in an agile environment. Preferred familiarity with AWS EMR and Airflow.

Qualifications

• 1 to 3 years of professional experience with Apache Spark, PySpark, and Amazon Glue.
• 2+ years building production ETL jobs using PySpark and SparkSQL.
• 2+ years writing complex SQL queries and stored procedures.
• Deep technical understanding of the DataFrame API and Spark 2.7+ transformations.

Responsibilities

• Production pipeline development using PySpark and Glue to build scalable ETL/ELT pipelines.
• SQL engineering: write and optimize SQL queries, views, and data structures in Amazon Athena and Amazon Redshift.
• Cross-functional collaboration with engineering, analytics, and business domain leads to build data products that support decision-making.
• Data quality and reliability: establish data quality rules and schema transformations.
• Delivery commitments: work in an agile, forward-deployed model to meet project milestones.

Skills

Apache Spark

PySpark

Amazon Glue

SQL

DataFrame API

SparkSQL

Airflow

AWS EMR

Tools

AWS EMR

Apache Airflow

Job description

Mactores is seeking an Associate Data Engineer to consolidate and migrate client data platforms on AWS. You will work with senior engineers, data scientists, and business leads to build trusted production data products, robust pipelines, and high-performance analytical models.

Responsibilities include developing production ETL/ELT pipelines, optimizing SQL for Athena and Redshift, and ensuring data quality in an agile environment. Preferred familiarity with AWS EMR and Airflow.