PySpark & AWS Data Engineer - Forward Deployed

The Briefed; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Mactores is seeking an Associate Data Engineer to consolidate and migrate client data platforms on AWS. You will work with senior engineers, data scientists, and business leads to build trusted production data products, robust pipelines, and high-performance analytical models.

Responsibilities include developing production ETL/ELT pipelines, optimizing SQL for Athena and Redshift, and ensuring data quality in an agile environment. Preferred familiarity with AWS EMR and Airflow.

Qualifications

• 1 to 3 years of professional experience with Apache Spark, PySpark, and Amazon Glue.
• 2+ years building production ETL jobs using PySpark and SparkSQL.
• 2+ years writing complex SQL queries and stored procedures.
• Deep technical understanding of the DataFrame API and Spark 2.7+ transformations.

Responsibilities

• Production pipeline development using PySpark and Glue to build scalable ETL/ELT pipelines.
• SQL engineering: write and optimize SQL queries, views, and data structures in Amazon Athena and Amazon Redshift.
• Cross-functional collaboration with engineering, analytics, and business domain leads to build data products that support decision-making.
• Data quality and reliability: establish data quality rules and schema transformations.
• Delivery commitments: work in an agile, forward-deployed model to meet project milestones.

Skills

Apache Spark

PySpark

Amazon Glue

SQL

DataFrame API

SparkSQL

Airflow

AWS EMR

Tools

AWS EMR

Apache Airflow

Job description

Mactores is the agent-native AWS modernization firm built to ship production-ready systems, retire legacy infrastructure, and measure real outcomes. Delivery at Mactores is built on Aedeon—an agent platform that automates up to 70% of repetitive discovery, dependency mapping, schema mapping, and test validation work—allowing engineers to focus on architecture, judgment, data modeling, and cutovers.

As an Associate Data Engineer in the Data Platform Modernization pillar, you will consolidate and migrate client data platforms on AWS in weeks, not quarters. Working alongside senior forward-deployed engineers, business leads, analysts, and data scientists, you will construct trusted production data products, robust pipelines, and high-performance analytical models.

Key Responsibilities

• Production Pipeline Development: Write efficient PySpark and Amazon Glue code to build, deploy, and maintain scalable ETL/ELT pipelines in production environments.
• SQL Engineering: Write and optimize SQL queries, views, and data structures in Amazon Athena and Amazon Redshift.
• Cross-Functional Collaboration: Partner with cross-functional engineering, analytics, and business domain leads to build data products that support active business decision-making.
• Data Quality & Reliability: Establish data quality, validation rules, and schema transformations using automated platform harnesses.
• Delivery Commitments: Work in an agile, forward-deployed model to meet project delivery dates and customer migration milestones.

Requirements

Experience & Technical Skills

• 1 to 3 years of professional experience working with Apache Spark, PySpark, and Amazon Glue.
• 2+ years of hands-on experience building production ETL jobs using PySpark and SparkSQL.
• 2+ years of experience writing complex SQL queries and stored procedures.
• Deep technical understanding of the DataFrame API and transformation functions supported by Spark 2.7+.

Mindset & Operating Style

• Curiosity and eagerness to quickly master new technologies and apply them directly to customer problems.
• Strong problem-solving abilities with a commitment to writing clean, production-grade code.
• Comfort embedding directly within client ecosystems and collaborating alongside senior engineers.

Preferred Qualifications

• Prior hands-on experience with AWS EMR and workflow orchestration platforms like Apache Airflow.
• Active AWS or Big Data Certifications (e.g., AWS Certified Data Engineer / Big Data Specialty, Cloudera Certified Big Data Engineer, or Hortonworks Certified Big Data Engineer).
• Practical understanding of DataOps Engineering principles, continuous integration, and automated deployments.