Data Engineer

Applied Resource Group; Atlanta (GA); On-site; USD 110,000 - 130,000; Full time

Job summary

A technology resource management firm is looking for a skilled Data Engineer with expertise in AWS Glue to design and develop scalable data pipelines in the AWS cloud. In this role, you will build and optimize ETL processes using AWS Glue and PySpark, integrate data into Amazon Redshift, and collaborate with analytics teams to ensure high data quality. The ideal candidate has strong programming skills and experience with large datasets, making it essential to have hands-on knowledge of AWS data architecture and ETL practices.

Qualifications

• 3+ years of experience in Data Engineering or similar role.
• Hands-on experience with AWS Glue for ETL development.
• Strong programming skills in Python and PySpark.

Responsibilities

• Design, develop, and maintain ETL pipelines using AWS Glue.
• Build scalable data solutions within the AWS cloud ecosystem.
• Integrate data from multiple sources into Amazon Redshift.

Skills

AWS Glue

Python

PySpark

Amazon Redshift

ETL processes

Data Engineering

Tools

AWS

Airflow

Job description

We are seeking a skilled Data Engineer with strong AWS Glue experience to support the design and development of scalable data pipelines within the Amazon Web Services (AWS) cloud environment. This role will focus on building and optimizing ETL and data transformation processes, leveraging tools such as AWS Glue, Amazon Redshift, and Python/PySpark to support enterprise analytics and data initiatives.

The ideal candidate will have a strong understanding of AWS data architecture, experience working with large datasets, and the ability to build reliable data pipelines that enable business intelligence and analytics teams.

Key Responsibilities

• Design, develop, and maintain ETL pipelines using AWS Glue and PySpark to process and transform large datasets.
• Build scalable data solutions within the AWS cloud ecosystem, ensuring performance, reliability, and efficiency.
• Integrate data from multiple sources into Amazon Redshift and other data storage solutions.
• Optimize and manage data transformation workflows for high-volume data processing.
• Develop and maintain Python-based ETL scripts and data pipelines.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
• Ensure data quality, governance, and consistency across the data platform.
• Monitor and troubleshoot AWS Glue jobs and data pipeline performance.
• Implement best practices for data modeling, partitioning, and performance optimization.
• Support continuous improvements to the data platform architecture and ETL processes.

Required Qualifications

• 3+ years of experience in Data Engineering or similar role
• Hands‑on experience with AWS Glue for ETL development
• Strong programming skills in Python and PySpark
• Experience working with Amazon Redshift
• Solid understanding of ETL processes and data transformation frameworks
• Experience designing and building data pipelines in AWS
• Familiarity with AWS architecture and services (S3, IAM, Lambda, CloudWatch, etc.)
• Experience working with structured and semi‑structured data
• Knowledge of SQL and relational databases

Preferred Qualifications

• Experience with data lake architectures
• Familiarity with Apache Spark ecosystem
• Experience with data orchestration tools (Airflow, Step Functions, etc.)
• Understanding of CI/CD pipelines for data engineering
• Experience supporting analytics or business intelligence platforms