Senior Python Data Engineer – AWS ETL & Data Pipelines

Quantum Technologies. LLC; Atlanta (GA); On-site; USD 110,000 - 160,000; Full time

Job summary

Quantum Technologies LLC in Atlanta, GA, seeks a Python Developer with 6–10 years of software development experience to design and deliver robust AWS-based data engineering solutions for analytics and data product use cases.

You will design, build, and maintain scalable ETL pipelines with Python and PySpark, orchestrate workflows using Glue, Lambda, and Step Functions, and implement data validation, monitoring, and API integrations across internal and external systems.

Qualifications

• 6-10 years of software development experience.
• 5-8+ years on the appropriate platform.
• Solid IT background.
• Experience as an application developer on projects similar in scope and responsibility.
• Strong Python, PySpark, and SQL skills.
• Hands-on experience with AWS services and data engineering.

Responsibilities

• Design, build, and maintain scalable ETL and data pipelines with Python, PySpark, and AWS.
• Develop workflows using Glue, Lambda, and Step Functions.
• Build event-driven integrations using SNS, SQS, and related AWS services.
• Design and optimize data lake/warehouse with S3, Athena, and Redshift.
• Write and tune high-performance SQL for transformations and reporting.
• Implement data validation, monitoring, logging, alerting, and error handling.
• Design and support API/JSON integrations across systems.
• Troubleshoot operational issues and perform root-cause analysis.
• Contribute to documentation, deployment automation, and best practices.
• Collaborate with stakeholders and mentor junior engineers.

Skills

Python

PySpark

SQL

Data engineering

Tools

AWS Glue

Redshift

RDS

Lambda

Step Functions

SNS

SQS

S3

Athena

EMR

CloudWatch

CloudTrail

GitLab

Terraform

CI/CD

Job description

Quantum Technologies LLC in Atlanta, GA, seeks a Python Developer with 6–10 years of software development experience to design and deliver robust AWS-based data engineering solutions for analytics and data product use cases.

You will design, build, and maintain scalable ETL pipelines with Python and PySpark, orchestrate workflows using Glue, Lambda, and Step Functions, and implement data validation, monitoring, and API integrations across internal and external systems.