Senior Python Data Engineer - AWS ETL & Data Pipelines

Valiant Technologies LLC; Atlanta (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Valiant Technologies LLC in Atlanta is seeking a senior Python Developer to design and deliver robust AWS-based data engineering solutions for analytics and data product use cases.

The role focuses on building scalable ETL pipelines with Python and PySpark, managing workflows with Glue, Lambda, and Step Functions, and optimizing data lakes and warehouses using S3, Athena, and Redshift. Strong SQL skills and cloud tooling are essential.

Qualifications

• 6–10 years of software development experience.
• 5–8+ years on the appropriate platform.
• Experience as an application developer on projects similar in scope and responsibility.
• Strong expertise in Python, PySpark, and SQL.
• Strong hands-on experience with AWS services such as Glue, Redshift, RDS, Lambda, Step Functions, SNS, SQS, S3, Athena, EMR, CloudWatch, and CloudTrail.
• Experience with enterprise data lakes, data warehouses, data marts, and big data environments.
• Strong understanding of ETL best practices, data quality, and operational support.
• Experience with APIs, JSON, and cloud-based data integration patterns.
• Familiarity with GitLab, Terraform, CI/CD, and AWS developer tools.

Responsibilities

• Design, build, and maintain scalable ETL and data pipelines using Python, PySpark, and AWS services.
• Develop and manage workflows using Glue, Lambda, and Step Functions.
• Build event-driven integrations using SNS, SQS, and related AWS services.
• Design and optimize data lake and warehouse solutions using S3, Athena, and Redshift.
• Write and tune high-performance SQL for complex transformations and reporting.
• Design and implement data validation, monitoring, logging, alerting, and error-handling frameworks.
• Design and support API-based and JSON-based integrations across internal and external systems.
• Troubleshoot operational issues, identify root causes, and implement corrective actions.
• Contribute to documentation, deployment automation, and engineering best practices.
• Collaborate with stakeholders and mentor junior team members where needed.

Skills

Python

PySpark

SQL

AWS

Glue

Redshift

RDS

Lambda

Step Functions

S3

Athena

EMR

CloudWatch

CloudTrail

GitLab

Terraform

CI/CD

AWS developer tools

Tools

Glue

Lambda

Step Functions

SNS

SQS

S3

Athena

Redshift

EMR

CloudWatch

CloudTrail

GitLab

Terraform

CI/CD

AWS developer tools

Job description

Valiant Technologies LLC in Atlanta is seeking a senior Python Developer to design and deliver robust AWS-based data engineering solutions for analytics and data product use cases.

The role focuses on building scalable ETL pipelines with Python and PySpark, managing workflows with Glue, Lambda, and Step Functions, and optimizing data lakes and warehouses using S3, Athena, and Redshift. Strong SQL skills and cloud tooling are essential.