Senior AWS Data Engineer — ETL & Big Data

Cloud Analytics Technologies, LLC; Chicago (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

Cloud Analytics Technologies, LLC is seeking a Senior ETL Developer to design, build, and optimize enterprise-scale ETL pipelines within AWS. The role focuses on Python, PySpark, SQL, and data processing across S3, Redshift, Lambda, and Step Functions.

You will implement CI/CD with Harness and CloudFormation, ensure data quality, and provide production support. Ideal candidates have 6+ years in ETL development, strong SQL, and 3–4 years with Python/PySpark and AWS services.

Qualifications

• 6+ years of ETL development, data integration, transformation, and scheduling.
• 6+ years of strong SQL experience.
• 3–4 years of Python/PySpark with large-scale datasets.
• 3–4 years of AWS Cloud experience including Glue, S3, Redshift, Lambda, Step Functions, RDS Aurora/MySQL, CloudWatch, SNS, SQS, EventBridge.
• 3+ years in Big Data processing, data modeling, and scalable data architectures.
• Hands-on DevOps/CI/CD experience with Harness, YAML, AWS CloudFormation.

Responsibilities

• Design, develop, implement, and optimize enterprise-scale ETL pipelines and backend data processing solutions.
• Build and maintain scalable data pipelines using AWS Glue, Python, PySpark, and SQL.
• Develop complex SQL queries, stored procedures, and data transformation workflows.
• Work with AWS services including S3, Redshift, Lambda, Step Functions, RDS, CloudWatch, SNS, SQS, and EventBridge.
• Perform data cleansing, profiling, transformation, and workflow scheduling.

Skills

ETL development

SQL

Python

PySpark

AWS Cloud

Big Data

DevOps / CI/CD

Apache Iceberg

Scripting / data transformation

Tools

Harness

YAML

AWS CloudFormation

Dynatrace

Datadog

Job description

Cloud Analytics Technologies, LLC is seeking a Senior ETL Developer to design, build, and optimize enterprise-scale ETL pipelines within AWS. The role focuses on Python, PySpark, SQL, and data processing across S3, Redshift, Lambda, and Step Functions.

You will implement CI/CD with Harness and CloudFormation, ensure data quality, and provide production support. Ideal candidates have 6+ years in ETL development, strong SQL, and 3–4 years with Python/PySpark and AWS services.