Senior AWS Data Engineer — Python, PySpark & ETL

Cloud Analytics Technologies, LLC; Reston (VA); On-site; USD 110,000 - 170,000; Full time

Job summary

Cloud Analytics Technologies, LLC in Reston, Virginia, is seeking a data engineer to build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms. You will orchestrate workflows with AWS Step Functions and serverless components, implement event-driven patterns with SNS and SQS, and design storage and queries in Redshift.

You will write performant SQL, ensure data quality and monitoring, collaborate with data consumers and engineers, and contribute to CI/CD

Qualifications

• Strong experience with Python and PySpark for large-scale data processing.
• Hands-on experience with AWS services: Lambda, SNS, SQS, Glue, Redshift, Step Functions.
• Solid SQL skills and familiarity with data modeling and query optimization.
• Experience with ETL best practices, data quality checks, and monitoring/alerting.
• Familiarity with version control (Git) and basic DevOps/CI-CD workflows.

Responsibilities

• Build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms
• Orchestrate workflows with AWS Step Functions and serverless components ( Lambda )
• Implement messaging and event-driven patterns using AWS SNS and SQS
• Design and optimize data storage and querying in Amazon Redshift
• Write performant SQL for data transformations, validation, and reporting
• Ensure data quality, monitoring, error handling and operational support for pipelines
• Collaborate with data consumers, engineers, and stakeholders to translate requirements into solutions
• Contribute to CI/CD , infrastructure-as-code, and documentation for reproducible deployments

Skills

Python

PySpark

AWS Lambda

AWS SNS

AWS SQS

AWS Glue

Redshift

SQL

CI/CD

Data quality

Tools

AWS Glue

Redshift

Lambda

Step Functions

SQS

SNS

Git

Job description

Cloud Analytics Technologies, LLC in Reston, Virginia, is seeking a data engineer to build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms. You will orchestrate workflows with AWS Step Functions and serverless components, implement event-driven patterns with SNS and SQS, and design storage and queries in Redshift.

You will write performant SQL, ensure data quality and monitoring, collaborate with data consumers and engineers, and contribute to CI/CD