Senior AWS Python Data Engineer - PySpark ETL

Robotic Technologies Automation; Reston (VA); Presencial; USD 120.000 - 170.000; Jornada completa

Descripción de la vacante

ROBOTIC PROCESS AUTOMATION LLC is seeking a skilled Data Engineer to design and maintain scalable ETL pipelines using Python and PySpark on AWS Glue, while orchestrating workflows with Step Functions and Lambda. The role emphasizes data quality, monitoring, and collaboration with stakeholders.

You will optimize storage in Amazon Redshift, write efficient SQL, and implement event-driven patterns with SNS/SQS, along with Git-based CI/CD and IaC practices.

Formación

• Proficient in Python and PySpark for large-scale data processing.
• Hands-on experience with AWS services: Lambda, SNS, SQS, Glue, Redshift, Step Functions.
• Solid SQL skills with data modeling and query tuning.
• Experience with ETL best practices, data quality checks, and monitoring/alerting.
• Familiarity with Git and basic DevOps/CI/CD workflows.

Responsabilidades

• Build and maintain ETL pipelines using Python and PySpark on AWS Glue and other compute platforms.
• Orchestrate workflows with AWS Step Functions and serverless components (Lambda).
• Implement messaging and event-driven patterns using AWS SNS and SQS.
• Design and optimize data storage and querying in Amazon Redshift.
• Write performant SQL for data transformations, validation, and reporting.
• Ensure data quality, monitoring, error handling and operational support for pipelines.
• Collaborate with data consumers, engineers, and stakeholders to translate requirements into solutions.
• Contribute to CI/CD, infrastructure-as-code, and documentation for reproducible deployments.

Conocimientos

Python

PySpark

SQL

Data modeling

ETL best practices

Git & CI/CD

Herramientas

AWS Lambda

AWS SNS

AWS SQS

AWS Glue

Amazon Redshift

AWS Step Functions

Descripción del empleo

ROBOTIC PROCESS AUTOMATION LLC is seeking a skilled Data Engineer to design and maintain scalable ETL pipelines using Python and PySpark on AWS Glue, while orchestrating workflows with Step Functions and Lambda. The role emphasizes data quality, monitoring, and collaboration with stakeholders.

You will optimize storage in Amazon Redshift, write efficient SQL, and implement event-driven patterns with SNS/SQS, along with Git-based CI/CD and IaC practices.