Senior AWS Data Engineer: Python, Spark & Redshift

Visionary Innovative Technology Solutions LLC; Atlanta (GA); On-site; USD 140,000 - 220,000; Full time

Job summary

Visionary Innovative Technology Solutions LLC is seeking an AWS Data Engineer to design and implement scalable data pipelines using Hadoop/EMR, Spark, and Hive within a modern cloud architecture on AWS. The role focuses on building robust data platforms, APIs, and analytics capabilities for decision support.

The candidate will work with Lambda, EventBridge, Step Functions, Redshift, S3, and Glue to deliver end-to-end data solutions, while collaborating across teams in a matrix organization.

Qualifications

• 10+ years hands-on Python development for big data applications.
• 5+ years development in AWS services such as EMR, Lambda, EventBridge, Step Functions, Redshift, S3, Glue.
• Hands-on API calls, SQL, data manipulation/transformation, analysis and visualization.
• Experience with CI/CD pipelines using Gitlab and Terraform.
• Strong analytical and problem-solving abilities.

Responsibilities

• Develop scalable data processing pipelines using Hadoop/EMR, Spark, and Hive.
• Implement AWS data services (Lambda, EventBridge, Step Functions, Redshift, S3, Glue).
• Develop APIs, write SQL queries, transform and analyze data, visualize results.
• Collaborate with teams across the organization to drive data solutions.
• Contribute to CI/CD pipelines and best practices.

Skills

Python development

Big data pipelines

Hadoop/EMR

Spark

Hive

AWS Lambda

AWS EventBridge

AWS Step Functions

AWS Redshift

AWS S3

AWS Glue

API calls

SQL

Data transformation

Visualization

Gitlab

Terraform

CI/CD

Tools

Gitlab

Terraform

Hadoop

Spark

Hive

Redshift

S3

Glue

Job description

Visionary Innovative Technology Solutions LLC is seeking an AWS Data Engineer to design and implement scalable data pipelines using Hadoop/EMR, Spark, and Hive within a modern cloud architecture on AWS. The role focuses on building robust data platforms, APIs, and analytics capabilities for decision support.

The candidate will work with Lambda, EventBridge, Step Functions, Redshift, S3, and Glue to deliver end-to-end data solutions, while collaborating across teams in a matrix organization.