Senior Python Big Data Engineer (AWS, Spark, Hadoop)

Visionary Innovative Technology Solutions LLC; Atlanta (GA); On-site; USD 140,000 - 170,000; Full time

Job summary

Visionary Innovative Technology Solutions LLC in Atlanta seeks a senior Python developer to design and optimize big data pipelines. You will work with Hadoop/EMR, Spark, Hive, and AWS services to process and analyze large datasets.

You will implement scalable data processing solutions, integrate APIs, perform data transformations and create visualization-ready outputs, collaborating across teams to deliver reliable data infrastructure.

Qualifications

• 10+ years hands-on Python development for big data applications.
• 5+ years in AWS services such as EMR, Lambda, EventBridge, Step Functions, Redshift, S3, Glue.
• Experience making API calls, SQL queries, and data manipulation/visualization.
• Strong CI/CD experience with GitLab and Terraform.
• Excellent analytical, problem-solving, and collaboration skills.

Skills

Python development

Big data

Hadoop/EMR

Spark

Hive

AWS

API calls

SQL queries

CI/CD pipelines

GitLab

Terraform

Collaboration

Problem solving

Tools

Hadoop/EMR

Spark

Hive

AWS Lambda

EventBridge

Step Functions

Redshift

S3

Glue

GitLab

Terraform

Job description

Visionary Innovative Technology Solutions LLC in Atlanta seeks a senior Python developer to design and optimize big data pipelines. You will work with Hadoop/EMR, Spark, Hive, and AWS services to process and analyze large datasets.

You will implement scalable data processing solutions, integrate APIs, perform data transformations and create visualization-ready outputs, collaborating across teams to deliver reliable data infrastructure.