Senior Python Big Data Engineer - AWS & Spark

Smart IT Frame LLC; Georgia; On-site; USD 140,000 - 190,000; Full time

Job summary

Smart IT Frame LLC is seeking an experienced Python developer to lead big data processing initiatives in a scalable AWS-based environment. You will design and optimize data pipelines, perform complex data transformations, and deliver actionable insights through analysis and visualization.

Collaboration across teams and strong problem-solving will be essential to succeed in a matrixed organization. The role emphasizes hands-on development, scalable architectures, and CI/CD practices using Gitlab

Qualifications

• 10+ years of Python development for big data applications.
• 5+ years of experience with AWS services including EMR, Lambda, EventBridge, Step Functions, Redshift, S3, Glue.
• 5+ years of API calls, SQL, Spark, data manipulation, analysis and visualization.
• Hands-on experience with CI/CD pipelines and related tooling.

Skills

Python development

Big data processing

Hadoop/EMR

Spark

Hive

AWS Lambda

EventBridge

Step Functions

Redshift

S3

Glue

API calls

SQL queries

Data transformation

Data analysis

Visualization

CI/CD pipelines

Gitlab

Terraform

Collaboration

Analytical thinking

Tools

Gitlab

Terraform

CI/CD pipelines

Job description

Smart IT Frame LLC is seeking an experienced Python developer to lead big data processing initiatives in a scalable AWS-based environment. You will design and optimize data pipelines, perform complex data transformations, and deliver actionable insights through analysis and visualization.

Collaboration across teams and strong problem-solving will be essential to succeed in a matrixed organization. The role emphasizes hands-on development, scalable architectures, and CI/CD practices using Gitlab