AWS Big Data Lead Software Engineer

JPMorgan Chase & Co.; Columbus (OH); On-site; USD 120,000 - 180,000; Full time

Job summary

JPMorgan Chase & Co. is seeking an experienced software engineer to design and build modern applications on AWS, leveraging Spark or Flink for large-scale data processing.

You will work with multi-national teams to deliver batch and real-time components with reliability and security. The role requires strong SDLC and Agile experience, plus hands-on scripting (Python/Bash) and infrastructure-as-code with Terraform.

Qualifications

• 5+ years of software engineering experience with design, development and testing.
• Hands-on experience with Spark, Flink or similar data engines.
• Experience deploying on AWS using EC2, EKS, Lambda, S3.
• Experience with distributed datastores and big data tech.
• Familiarity with CI/CD, security and operational stability.
• Strong understanding of SDLC and agile practices.

Responsibilities

• Design and build new applications using modern technologies.
• Implement batch and real-time components with reliability and security.
• Ensure code quality with automated tests (unit/integration/acceptance).
• Collaborate with global agile teams to meet sprint goals.
• Participate in daily standups, planning, reviews and retrospectives.
• Provide level 2 production support.
• Apply processes for secure, stable code and systems.
• Develop applications leveraging distributed compute like Spark/Flink on large datasets.
• Design apps on AWS (EC2, EKS, Lambda, S3) and deploy components.
• Lead projects from concept to production across PNI platform.

Skills

5+ years exp

Apache Spark

Apache Flink

Distributed Datastores

AWS Deployment

Big Data

CI/CD

Security

SDLC

Agile Methodologies

Cloud Knowledge

Tools

Terraform

Airflow

Kubeflow

Bash

Python

KSH

Job description

Job responsibilities

• Design & build new applications utilizing leading edge technologies and modernize existing applications
• Implement batch & real-time software components consistent with architectural best-practices of reliability, security, operational efficiency, cost-effectiveness and performance
• Ensure quality of deployed code via automated unit, integration & acceptance testing
• Collaborate with multi-national agile development, support and business teams to meet sprint objectives
• Participate in all agile meetings & rituals, including daily standups, sprint planning, backlog reviews, demos, and retrospectives
• Provide level 2 support for production systems
• Learn and applies system processes, methodologies, and skills for the development of secure, stable code and systems
• Hands on applicaitn development leveraging distributed compute such as Apache flink or Spark on very large datasets
• Design and development of applications that leverage the AWS infrastructure deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Lead and deliver projects from concept to production across PNI (Personalization and Insights) platform

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands‑on practical experience in Frameworks, system design, application development, testing, and operational stability
• Experience with Apache Spark, Apache Flink or similar large-scale data processing engines
• Experience with Distributed Datastores (e.g. Cassandra, Red Shift)
• Experience designing, developing and deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Experience with Big Data / Distributed / cloud technology (AWS Big data services like lambda, glue, glue emr, Performance tuning, Streaming, KAFKA, Entitlements etc., )
• Experience with Apache Spark, Ray or similar large-scale data processing engines
• Proficiency in automation and continuous delivery methods
• Proficient in all aspects of the Software Development Life Cycle
• Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
• Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, BigData, artificial intelligence, machine learning, mobile, etc.)

Preferred qualifications, capabilities, and skills

• Experience building ETL/Feature processing pipelines
• Experience using workflow orchestration tools Airflow, Kubeflow etc.
• Experience using Terraform to deploy infrastructure-as-code to public cloud
• Experience with Linux scripting such as Bash, KSH, or Python
• Certified AWS Cloud Practitioner, Developer or Solutions Architect strongly preferred