AWS Big Data Lead Software Engineer

Testing Site - JPMC; Columbus (OH); On-site; USD 90,000 - 130,000; Full time

Job summary

Testing Site - JPMC is seeking a software engineer to design and build new applications using leading edge technologies and modernize existing systems.

The role emphasizes batch and real-time components, AWS deployments, and big data processing with Spark/Flink. You will collaborate across multinational teams in an agile environment and contribute to automated testing and robust software delivery.

Qualifications

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on practical experience in Frameworks, system design, application development, testing, and operational stability
• Experience with Apache Spark, Apache Flink or similar large-scale data processing engines
• Experience with Distributed Datastores (e.g. Cassandra, Red Shift)
• Experience designing, developing and deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Experience with Big Data / Distributed / cloud technology (AWS Big data services like lambda, glue, glue emr, Performance tuning, Streaming, KAFKA, Entitlements etc., )
• Experience with Apache Spark, Ray or similar large-scale data processing engines
• Proficiency in automation and continuous delivery methods
• Proficient in all aspects of the Software Development Life Cycle
• Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
• Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, BigData, artificial intelligence, machine learning, mobile, etc.)

Responsibilities

• Design & build new applications utilizing leading edge technologies and modernize existing applications
• Implement batch & real-time software components consistent with architectural best-practices of reliability, security, operational efficiency, cost-effectiveness and performance
• Ensure quality of deployed code via automated unit, integration & acceptance testing
• Collaborate with multi-national agile development, support and business teams to meet sprint objectives
• Participate in all agile meetings & rituals, including daily standups, sprint planning, backlog reviews, demos, and retrospectives
• Provide level 2 support for production systems
• Learn and applies system processes, methodologies, and skills for the development of secure, stable code and systems
• Hands on applicaitn development leveraging distributed compute such as Apache flink or Spark on very large datasets
• Design and development of applications that leverage the AWS infrastructure deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Lead and deliver projects from concept to production across PNI (Personalization and Insights) platform

Skills

Large-scale data processing

Cloud computing

CI/CD / DevOps

Agile methodologies

Distributed datastores

Python

SQL

Tools

Apache Spark

Apache Flink

Cassandra

Redshift

AWS

Terraform

Airflow

Kubeflow

Kubernetes

Bash

Python

EC2

S3

Job description

Job responsibilities

• Design & build new applications utilizing leading edge technologies and modernize existing applications
• Implement batch & real-time software components consistent with architectural best-practices of reliability, security, operational efficiency, cost-effectiveness and performance
• Ensure quality of deployed code via automated unit, integration & acceptance testing
• Collaborate with multi-national agile development, support and business teams to meet sprint objectives
• Participate in all agile meetings & rituals, including daily standups, sprint planning, backlog reviews, demos, and retrospectives
• Provide level 2 support for production systems
• Learn and applies system processes, methodologies, and skills for the development of secure, stable code and systems
• Hands on applicaitn development leveraging distributed compute such as Apache flink or Spark on very large datasets
• Design and development of applications that leverage the AWS infrastructure deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Lead and deliver projects from concept to production across PNI (Personalization and Insights) platform

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on practical experience in Frameworks, system design, application development, testing, and operational stability
• Experience with Apache Spark, Apache Flink or similar large-scale data processing engines
• Experience with Distributed Datastores (e.g. Cassandra, Red Shift)
• Experience designing, developing and deploying software components onAWS using common compute and storage services such as EC2, EKS, Lambda, S3
• Experience with Big Data / Distributed / cloud technology (AWS Big data services like lambda, glue, glue emr, Performance tuning, Streaming, KAFKA, Entitlements etc., )
• Experience with Apache Spark, Ray or similar large-scale data processing engines
• Proficiency in automation and continuous delivery methods
• Proficient in all aspects of the Software Development Life Cycle
• Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
• Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, BigData, artificial intelligence, machine learning, mobile, etc.)

Preferred qualifications, capabilities, and skills

• Experience building ETL/Feature processing pipelines
• Experience using workflow orchestration tools—Airflow, Kubeflow etc.
• Experience using Terraform to deploy infrastructure-as-code to public cloud
• Experience with Linux scripting such as Bash, KSH, or Python
• Certified AWS Cloud Practitioner, Developer or Solutions Architect strongly preferred