Lead Software Engineer - SPARK, Java, Python, AWS

Social network you want to login/join with:

Lead Software Engineer - SPARK, Java, Python, AWS, London

Client:

Location:

London, United Kingdom

Job Category:

Other

-

EU work permit required:

Yes

Job Reference:

d841d0cc5867

Job Views:

6

Posted:

26.04.2025

Expiry Date:

10.06.2025

Job Description:

We have an exciting opportunity for a Lead Software Engineer at JPMorgan Chase within the Data Pipeline Technology Engineering team. You will play a key role in developing data pipeline applications on AWS, working closely with business and product teams to create robust data transformation (ETL) functionalities that meet complex business needs. Your expertise will help transition our systems to the public cloud, focusing on AWS services, security best practices, performance optimization, and database migration.

Job Responsibilities:

• Configure and customize tools to generate products based on business or customer requests.
• Update data models to reflect new use cases.
• Create architecture and design artifacts for complex applications, ensuring compliance with design constraints.
• Analyze large data sets to develop visualizations and reports for continuous improvement.
• Support business engineering through solution design and technical architecture development.
• Build distributed systems using Java (11/17) on AWS.
• Design architectures that align with business needs and facilitate application development.

Required Qualifications, Capabilities, and Skills:

• Experience with Spark-based frameworks for ETL, ELT, and reporting, including Spark SQL & Spark Streaming.
• Strong knowledge of multi-threading and high-volume batch processing.
• Proficiency in performance tuning for Java/Python and Spark.
• Deep understanding of AWS services and container technologies like Kubernetes.
• Experience deploying applications across multi-AZ, multi-region, and multi-cloud environments.
• Knowledge of modern engineering practices for cloud scalability and automation.

Preferred Qualifications, Capabilities, and Skills:

• Understanding of the Big Data stack (Spark/Iceberg).
• Ability to quickly learn and apply new technologies and patterns.
• Familiarity with stability patterns, event-driven architecture, CQRS, and process orchestration.
• Experience designing architectures aligned with business requirements.