Lead Real-Time Data Engineer: Kafka, Spark, AWS

Testing Site - JPMC; Jersey City (NJ); On-site; USD 140,000 - 210,000; Full time

Job summary

JPMorganChase, within the Regulatory Reporting Team, seeks a Lead Software Engineer to design and maintain data processing solutions. You will contribute to real-time pipelines, ensure security and compliance, and mentor engineers in a fast-paced agile setting.

Lead initiatives across data ingestion, transformation, and storage, collaborating with cross-functional teams to deliver scalable platforms and optimize performance.

Qualifications

• Formal training or certification in software engineering concepts and 5+ years of applied experience.
• Hands-on software engineering experience.
• Advanced proficiency in Java, Kafka, and SQL.
• Experience developing Java applications for data ingestion, transformation, and storage.
• Ability to integrate data processing solutions with AWS services (e.g., Amazon MSK, S3, Lambda, EMR).
• Strong understanding of AWS cloud architectures.
• Experience with data enrichment, transformation, and optimization.
• Skilled in debugging and maintaining code in large-scale environments.
• Proficiency in designing and implementing real-time data processing solutions.
• Familiarity with modern programming languages and database querying.
• Strong problem-solving and analytical skills.

Responsibilities

• Design, develop, and troubleshoot software solutions for data processing workflows
• Implement real-time data processing systems to manage large datasets efficiently
• Ensure all data processing solutions meet security and compliance requirements
• Document workflows, system architecture, and best practices for data processing
• Optimize data pipelines for improved performance and scalability
• Monitor and resolve performance issues in Kafka and Spark applications
• Lead knowledge-sharing initiatives to promote adoption of new technologies within the engineering team
• Foster a collaborative and inclusive team environment
• Collaborate with cross-functional teams to deliver technology solutions
• Participate in code reviews and provide constructive feedback
• Support ongoing maintenance and enhancements of existing systems

Skills

Java

Kafka

SQL

AWS

Data ingestion

Real-time processing

Data quality

Debugging

Large-scale systems

Python

Education

Software engineering training

Tools

Docker

Kubernetes

S3

Lambda

EMR

MSK

Job description

JPMorganChase, within the Regulatory Reporting Team, seeks a Lead Software Engineer to design and maintain data processing solutions. You will contribute to real-time pipelines, ensure security and compliance, and mentor engineers in a fast-paced agile setting.

Lead initiatives across data ingestion, transformation, and storage, collaborating with cross-functional teams to deliver scalable platforms and optimize performance.