Senior Data Engineer: Real-Time Data Pipelines & Spark Expert

JobCubby; Jersey City (NJ); On-site; USD 128,000 - 182,000; Full time

Benefits offered by this job

Benefits eligible

Job summary

Bank of America is seeking a senior data engineering lead in Jersey City to drive enterprise data solutions and mentor teams across multiple domains. You will guide design, development and delivery of complex data integration projects using Spark, Hadoop, Starburst and Informatica, with real-time streaming emphasis.

You will collaborate with project management and cross‑functional groups to ensure robust interfaces, data quality, and compliance while delivering scalable data pipelines for

Qualifications

• 12+ years of data engineering experience.
• 4+ years hands-on experience in Scala.
• 4+ years of hands-on experience in Spark.
• Very good understanding of Hadoop components, architecture & tool sets Spark and Scala.
• Informatica, Starburst, Object Storage.
• Must have architected, developed & implemented solutions using real time streaming technologies (Spark Streaming/Kafka).
• Good knowledge of spark configurations and Spark/Scala performance tuning.
• Strong working knowledge on Hadoop, Hive, Impala, Oozie.
• Experience working with Exadata & Oracle SQL Tuning.
• Experience working with Starburst and Object Storage.
• Experience developing Streaming/Near Real time and Batch data integration solutions.
• Hands on experience in UNIX Shell Scripting
• Working experience in scheduling jobs with Autosys.

Responsibilities

• Assembles large, complex data sets that meet functional and non‑functional requirements across multiple systems.
• Maintains, cleans, and manipulates large data for operational and analytics data systems; builds complex data transformation processes.
• Defines and builds data pipelines to enable data-informed decision making.
• Leads data integration solutions involving Spark/Scala, Hadoop, Starburst, Object Storage, Cloud tech and Informatica products.
• Contributes to test suites including integration, regression, and performance; analyzes test reports and triages issues.
• Mentors Data Engineers in CI/CD and defines KPIs and internal controls.
• Coordinates with deployment and support teams to ensure production readiness and smooth deployments.

Skills

Analytical Thinking

Data Management

Solution Design

Architecture

Agile Practices

Collaboration

DevOps Practices

Data Quality

Financial Management

Risk Management

Decision Making

Application Development

Business Acumen

Test Engineering

Solution Delivery

Tools

Spark

Hadoop

Starburst

Informatica

Kafka

Spark Streaming

Oracle SQL

UNIX Shell Scripting

Autosys

Hive

Impala

Oozie

Exadata

Starburst & Object Storage

Object Storage

Job description

Bank of America is seeking a senior data engineering lead in Jersey City to drive enterprise data solutions and mentor teams across multiple domains. You will guide design, development and delivery of complex data integration projects using Spark, Hadoop, Starburst and Informatica, with real-time streaming emphasis.

You will collaborate with project management and cross‑functional groups to ensure robust interfaces, data quality, and compliance while delivering scalable data pipelines for