Senior Big Data Engineer — Hadoop, Spark & Data Platform

Bank of America; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

Bank of America is seeking an experienced Senior Big Data Engineer to build and optimize large-scale data platforms. You will design data pipelines, implement robust data processing, and ensure integration with broader applications and systems.

Your focus will be on Hadoop ecosystem, Spark, Hive, and modern data tools, collaborating with cross-functional teams to deliver reliable, scalable data solutions in an agile environment.

Qualifications

• 4+ years of relevant work experience.
• Proficiency in programming language(s) required for the role.
• Experience in Big Data and Data Engineering.
• Strong hands-on expertise with Hadoop ecosystem including Cloudera Hadoop, Hive, Spark, HDFS/Ozone, SQL performance tuning, Python, Scala, and Unix scripting.
• Proven experience supporting large-scale production data platforms, troubleshooting distributed systems, and delivering reliable Big Data solutions.
• Experience with Cloudera platform administration, capacity planning, performance tuning, and optimization of Hadoop workloads.
• Strong understanding of Hive, Spark, YARN, storage and compute performance tuning, including root cause analysis.
• Hands-on experience with Cloudera Private Cloud Data Services including implementation, onboarding, migration, and operational support.
• Solid understanding of data warehousing, modern data platform architectures, and Agile delivery practices.
• Experience with workload migrations, platform onboarding, platform modernization, and cloud-enabled data services is preferred.
• Strong analytical and problem-solving skills, customer focus, and ability to collaborate with cross-functional teams.

Responsibilities

• Codes solutions and unit test to deliver a requirement/story per the defined acceptance criteria and compliance requirements
• Utilizes multiple architectural components in design and development of client requirements
• Performs CI/CD activities
• Contributes to story refinement and definition of requirements
• Participates in estimating work necessary to realize a story through the delivery lifecycle
• Contributes to existing test suites, analyze test reports, and triage issues
• Performs spike/proof of concept to mitigate risk or implement new ideas

Skills

Application Development Automation

Collaboration

DevOps Practices

Solution Design

Agile Practices

Architecture

Result Orientation

Solution Delivery

Process

User Experience Design

Analytical Thinking

Data Management

Risk Management

Technical Strategy Development

Test Engineering

Education

Bachelor's degree or equivalent work experience

Tools

Cloudera Hadoop

Hive

Spark

HDFS/Ozone

SQL

Python

Scala

Unix scripting

Cloudera Private Cloud Data Services

Job description

Bank of America is seeking an experienced Senior Big Data Engineer to build and optimize large-scale data platforms. You will design data pipelines, implement robust data processing, and ensure integration with broader applications and systems.

Your focus will be on Hadoop ecosystem, Spark, Hive, and modern data tools, collaborating with cross-functional teams to deliver reliable, scalable data solutions in an agile environment.