Software Engineer III - Hadoop - Spark

Bank of America; South Dakota; On-site; USD 120,000 - 180,000; Full time

Job summary

Bank of America is seeking a Senior Hadoop Developer to lead the design, development, and delivery of large-scale data solutions. You will work on Hadoop, Hive, Spark, and ETL pipelines, ensuring secure, compliant data processing and robust test automation.

The role requires deep experience in big data ecosystems, SQL, and scripting, with leadership in CI/CD and Agile practices. This on-site US-based position offers a comprehensive benefits package and growth opportunities.

Qualifications

• 10+ years of relevant work experience.
• 5+ years in data warehousing architectures in big data.
• Experience with Hadoop ecosystem (Cloudera).
• Experience with Big Data technologies like Hadoop, Hive, Spark, Python, Scala.
• Python and Unix shell scripting experience.
• Experience with scheduling tools like Autosys.
• Understanding of Agile methodologies.

Responsibilities

• Codes solutions and unit test to deliver a requirement per acceptance criteria.
• Designs, develops, and modifies architecture components and interfaces.
• Mentors team on CI/CD practices and automation.
• Performs proof of concept to mitigate risk.
• Automates release activities and maintains automated test suites.
• Develops applications using Hadoop technologies and ETL processes.
• Processes large data volumes and analyzes datasets for insights.
• Implements data security controls and regulatory compliance.

Skills

Hadoop

Hive

Spark

Python

Scala

Unix shell scripting

Autosys

SQL

CI/CD

Agile

Distributed systems

Data security

ETL

Education

Bachelor's degree or equivalent work experience

Tools

Cloudera

Job description

Company Culture

At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. We do this by driving Responsible Growth and delivering for our clients, teammates, communities and shareholders every day. Being a Great Place to Work and providing a culture of caring is core to how we drive Responsible Growth. We are intentional about fostering an inclusive workplace where every teammate has the opportunity to succeed, build a career and contribute to our shared success. This includes attracting and developing exceptional talent, recognizing and rewarding performance, and supporting our teammates’ physical, emotional, and financial wellness through affordable, competitive and flexible benefits. We value the unique perspectives individuals bring from all backgrounds and career paths - whether shaped by military service, community college education, or a wide range of work and life experiences. These journeys foster resilience, leadership and innovation, strengthening our workforce and positively impact the communities we serve. Join us!

Job Overview

This job is responsible for developing and delivering complex requirements to accomplish business goals. Key responsibilities of the job include ensuring that software is developed to meet functional, non-functional and compliance requirements, and solutions are well designed with maintainability/ease of integration and testing built-in from the outset. Job expectations include a strong knowledge of development and testing practices common to the industry and design and architectural patterns. Data solution delivery team is looking for an experienced Senior Hadoop Developer. Candidate should have in-depth knowledge of Hadoop technologies, SQL and ETL processes. Candidate should possess excellent analytical capabilities, get a deep understanding of the data and build co-relation between various data elements/objects. Data solution delivery team is also responsible for ingesting the data from multiple sources, the candidate should be able to support the ingestion process using the big data tools and frameworks.

Key Responsibilities

• Codes solutions and unit test to deliver a requirement/story per the defined acceptance criteria and compliance requirements
• Designs, develops, and modifies architecture components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained
• Mentors other software engineers and coach team on Continuous Integration and Continuous Development (CI-CD) practices and automating tool stack
• Executes story refinement, definition of requirements, and estimating work necessary to realize a story through the delivery lifecycle
• Performs spike/proof of concept as necessary to mitigate risk or implement new ideas
• Automates manual release activities
• Designs, develops, and maintains automated test suites (integration, regression, performance)
• Design, develop, and implement applications using Hadoop technologies.
• Create ETL (Extract, Transform, Load) processes - Ingest data from multiple sources, transform the data and create reporting views
• Process large volumes of data efficiently - Analyze datasets to identify trends and insights, Support reporting and analytics requirements
• Implement data security controls, Ensure data privacy and regulatory compliance
• Test applications and data pipelines, Debug production issues and performance bottlenecks, Resolve runtime and data quality problems
• Work on the stories assigned and provide updates in the daily scrum meetings
• Lead some of the initiatives within the team and co-ordinate the requirements with the line of business

Required Qualifications

• 10+ years of relevant work experience
• Proficiency in programming language(s) required for the role
• 5+ years of experience in data warehousing architectural approaches in big data.
• Sound understanding and experience with Hadoop ecosystem (Cloudera).
• Able to understand and explore the constantly evolving tools within Hadoop ecosystem and apply them appropriately to the relevant problems at hand.
• Experience in working with a Big Data implementation in production environment.
• Must have experience with Big Data technologies like Hadoop, Hive, Spark, python, scala etc.
• Experience in python and Unix shell scripting.
• Experience in scheduling tool like Autosys
• Understanding of Agile methodologies and technologies
• Sound knowledge of relational databases (SQL) and experience with large SQL based systems.
• Exposure to and strong working knowledge of distributed systems.
• Excellent understanding of client-service models and customer orientation in service delivery.
• Ability to grasp the 'big picture' for a solution by considering all potential options in impacted area.
• Aptitude to understand and adapt to newer technologies.
• The ability to work with teammates in a collaborative manner to achieve a mission.
• Experience in query optimization, performance tuning of the complex SQL queries.
• Benchmark and debug critical issues with algorithms and software as they arise

Desired Qualifications

• Previous experience in the financial services industry
• BofA technical experience and good understanding of existing testing/operational processes and an open mind on how to enhance those
• Understanding of industry trends and relevant application technologies
• Experience in designing and implementing analytical environments and business intelligence solutions

Skills

• Application Development
• Automation
• Influence
• Solution Design
• Technical Strategy Development
• Architecture
• Business Acumen
• DevOps Practices
• Result Orientation
• Solution Delivery
• Process
• Analytical Thinking
• Collaboration
• Data Management
• Risk Management
• Test Engineering

Minimum Education Requirement

Minimum Education Requirement: Bachelor’s degree or equivalent work experience.

Work Schedule and Compensation

Shift: 1st shift (United States of America) Hours Per Week: 40

Pay Transparency details US - NJ - Pennington - 1500 American Blvd - Hopewell Bldg 5 (NJ2150) Pay and benefits information Pay range $102,900.00 - $179,900.00 annualized salary, offers to be determined based on experience, education and skill set. Discretionary incentive eligible This role is eligible to participate in the annual discretionary plan. Employees are eligible for an annual discretionary award based on their overall individual performance results and behaviors, the performance and contributions of their line of business and/or group; and the overall success of the Company.

Benefits Overview

Benefits This role is currently benefits eligible. We provide industry-leading benefits, access to paid time off, resources and support to our employees so they can make a genuine impact and contribute to the sustainable growth of our business and the communities we serve.

Privacy Statement

Privacy Statement: https://careers.bankofamerica.com/en-us/privacy-notice

Company Values and Community Impact

At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work for our teammates around the world. We’re devoted to being a diverse and inclusive workplace for everyone. We hire individuals with a broad range of backgrounds and experiences and invest heavily in our teammates and their families by offering competitive benefits to support their physical, emotional, and financial well-being. Bank of America believes both in the importance of working together and offering flexibility to our employees. We use a multi-faceted approach for flexibility, depending on the various roles in our organization. Working at Bank of America will give you a great career with opportunities to learn, grow and make an impact, along with the power to make a difference. Join us! Partnering Locally Learn about some of the ways Bank of America is making a difference in the communities we serve. Global Impact Learn about the six areas that guide Bank of America’s efforts to help make financial lives better for customers, clients, communities and our teammates. Opportunity and Inclusion Each employee brings unique skills, background and opinions. We see opportunity and inclusion as our platform for innovation and a key component in our success. Our Values Learn about our four values that represent what we believe.

Pay Transparency Link

Pay Transparency: https://careers.bankofamerica.com/en-us/pay-transparency