Senior Software Engineer - Hadoop Infrastructure

Bloomberg; New York (NY); On-site; USD 160,000 - 240,000; Full time

Benefits offered by this job

Merit increases

Incentive compensation

Paid holidays

Paid time off

Medical

Dental

Vision

Short and long term disability

401(k) + match

Life insurance

Wellness programs

Job summary

Bloomberg is seeking a Senior Software Engineer to join Hadoop Infrastructure in New York. You will build and manage large-scale distributed systems, focusing on data storage, low-latency access, and high availability across thousands of servers.

You’ll work with open-source Hadoop ecosystems (HDFS, HBase, YARN) and collaborate with multiple product teams to improve usability, reliability, and performance while contributing to open-source software.

Qualifications

• 4+ years of experience with Java and related technologies (Spring/JMX/JDBC).
• Degree in Computer Science, Engineering or equivalent work experience.
• 3+ years in the Hadoop ecosystem (HBase, HDFS, YARN).
• Knowledge of Jenkins, Maven, Jira.
• Solid Linux, shell scripting, and OS troubleshooting skills.
• Strong problem-solving and communication abilities.

Responsibilities

• Advance how tenants from multiple product domains leverage Hadoop Infrastructure services.
• Migrate and support a massive data and compute footprint (10s of PB) to newer versions.
• Improve tenants' user experience when securely interacting with infrastructure frameworks.
• Enhance usability, reliability, and scalability of open-source Hadoop services for Bloomberg teams.

Skills

Java

Education

Bachelor's degree in Computer Science or Engineering

Tools

Spring

JMX

JDBC

Hadoop

YARN

HBase

Linux

Job description

Senior Software Engineer - Hadoop Infrastructure

Location

New York

Business Area

Engineering and CTO

Ref #

10054847

Description & Requirements

The Team

Bloomberg runs on data! It's our business and our product. Financial institutions need timely, accurate data to capture opportunities and evaluate risk in fast-moving markets, from the biggest banks to elite hedge funds.

Bloomberg's application teams face complex challenges - large-scale data storage, low-latency retrievals, high-volume requests, and high availability over a distributed computing and storage environment.

Hadoop Infrastructure's mission is to provide a multi-tenant, observable, and highly available infrastructure backed by the open-source Apache Hadoop platform (HDFS, HBase, YARN, etc.) to support large-scale data applications across Bloomberg. The Hadoop Infrastructure team manages clusters that handle 10s of PB of storage spread across 1000s of servers running 100s of billions of requests per day and running 10s of thousands of jobs hitting 100s of thousands of tables every day. We also provide standard methodologies and domain expertise on Hadoop services to applications across various product domains at Bloomberg.

Who are you?

You are a dedicated and motivated engineer interested in building and managing large-scale distributed systems, looking for a tight-knit, collaborative team.

You are an innovative problem solver who enjoys working in multiple roles and thrives in a fast-paced environment.

You want to make a significant impact and contribute to open-source software.

We’ll trust you to:

Advance how tenants from multiple product domains leverage Hadoop Infrastructure services to meet their goals

Provide and improve our capabilities to migrate and support a massive data and compute footprint (10s of PB) to newer versions.

Improve our tenants' user experience when securely interacting with powerful underlying infrastructure frameworks.

Understand and improve the usability, reliability, and scalability of open-source Apache Hadoop services to optimize for the needs of Bloomberg application teams.

You’ll need to have:

4+ years of demonstrated experience working with an object-oriented programming language (Java) and associated technologies (eg. Spring/JMX/JDBC)

A degree in Computer Science, Engineering or similar field of study or equivalent work experience.

3+ years of experience in the Hadoop ecosystem and related technologies (HBase, HDFS, YARN)

Knowledge of modern development methodologies and tools (Jenkins/Maven/Jira).

Solid understanding of Linux Operating system, Shell scripting, OS troubleshooting

Strong problem-solving and communication skills.

We'd love to see:

Experience working with open-source software/community.

Experience with distributed systems architecture and system design.

Knowledge of Ansible.

Experience with workflow automation tools like Airflow, Argo, etc.

Salary Range = 160,000-240,000 USD Annual+ Benefits + Bonus

The referenced salary range is based on the Company's good faith belief at the time of posting. Actual compensation may vary based on factors such as geographic location, work experience, market conditions, education/training and skill level.

• merit increases
• incentive compensation (exempt roles only)
• paid holidays
• paid time off
• medical
• dental
• vision
• short and long term disability benefits
• 401(k) +match
• life insurance
• various wellness programs

The Company does not provide benefits directly to contingent workers/contractors and interns.

Discover what makes Bloomberg unique - watch our podcast series for an inside look at our culture, values, and the people behind our success.