Senior Data Infrastructure Engineer: Scale, Reliability & Tooling

Roberts Recruiting, LLC; Cambridge (MA); On-site; Full time

Job summary

A leading recruiting firm in Massachusetts seeks a qualified candidate to join their data infrastructure team. The role requires experience in datastores like MySQL, HBase, ElasticSearch, and Kafka. Candidates should be passionate about building developer tools and managing infrastructure at scale while being eager to learn and grow. This position provides an opportunity to impact operations significantly.

Qualifications

• Must have experience with MySQL, HBase, ElasticSearch, and Kafka.
• Passionate about building tools for developer efficiency.
• Interest in infrastructure management at scale.

Skills

Experience with MySQL

Experience with HBase

Experience with ElasticSearch

Experience with Kafka

Tool building for developers

Operational support for infrastructure at scale

Ability to learn and grow

Job description

TheData Infrastructure team is responsible for the various datastores we use, including MySQL, HBase, ElasticSearch, and Kafka. We aim to provide four nines (or better) of reliability for all of these, as well as to help our developers use them easily and safely.

We spend a lot of time building automation, tooling, and monitoring to equip developers to understand and optimize their datastore usage and minimize the impact of any operational issues on customers and developers alike.

We’re also operationally responsible for a huge volume of traffic to and from these datastores. Our HBase clusters serve over 3 million requests/second across 220+ tables, while our ElasticSearch clusters serve over 20k searches/second and 50k indexes/second to 90+ billion documents. Streaming that data to and from applications amounts to more than 3 GB/sec of data through our Kafka clusters, with hundreds of producers and consumers.

• Experienced with some or most of our datastores (MySQL, HBase, ElasticSearch, Kafka) as a user and/or operationally
• Excited to build great tools for developers so they can work faster and more reliably
• Interested in working with and supporting infrastructure at scale
• Eager to learn and grow

Get in touch! We’d love to talk to you about our data infrastructure team.