Senior Data Infrastructure Engineer: Scale, Automation & OSS

Roberts Recruiting, LLC; Cambridge (MA); On-site; USD 80,000 - 120,000; Full time

Job summary

A leading technology firm in Cambridge is seeking talented software engineers to enhance data infrastructure and automate existing systems. You'll help make database access intuitive and reliable for development purposes. Ideal candidates will have experience with data storage technologies and strong problem-solving skills. The company fosters a culture of flexibility and growth, aiming to support developers with effective tools.

Qualifications

• Experience with data storage technologies like MySQL, HBase, Kafka.
• Strong problem-solving abilities and pragmatic decision-making.
• Ability to work closely with developers to create automation tools.

Responsibilities

• Build reliable and efficient database access for application developers.
• Implement intelligent features for data stores to enhance productivity.
• Collaborate with teams to improve client-side interactions with data.

Skills

Automation development

MySQL

Hadoop HBase

Kafka

Spark

Elastic Search

AWS

Job description

Senior Software Engineer, Data Infrastructure

• Cambridge, Massachusetts

The Data Infrastructure teams are responsible for building and maintaining data storage technologies across the product. Teams work with a variety of open source technologies like MySQL, Vitess, Hadoop HBase, Kafka, Spark, and Elastic Search all running on AWS.

One of the primary objectives is to automate the deployments, configuration, and recoverability of data stores to efficiently scale storage systems. Teams also take on projects to improve the client-side interaction with the data stores with analysis tools and wrapper client interfaces. A number of our teams are active contributors to the Open Source projects that they work on.

We’re operationally responsible for a huge volume of traffic to and from these data stores. Our HBase clusters serve over 3 million requests/second across 220+ tables, while our ElasticSearch clusters serve over 20k searches/second and 50k indexes/second to 90+ billion documents.

Streaming that data to and from applications amounts to more than 3 GB/sec of data through our Kafka clusters, with hundreds of producers and consumers.

What we’re looking for

We’re looking for talented software engineers to help us build the vision of making our database access simple, intuitive, highly performant and highly reliable to our customers and ohapplication developers.

The goal of the team is to implement a comprehensive set of intelligent features that will drive the productivity of using any of our data stores at scale and with the highest possible reliability.

Bonus points for being:

• Someone with experience developing automation or is a power user of one or more of the above-mentioned data storage technologies.
• Willingness and ability to work with the user base, in this case, other developers, to define and build tools that make the lives of application developers easier and their code less error-prone through automation.
• Interested in working with and supporting infrastructure at scale.
• Someone who has shown that they can solve complicated technical problems and analyze tradeoffs with empathy for the developers, yet building creative solutions.
• Able to demonstrate pragmatic decision making and problem-solving abilities.

About Us

We help millions of organizations grow better, and we’d love to grow better with you. Our business builds the software and systems that power the world’s small to medium-sized businesses. Our company culture builds connections, careers, and employee growth. How? By creating a workplace that values flexibility, autonomy, and transparency. If that sounds like something you’d like to be part of, we’d love to hear from you.