Engineering Lead Manager, Data Engineering

We shaped the earliest forms of ad tech, and we’re looking for the technical expertise to help shape its future. Our customers have unique problems that can only be solved at internet scale, and that’s where the technical skills of our team make a real difference.

Our exchange handles over 500 billion requests every day (for comparison Google serves an estimated 9 billion searches a day), all running in our own global data centers. Every member of our technology team has an enormous amount of autonomy in building and managing our systems to support and enable our growing level of scale. Through the transparency of our technology, dedication to innovation and integrity, and long-standing customer relationships, we lead through change.

What’s it like to work at Index?

We have more than 550 Indexers around the globe dedicated to building a safe and transparent marketplace that provides a trusted experience for consumers.

Index is an exciting and fast-paced place to work. We’re built on our values of change, support, learning and teaching, trust, and intention. We pride ourselves on our independence and openness, not only in our technology, but in our teams, too. Our diverse and inclusive culture celebrates how we can leverage our unique differences to help drive Index forward.

Our culture of success is truly supportive and collaborative. In working together across our teams, we’re continually investing in the people and technology to solve the industry’s most complex problems. As we extend the promise of ad tech to every channel, we’re looking for talented engineers to help advance Index, and the industry, forward.

About The Role

The Engineering Lead Manager is a critical role within the Data Engineering team with ownership over our high scale and low latency data processing pipelines, analytics platforms, and reporting products.

Our systems process over 550 billion auction events and petabytes of data daily. Index’s scale spans the globe, our transactions happen 24x7 in our global data centers, and every second that passes millions of events are processed and serviced through our data platforms.

To achieve our mission, efficiency, reliability, scalability and data quality are key, as all our data is crucial in providing business insights and driving decisions in Index’s and our partners’ business. Our dynamic reporting products are the main use case for Index’s UI, and partnering with internal and external stakeholders is a critical part of this leadership role.

What We’re Looking For

1. Data Engineering: Lead the transformation of the next generation of big data solutions at Index Exchange with real-time streaming and batch analytical capabilities and managing petabyte scale Data warehouse systems. Collaborate and service to both internal and external customers and partners for their reporting and analytics as well as machine model training needs.
2. Technical Leadership: You have gained experience and skills that enable you to chart a technical path for products, systems, and architectures. You are happy to be hands-on in the code, but your biggest impact is leading from the front and guiding others. Your role is to represent the technical health and future of the products you support while applying industry best practices towards designs and implementations.
3. People Leadership: You know how to work with and lead a distributed team of data and software engineers. You have demonstrated mentorship skills and set a technical vision to challenge your team in a way that empowers them to have incredible impact. You empower teams, who in turn build intricate systems that deliver great outcomes.
4. Handler of Broad Scope: You thrive in environments where multiple projects and initiatives are under your ownership at different maturity levels. Specifically, when the number of initiatives is larger than the resources. You can manage your own and prioritize your team’s time in each project to achieve the highest impact for the organization. You take sequencing of projects and tasks into consideration and proactively report progress and delays to drive coordination.
5. Driver of Outcomes: You are intentional in your path, establishing a vision or hypothesis, setting a direction, and guiding your team through execution. You use data to inform pivots or to validate assumptions, and partner with the Product team to set a roadmap, establish goals and forecasts to achieve both technical and business outcomes.
6. End-to-End Owner: You take great pride in the platforms your team owns and delivers, and take full responsibility for their quality, health, and outcomes. You lead your team in ensuring services remain healthy and that technical debt is regularly paid down. When issues arise, you lead your team in responding, mitigating, and addressing the issues to learn from them for the future.
7. Organizational Coordinator: You are the first point of contact for your team, your platforms, your products, and your delivery plan. You foster a collaborative approach to delivery and leverage those relationships to unblock dependencies. You are a strong communicator and collaborator, responsive when issues arise, and celebrate successes by representing your team's achievements to the rest of the organization.

Here's What You Need

• A college or university degree/diploma in Computer Science, Software Engineering, or a related field.
• A minimum of 3 years' experience in leadership and management of 3-10 person teams.
• Extensive experience in designing, building and managing large scale low latency data processing pipelines and analytical platforms.
• Extensive experience in partnering with Product Management and other internal stakeholders on developing roadmaps, deliverables and timelines.
• Experience working with external/client stakeholders is a strong asset.
• Deep understanding of design principles of distributed systems with streaming data platform frameworks and technologies.
• Experience with data modeling, data warehousing, ETL processes, and business intelligence reporting tools.
• Knowledge of big data stack including Kafka, Hadoop, Spark, Flink, Scala, and Airflow, as well as data warehouse technologies such as Druid, Starrocks, Vertica.
• Knowledge of BI tools like Looker, Superset and/or Redash will be very helpful.
• Experience with Docker and Kubernetes is a plus.

Why You’ll Love Working Here:

• Comprehensive health, dental, and vision plans for you and your dependents.
• Paid time off, health days, and personal obligation days plus flexible work schedules.
• Competitive retirement matching plans.
• Equity packages.
• Generous parental leave available to birthing, non-birthing, and adoptive parents.
• Annual well-being allowance plus fitness discounts and group wellness activities.
• Commuter benefits and discounts, where available.
• Employee assistance program.
• Mental health first aid program that provides an in-the-moment point of contact and reassurance.
• One day of volunteer time off per year and a donation-matching program.
• Bi-weekly town halls and regular community-led team events.
• Multiple resources and programming to support continuous learning.
• A workplace that supports a diverse, equitable, and inclusive environment.