Engineering Lead Manager, Data Engineering

Index Exchange; Toronto; On-site; CAD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking a talented Engineering Lead Manager to oversee high-scale data processing pipelines and analytics platforms. In this critical role, you will lead a dynamic team, driving innovation and technical excellence while collaborating with internal and external stakeholders. The ideal candidate will possess extensive experience in data engineering and leadership, guiding the development of cutting-edge solutions that enhance decision-making and business insights. Join a vibrant and inclusive culture that values collaboration, continuous learning, and the pursuit of excellence in the fast-evolving ad tech landscape.

Benefits

Comprehensive health, dental, and vision plans

Paid time off and flexible work schedules

Competitive retirement matching plans

Equity packages

Generous parental leave

Annual well-being allowance

Commuter benefits

Employee assistance program

Mental health first aid program

Volunteer time off and donation-matching program

Qualifications

• 3+ years of leadership experience managing teams of 3-10 engineers.
• Expertise in designing large scale low latency data processing pipelines.

Responsibilities

• Lead the transformation of big data solutions with real-time streaming capabilities.
• Manage multiple projects and initiatives to achieve high organizational impact.

Skills

Leadership

Data Engineering

Distributed Systems

Data Modeling

Project Management

Analytical Skills

Education

Bachelor's Degree in Computer Science

Diploma in Software Engineering

Tools

Kafka

Hadoop

Spark

Flink

Scala

Airflow

Docker

Kubernetes

Looker

Superset

Job description

We shaped the earliest forms of ad tech, and we’re looking for the technical expertise to help shape its future. Our customers have unique problems that can only be solved at internet scale, and that’s where the technical skills of our team make a real difference.

Our exchange handles over 500 billion requests every day (for comparison Google serves an estimated 9 billion searches a day), all running in our own global data centers. Every member of our technology team has an enormous amount of autonomy in building and managing our systems to support and enable our growing level of scale. Through the transparency of our technology, dedication to innovation and integrity, and long-standing customer relationships, we lead through change.

What’s it like to work at Index?

We have more than 550 Indexers around the globe dedicated to building a safe and transparent marketplace that provides a trusted experience for consumers.

Index is an exciting and fast-paced place to work. We’re built on our values of change, support, learning and teaching, trust, and intention. We pride ourselves on our independence and openness, not only in our technology, but in our teams, too. Our diverse and inclusive culture celebrates how we can leverage our unique differences to help drive Index forward.

Our culture of success is truly supportive and collaborative. In working together across our teams, we’re continually investing in the people and technology to solve the industry’s most complex problems. As we extend the promise of ad tech to every channel, we’re looking for talented engineers to help advance Index, and the industry, forward.

About The Role

The Engineering Lead Manager is a critical role within the Data Engineering team with ownership over our high scale and low latency data processing pipelines, analytics platforms, and reporting products.

Our systems process over 550 billion auction events and petabytes of data daily. Index’s scale spans the globe, our transactions happen 24x7 in our global data centers, and every second that passes millions of events are processed and serviced through our data platforms.

To achieve our mission, efficiency, reliability, scalability and data quality are key, as all our data is crucial in providing business insights and driving decisions in Index’s and our partners’ business. Our dynamic reporting products are the main use case for Index’s UI, and partnering with internal and external stakeholders is a critical part of this leadership role.

What We’re Looking For

1. Data Engineering: Lead the transformation of the next generation of big data solutions at Index Exchange with real-time streaming and batch analytical capabilities and managing petabyte scale Data warehouse systems. Collaborate and service to both internal and external customers and partners for their reporting and analytics as well as machine model training needs.
2. Technical Leadership: You have gained experience and skills that enable you to chart a technical path for products, systems, and architectures. You are happy to be hands-on in the code, but your biggest impact is leading from the front and guiding others. Your role is to represent the technical health and future of the products you support while applying industry best practices towards designs and implementations.
3. People Leadership: You know how to work with and lead a distributed team of data and software engineers. You have demonstrated mentorship skills and set a technical vision to challenge your team in a way that empowers them to have incredible impact. You empower teams, who in turn build intricate systems that deliver great outcomes.
4. Handler of Broad Scope: You thrive in environments where multiple projects and initiatives are under your ownership at different maturity levels. Specifically, when the number of initiatives is larger than the resources. You can manage your own and prioritize your team’s time in each project to achieve the highest impact for the organization. You take sequencing of projects and tasks into consideration and proactively report progress and delays to drive coordination.
5. Driver of Outcomes: You are intentional in your path, establishing a vision or hypothesis, setting a direction, and guiding your team through execution. You use data to inform pivots or to validate assumptions, and partner with the Product team to set a roadmap, establish goals and forecasts to achieve both technical and business outcomes.
6. End-to-End Owner: You take great pride in the platforms your team owns and delivers, and take full responsibility for their quality, health, and outcomes. You lead your team in ensuring services remain healthy and that technical debt is regularly paid down. When issues arise, you lead your team in responding, mitigating, and addressing the issues to learn from them for the future.
7. Organizational Coordinator: You are the first point of contact for your team, your platforms, your products, and your delivery plan. You foster a collaborative approach to delivery and leverage those relationships to unblock dependencies. You are a strong communicator and collaborator, responsive when issues arise, and celebrate successes by representing your team's achievements to the rest of the organization.

Here's What You Need

• A college or university degree/diploma in Computer Science, Software Engineering, or a related field
• A minimum of 3 years' experience in leadership and management of 3-10 person teams
• Extensive experience in designing, building and managing large scale low latency data processing pipelines and analytical platforms
• Extensive experience in partnering with Product Management and other internal stakeholders on developing roadmaps, deliverables and timelines
• Experience working with external/client stakeholders is a strong asset
• Deep understanding of design principles of distributed systems with streaming data platform frameworks and technologies
• Experience with data modeling, data warehousing, ETL processes, and business intelligence reporting tools
• Knowledge of big data stack including Kafka, Hadoop, Spark, Flink, Scala, and Airflow, as well as data warehouse technologies such as Druid, Starrocks, Vertica
• Knowledge of BI tools like Looker, Superset and/or Redash will be very helpful
• Experience with Docker and Kubernetes is a plus

Why You’ll Love Working Here:

• Comprehensive health, dental, and vision plans for you and your dependents
• Paid time off, health days, and personal obligation days plus flexible work schedules
• Competitive retirement matching plans
• Equity packages
• Generous parental leave available to birthing, non-birthing, and adoptive parents
• Annual well-being allowance plus fitness discounts and group wellness activities
• Commuter benefits and discounts, where available
• Employee assistance program
• Mental health first aid program that provides an in-the-moment point of contact and reassurance
• One day of volunteer time off per year and a donation-matching program
• Bi-weekly town halls and regular community-led team events
• Multiple resources and programming to support continuous learning
• A workplace that supports a diverse, equitable, and inclusive environment.