Manager/Lead - Data Engineering

HRB; Canada; Remote; CAD 90,000 - 150,000; Full time

Job summary

An innovative consulting firm is seeking a Manager/Lead Data Engineer to join their rapidly growing team in Toronto. This role offers the unique opportunity to manage a talented team while tackling enterprise-level challenges using cutting-edge GCP technologies. You will lead the development of scalable data solutions, mentor junior engineers, and ensure best practices in data architecture and analytics. This position promises a dynamic work environment where you can leverage your expertise in cloud solutions to make a significant impact on high-profile projects across various industries. Join us in shaping the future of data engineering!

Qualifications

• Experience in leading significant project steps and communicating with peers.
• Good understanding of modern data architecture and GCP services.

Responsibilities

• Manage a team working with backend engineers and product managers.
• Write high-quality and testable batch and real-time data pipelines.
• Support clients in troubleshooting issues in production environments.

Skills

GCP (Google Cloud Platform)

Python

Java

Scala

Dataflow

Spark

Airflow

Data Governance

Agile Methodologies

Business Intelligence Tools

Tools

Dataflow

Spark

Airflow

QuickSight

Looker

Data Studio

Job description

Manager/Lead Data Engineering

We are a boutique, rapidly growing, GCP (Google Cloud Platform) consulting company based out of Toronto. We work with GCP’s top customers (banking, telco, energy, retail, etc.) to help them with cloud transformation, security, analytics, ML, and data governance. Clients usually engage us to solve their most challenging business problems and help raise the bar at their organization.

Why Us?
You get the best of both worlds. We operate like an early-stage startup with all the associated benefits (e.g., talent, growth, learning opportunities, flexibility), but get to solve enterprise-level technical challenges.

• People: We hire top-tier talent. Our team consists of ex-Googlers, YCombinator Alumni, and individuals that built software for 400M users.
• GCP is the Best Cloud: Maybe we are biased, but GCP is the most cutting-edge cloud provider, building on technologies that have been pioneered by Google: BigQuery, K8s/Anthos, Vertex AI, etc.
• Growth: We have doubled in size in the last 6 months, and are looking to double again this year.

You will
• Manage a team who works closely with backend engineers, product managers, and analysts
• Provide leadership and contribute to the definition, development, integration, test, documentation, and support across multiple platforms
• Serve as a process expert and provide architectural leadership to Jr engineers as needed
• Work closely with technical leads and client teams to fully demonstrate the benefits of GCP technology•
• Introduce clients to data architecture and analytics best practices
• Solve some of the most challenging, high-scale data, and IoT problems (banking, telco, energy, retail, etc.).
• Write high-quality and testable batch and real-time data pipelines
• Work with Data Scientists to design pipelines that improve their productivity and enable them to implement their ideas
• Support clients in troubleshooting issues in their test and/or production environments and identifying root causes and solutions

You have
• Mentoring and helping the team build technical capability
• Ability to effectively communicate data insights and negotiate options at senior management levels
• Experience in leading significant project steps and communicating progress/approach with technical/non-technical peers/clients and leaders
• GCP experience ( for junior candidates, or exceptional talent with other cloud experience, we will provide training programs)
• Experience in large-scale, secure, and high-availability solutions in a Cloud environment, such as GCP.
• Experience with one Dataflow or Spark
• Experience with orchestration frameworks such as Airflow, Kubeflow, Azkaban, etc.
• Extensive programming experience in Python, Java and/or Scala.
• Good understanding of modern data architecture
• Good understanding of GCP services such as IAM, and Google Cloud Storage buckets
• Experience with business intelligence tools like QuickSight, Looker and Data Studio
• Technical writing experience; preparing and presenting technical material to a variety of audiences.
• Experience in working in or with, Agile delivery teams.

Nice to have
• Streaming experience with Dataflow, Spark, Flink, Kafka Streams, etc
• Strong understanding of Data Governance principles and experience working with tools such as Collibra or Immuta