Lead Data Engineer

Placements24; Kimberley; Hybrid; CAD 120,000 - 160,000; Full time

Job summary

Placements24 in Kimberley, Canada, seeks a Lead Data Engineer to drive data infrastructure development, architecture scalable pipelines, and manage data warehousing for analytics and ML initiatives.

You will lead a team of data engineers, mentor juniors, and collaborate with data scientists and business analysts to deliver impactful data solutions. Hybrid work arrangement with a Kimberley office is offered.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 7+ years of experience in data engineering or similar.
• Proven design/implementation experience for large-scale data pipelines/ETL.
• Strong SQL and DB tech (PostgreSQL, Snowflake, Redshift).
• Proficiency in Python or Scala.
• Hands-on with big data (Spark, Hadoop).
• Experience with cloud data services (AWS/Azure/GCP).
• Knowledge of data modeling and warehousing concepts.
• Excellent leadership, comms, and problem-solving.
• Experience in Agile environments.

Responsibilities

• Design and maintain scalable data pipelines using ETL/ELT tools and languages.
• Develop and manage data warehousing solutions and ensure performance.
• Architect data models for BI and analytics.
• Collaborate with data scientists and analysts to provide high-quality datasets.
• Optimize data processing jobs for efficiency and cost.
• Manage cloud-based data infrastructure (AWS/Azure/GCP).
• Implement data governance, security, and compliance.
• Lead and mentor a team of data engineers.
• Evaluate new data technologies and methods.
• Troubleshoot complex data issues.

Skills

Leadership

SQL

Python/Scala

ETL/ELT design

Spark/Hadoop

Cloud platforms

Data modeling

Data warehousing

Agile

Communication

Education

Bachelor's or Master's degree in CS/Engineering

Tools

Airflow

dbt

Snowflake

Redshift

PostgreSQL

Kafka

Job description

About the Role

Our client, a prominent data analytics firm, is seeking a Lead Data Engineer to drive the development and maintenance of their data infrastructure in Kimberley . This senior role involves architecting and building scalable data pipelines, managing data warehousing solutions, and ensuring the quality and accessibility of data for analytics and machine learning initiatives. The ideal candidate will have extensive experience with big data technologies, cloud platforms, and data modeling. You will lead a team of data engineers, mentor junior members, and collaborate closely with data scientists and business analysts to deliver impactful data solutions. This is an excellent opportunity to shape the data strategy of a growing organization. Join our innovative team in Kimberley .

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines using various ETL/ELT tools and programming languages.
• Develop and manage data warehousing solutions, ensuring data integrity and performance.
• Architect and implement data models that support business intelligence and analytical requirements.
• Collaborate with data scientists and analysts to understand data needs and provide them with accessible, high-quality datasets.
• Optimize data processing jobs for efficiency, cost, and speed.
• Manage and maintain cloud-based data infrastructure (e.g., AWS, Azure, GCP).
• Implement data governance policies and ensure data security and compliance.
• Lead and mentor a team of data engineers, fostering a collaborative and productive environment.
• Evaluate and adopt new data technologies and methodologies to enhance data capabilities.
• Troubleshoot and resolve complex data-related issues.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• Minimum of 7 years of experience in data engineering or a related role.
• Proven experience designing and implementing large-scale data pipelines and ETL/ELT processes.
• Expertise in SQL and experience with various database technologies (e.g., PostgreSQL, Snowflake, Redshift).
• Strong proficiency in programming languages commonly used in data engineering, such as Python or Scala.
• Hands-on experience with big data technologies (e.g., Spark, Hadoop ecosystem).
• Experience with cloud platforms (AWS, Azure, or GCP) and their data services.
• Knowledge of data modeling techniques and best practices.
• Experience with data warehousing concepts and tools.
• Excellent leadership, communication, and problem-solving skills.
• Experience working in an Agile environment.

Benefits

Our client offers a highly competitive salary, performance bonuses, and comprehensive health benefits. Employees receive a generous retirement contribution and ample paid time off. We provide significant opportunities for professional development, including budgets for training, certifications, and conference attendance. This role offers a hybrid work arrangement, allowing for flexibility. The office is conveniently located in Kimberley .