Senior Data Engineer

ZipRecruiter; Vancouver; On-site; CAD 100,000 - 140,000; Full time

Job summary

A leading company is seeking a Senior Data Engineer to join their team in Vancouver. The ideal candidate will have extensive experience in data engineering, leading teams, and working with diverse datasets. Responsibilities include designing data pipelines, troubleshooting data services, and driving process improvements. A Bachelor's degree in Computer Science and a strong background in big data technologies are required.

Qualifications

• 8+ years of industry experience in Data Engineering.
• Experience with big data and analytics programming.
• Technical team leadership and mentorship experience.

Responsibilities

• Design and build efficient data pipelines.
• Troubleshoot and resolve data service problems.
• Help drive transformation and optimize processes.

Skills

Data Engineering

Python

R

Spark

EventHub

Kafka

Data Processing

Data Transformation

Technical Leadership

Education

Bachelor's Degree in Computer Science

Tools

Azure Data Stack

Databricks

C#/.Net

Java

Job description

Job Description

Our client has an immediate opening for a Senior Data Engineer to join their team.

An ideal candidate will have experience leading and mentoring junior and intermediate team members, working with business stakeholders to gather/understand requirements, designing and building efficient & reliable data pipelines, cleansing and normalizing diverse datasets for analytics, troubleshooting and resolving problems of data services that support business & applications systems and provide solutions to pro-actively prevent problems from happening while utilizing technologies and tools such as Azure Data Stack, Python, R, Spark, EventHub, Kafka, and have solid experience in Financial Services domain.

A typical day would involve:

• Design and build efficient & reliable data pipelines, cleanse, and normalize diverse datasets for Analytics.
• Troubleshoot and resolve problems of data services that support business & applications systems and provide solutions to pro-actively prevent problems from happening.
• Create and manage knowledgebase, access controls, deployment pipeline architecture and infrastructure related to Data Platform.
• Help drive transformation by continuously looking for ways to automate existing processes and optimize for efficiency and data quality, redesigning infrastructure for greater scalability, etc
• Represent team as a subject matter expert for strategic projects and contribute to end-to-end data architecture solutions.

You are:

• Customer-centric: You define and measure success through the eyes of your internal customers and anticipate, understand and respond to their evolving needs. Demonstrate strong follow through and consistently keep commitments to members and team.
• Results-oriented: You have a demonstrated ability to engage stakeholders to elicit requirements, analyze findings, generate appropriate solutions, and build deliverables all while securing the team's commitment to these solutions.
• Technology savvy: You have education and/or experience supporting technology development/implementation efforts.
• Curious & innovative: You always do your research and analysis to assess and recommend a course of action based on industry, political and economic trends, you can identify risks and opportunities that may impact the business unit and or designated area.

You have:

• Bachelor's Degree in Computer Science or a related technical discipline.
• 8+ years of industry experience in Data Engineering or related field handling data processing and transformation of disparate datasets
• Experience with big data and analytics programming such as Python, R, Spark in Databricks
• Experience with batch and real-time processing frameworks (EventHub, Apache Spark, Apache Kafka, etc.)
• Recent full stack or back end development experience with C#/.Net and/or Java
• Technical team leadership and mentorship experience
• Experience working with business stakeholders to gather/understand requirements
• Willingness to work in a highly flexible environment with multiple competing priorities.
• Proven ability to innovate and adapt to the latest development in area of expertise. Ability to adapt and grasp new skills and content.

Bonus points, if you have:

• Experience in design and implementation of data integrations in Azure data stack such as Azure Data Lake, Azure Data Factory, Azure Databricks, EventHub.
• Exposure to DevSecOps practices to incorporate security first principles.
• Good understanding of data storage, data mapping and modelling, real-time data handling