Lead Data Engineer – Ottawa

IQ PARTNERS Inc; Ottawa; On-site; CAD 100,000 - 130,000; Full time

Job summary

A leading recruitment firm in Ottawa seeks a Lead Data Engineer to lead a team in developing big data solutions. You will mentor junior engineers and write efficient code in an agile environment. Candidates should have 5+ years of experience in data engineering and familiarity with big data tools like Spark and Hadoop. A valid Secret-level government security clearance is required.

Qualifications

• 5+ years of experience in a data engineering role, with 2 years in a lead role.
• Proficiency in programming languages relevant to data engineering.
• Experience with production level data processing systems.

Responsibilities

• Lead and mentor a team of junior data engineers.
• Write efficient code for data processing systems.
• Configure and deploy software using big data tools.

Skills

Scala

Java

Python

Data Engineering

Agile Methodologies

Tools

Spark

Hadoop

Elasticsearch

Git

Jenkins

Docker

Job description

Overview

The Quantexa Delivery team is all about bringing context to data. Lead Data Engineers are drivers within the fast-paced delivery teams, implementing Quantexa's innovative technology for an ever-expanding list of domains including banking, insurance, government, healthcare. They are problem solvers, delivering sophisticated end-to-end big data solutions to the internal and external client stakeholders. As a central part of the team, you will experience opportunities beyond that of an engineer. You enjoy leading by example, guiding and mentoring data engineers to solve complex technical problems, sharing success with the team and taking pride in achieving milestones together. You share success with the team and take pride in the sum of its achievements. You will balance your time engaging with the business teams and clients to break down requirements into technical tasks, working hands-on as a programmer, and supporting the work of other engineers within a collaborative environment.

What you\'ll be doing

• You will lead, support, and mentor a team of junior data engineers within an agile framework.
• Write defensive, fault tolerant and efficient code for production level data processing systems.
• You\'ll configure and deploy Quantexa\'s software using tools such as Spark, Hadoop, Scala, Elasticsearch, with the platform being hosted on both private and public virtual clouds, such as Google cloud, Microsoft Azure and Amazon.
• You will be a trusted source of knowledge to your clients. You will articulate technical concepts to a non-technical audience helping them to make big decisions.
• Collaborate with both the solution architects and the R&D engineers to champion solutions and standards for complex big data challenges. You proactively promote knowledge sharing and ensure best practice is followed.

What you\'ll bring

• You\'ll have a background in hands-on technical development, with at least five years of industry experience in a data engineering role or engineering equivalent, and at least 2 years of experience acting in a lead role within a data or software engineering team.
• Proficiency in Scala, Java, Python, or a programming language associated with data engineering. The primary language is Scala, but don\'t worry if that\'s not currently your strongest language, we believe that strong engineering principles are universal and transferrable.
• You\'ll have expertise building and deploying production level data processing batch systems maintained by application support teams. Importantly you\'ll share an appreciation of what makes a high quality, operationally stable system and how to streamline all areas of development, release, and operations to achieve this.
• Experience with a variety of modern development tooling (e.g. Git, Gradle, Nexus) and technologies supporting automation and DevOps (e.g. Jenkins, Docker and a little bit of good old Bash scripting). You\'ll be familiar with developing within a version-controlled process that regularly makes use of these tools and technologies.
• A strong technical communication ability with demonstrable experience of working in rapidly changing client environments.
• Knowledge of testing libraries of common programming languages (such as ScalaTest or equivalent). Knows the difference between different test types (unit test, integration test) and can cite specific examples of what they have written themselves.
• Avalid Secret-level government security clearance, either current or in the past 12 months, is a mandatory requirement for this position.

Nice to haves

• Experience growing and up-skilling team members as a career mentor, technical line manager, or acting in an equivalent guidance role.
• A passion for expanding the business - and leading initiatives to do so.

IQ PARTNERS is committed to operating an inclusive, barrier-free recruitment and selection process, and we encourage candidates of every race, gender, age, religion, identity, and experience to apply for this position. We’d be happy to provide accommodation for any candidate that requires assistance due to a disability or medical need. Please contact us at 416-599-4700 or by email at [e-mail] to discuss specific accommodations.

Please upload the most recent version of your resume in DOC, DOCX, RTF, PDF or HTML file formats (maximum size: 400kb)