Senior Data Engineer

Live Assets; Toronto; On-site; CAD 100,000 - 140,000; Full time

Job summary

Live Assets is seeking a Senior Data Engineer for a leading Veterinary Provider in Toronto. The role involves developing efficient data pipelines, optimizing data processes, and working with the latest technologies in an agile environment. Ideal candidates will have extensive experience in data engineering, SQL, and Azure data stack, along with strong problem-solving and communication skills.

Qualifications

• 10+ years of hands-on experience in data engineering or related technical field.
• Strong proficiency in data warehousing, data modelling, and ETL development.
• Experience with advanced SQL queries and Microsoft Azure data stack.

Responsibilities

• Develop and implement scalable end-to-end data pipelines.
• Collaborate with data producers and consumers for product requirements.
• Write advanced SQL queries for data extraction and analysis.

Skills

Data warehousing

Data modelling

ETL development

SQL

Python

Azure Synapse Analytics

Linux/Unix shell scripting

Problem-solving

Communication

Job description

IT Jobs in Canada

Job Description
Live Assets | IT Staffing Solutions is hiring a Senior Data Engineer to join our client, a leading Veterinary Provider.

Key Responsibilities

• Develop and implement scalable end-to-end data pipelines using modern data processing and orchestration techniques, for both batch and streaming data.
• Lead collaboration with data producers and data consumers to identify clear product requirements and structure an effective implementation of solutions, guiding the overhaul of pre-existing methods, when necessary.
• Write advanced SQL queries for data extraction, transformation, and analysis from structured and semi-structured sources.
• Identify, evolve, and champion data/software engineering best practices and AI automations, as they best fit a small and agile team.
• Evaluate and recommend new technologies to enhance data infrastructure, optimize data processes, and improve overall efficiency.
• Develop Python scripts to automate data workflows, conduct data validation, and perform complex transformations.
• Optimize performance of large-scale datasets in Azure Synapse, including partitioning, indexing, and workload management.
• Provide technical guidance on improving data flows, storage solutions, and reporting.

Necessary Skills

• 10+ years of hands-on experience in data engineering or related technical field with a strong focus on data.
• Strong proficiency in data warehousing, data modelling and ETL development.
• Experience in SQL, including stored procedures, CTEs, indexing strategies and performance tuning.
• Advanced OOP Python programming skills, particularly in data processing.
• Strong expertise in MS Azure data stack (preferred Azure Synapse Analytics) and the MS Office Suite of applications.
• Proficiency in Linux/Unix shell scripting and system troubleshooting.
• Strong attention to detail and ability to work with complex data.
• Excellent problem-solving, communication, and documentation skills.

About Live Assets | Leaders in IT Excellence | Greater Toronto Area

Live Assets is a premier IT agency dedicated to delivering excellence in tech solutions across various industries.

Our agency is renowned for its expertise in engineering, and we take pride in our track record of delivering outstanding IT projects with precision and innovation.

With a wide spectrum of clients spanning different industries, we cater to both permanent and contract IT opportunities.

Discover more about our agency and stay updated on our latest developments through our social media channels:

LinkedIn–Twitter–Facebook–Youtube–Glassdooor