Data Engineer

Affinity; London; On-site; CAD 90,000 - 110,000; Full time

Job summary

A technology and business consulting company in Ontario is looking for a skilled Data Engineer to design and maintain robust data infrastructure. You will develop scalable data pipelines and collaborate with cross-functional teams effectively. Candidates should have over 5 years of experience in data engineering and a deep knowledge of programming and database management. A bachelor’s or master’s degree in a relevant field is required. This role offers a competitive salary and the opportunity to work on innovative data-driven projects.

Qualifications

• 5+ years of experience in data engineering, software development, or related roles.
• Proficiency in programming languages commonly used in data engineering.
• Strong knowledge of database systems and data modeling techniques.

Responsibilities

• Design, build, and optimize scalable end-to-end data pipelines.
• Develop robust ETL/ELT processes and API integrations.
• Implement data validation and quality frameworks.

Skills

Data pipeline optimization

Python

SQL

ETL tools

Data governance

Collaboration skills

Problem-solving

Education

Bachelor's or Master’s degree in computer science, Engineering or related field

Tools

Databricks

Azure Data Factory

Hadoop

Spark

Kafka

AWS

Snowflake

Job description

Job Description

On behalf of our client, Affinity is seeking a Data Engineer who will be responsible for designing, building, and maintaining secure, stable, and scalable data pipelines and infrastructure that support data driven products and analytics initiatives. In this role, you will collaborate closely with cross functional teams, including data analysts, data scientists, software engineers, and product managers, to ensure the seamless flow, quality, and accessibility of data across the organization.

The ideal candidate is passionate about working with large scale datasets and will play a critical role in transforming raw data into actionable insights that drive informed business decisions.

Responsibilities

• Design, build, and optimize scalable end-to-end data pipelines for processing large volumes of structured and unstructured data
• Develop robust ETL/ELT processes and API integrations to seamlessly integrate data from diverse sources
• Implement data validation, quality frameworks, and monitoring systems to ensure accuracy and reliability
• Define and maintain company data assets, data models, and associated Spark/SparkSQL jobs
• Design efficient data models, schemas, and database structures to support analytical and operational use cases
• Evaluate and implement data storage solutions including databases, data lakes, and cloud-based services
• Configure and manage data infrastructure components while optimizing for performance and scalability
• Develop long-term data platform architecture strategy and evaluate tools for data lineage and metadata management
• Build reliable integrations with internal/external data sources and develop RESTful APIs for data access
• Implement data security protocols, access controls, and governance policies
• Ensure system compatibility and interoperability across platforms
• Partner with cross-functional teams (data scientists, analysts, engineers, product managers) to deliver tailored solutions
• Improve data models that feed BI tools to increase data accessibility across the organization
• Create technical documentation and provide troubleshooting support for data-related issues and maintain robust ETL/ELT pipelines to ingest, transform, and load data from various sources (e.g., DMV records, insurance databases, auction feeds)

Qualifications

• 5+ years of experience in data engineering, software development, or related roles
• Proficiency in programming languages commonly used in data engineering (e.g., Python, Java, Scala)
• Strong knowledge of database systems, data modeling techniques, normalization, and performance optimization with advanced SQL proficiency
• Hands-on experience with ETL tools commonly used in data engineering (e.g., SSIS, Databricks, Azure Data Factory)
• Experience with big data technologies and frameworks (e.g., Hadoop, Spark, Kafka)
• Experience with cloud platforms and services (e.g., AWS, Azure, Google Cloud Platform)
• Experience with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery)
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills in a team-oriented environment
• Experience with machine learning operations (MLOps) and supporting ML model deployment pipelines is considered an asset
• Knowledge of data governance frameworks and compliance requirements (GDPR, CCPA, etc.) is considered an asset
• Experience with containerization technologies (Docker, Kubernetes) is considered an assetExperience with real-time streaming data processing is considered an asset
• Knowledge of data visualization tools and analytics platforms is considered an asset
• Bachelor's or master’s degree in computer science, Engineering, or related field

Affinity Earn

Know someone who’s great for this, or any of our open roles? Earn up to $4,000/year for each successful referral through Affinity Earn. You can also earn up to $50,000 for helping us find new clients. Learn about our referral program at https://affinity-group.ca/earn/ or browse our jobs & follow us at https://www.linkedin.com/company/affinity-staffing/jobs/

About Affinity

Affinity Group is a technology and business consulting and services company. We believe in creating long term relationships between clients and consultants that foster a mutually beneficial partnership. Affinity is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All employment is decided on the basis of qualifications, merit and business need.

For more information on Affinity, please visit www.affinity-group.ca

Job Number: 13013