Senior Data Engineer (AWS, Big Data, ETL, Netezza, Spark, Kafka, Data Warehousing)

Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!; Toronto; On-site; CAD 120,000 - 180,000; Full time

Job summary

Astra-North Infoteck Inc. is seeking a Senior Data Engineer to design, build, and optimize scalable data pipelines on AWS. You will work on data platform modernization, cloud migrations, and real-time data processing in a fast-paced enterprise environment.

We require 10+ years in Data Engineering, with hands-on Netezza experience and strong AWS skills (EMR, S3, Redshift, Glue). Proficiency in SQL, Python, and Airflow is highly valued.

Qualifications

• 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
• Netezza experience is mandatory.
• Strong experience with AWS Cloud services (EMR, S3, EC2, Redshift, Glue).
• Experience with ETL/ELT tooling (SSIS, Pentaho) and data warehousing.

Responsibilities

• Design, develop, maintain, and support scalable data pipelines and distributed systems.
• Lead data platform migrations from On-Prem to AWS, ideally with Netezza background.
• Develop and manage ETL/ELT processes using SSIS, Pentaho, or similar tools.
• Implement and manage AWS services (EMR, S3, EC2, Redshift, Glue, Airflow).
• Build and optimize data workflows and orchestration pipelines using Airflow.
• Work with real-time streaming technologies (Kafka, Spark Streaming).
• Ingest, transform, and validate data from multiple sources; optimize SQL queries.
• Monitor performance, troubleshoot issues, and ensure reliability.

Skills

SQL

Shell scripting

Python

Unix/Linux

Tools

AWS (EMR, EC2, S3, Redshift, Glue)

Airflow

Netezza

SSIS

Pentaho

Job description

Senior Data Engineer | AWS Cloud | Big Data | ETL | Netezza | Spark | Kafka | Data Platform Modernization

Job Summary

The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands‑on experience in Big Data ecosystem tools, AWS services, and real‑time data processing.

This role involves working on data platform modernization, cloud migrations, Data Warehousing, and ETL in a fast‑paced enterprise environment.

Required Technical Skills

Cloud Technologies

Strong experience in AWS Cloud services:

• EMR
• EC2
• S3
• VPC
• RDS
• Redshift
• AWS Glue
• IAM
• CloudWatch
• CloudFormation

Experience with:

• Airflow (or AWS Managed Workflows)

Databases

Experience working with:

• Netezza (Mandatory)

SQL-based systems:

• SQL Server
• PostgreSQL

Data warehouses:

• Teradata
• Redshift
• Netezza

ETL Tools

Hands‑on experience with:

• SSIS
• Pentaho or similar ETL tools

Programming & Scripting

Strong proficiency in:

• SQL
• Shell scripting

Good to have:

• Python

Operating Systems

Strong experience in Unix/Linux environments.

Key Qualifications

• 10+ years of experience in Data Engineering / Big Data / Platform Engineering.

Key Responsibilities

• Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
• Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
• Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
• Implement and manage AWS services such as:
  • EMR
  • S3
  • EC2
  • Redshift
  • Glue
  • Airflow
• Build and optimize data workflows and orchestration pipelines using Airflow.
• Work with real-time streaming technologies such as:
  • Kafka
  • Spark Streaming
• Perform data ingestion, transformation, and validation from multiple data sources.
• Optimize SQL queries for performance and scalability.
• Monitor system performance, troubleshoot issues, and ensure system reliability.
• Collaborate with cross‑functional teams including:
  • Developers
  • Architects
  • Infrastructure teams.