Senior Data Engineer: PySpark & Scalable Data Pipelines

RBC; Halifax; On-site; CAD 100,000 - 130,000; Full time

Job summary

A major financial institution in Halifax is seeking a skilled professional to join their Compliance Technology team. The role involves designing and maintaining data pipelines using Python and PySpark while ensuring compliance with trading systems. The ideal candidate has extensive experience in data management and cloud technologies. This position offers a comprehensive rewards program and flexible work options.

Benefits

Comprehensive Total Rewards Program

Competitive compensation

Flexible work/life balance options

Qualifications

• 8+ years of experience in Python and Java.
• Proven ability to develop and troubleshoot large-scale applications.
• Experience as a technology lead, applying best engineering practices.

Responsibilities

• Design and maintain scalable data pipelines using Python and PySpark.
• Optimize data workflows and processes for performance.
• Collaborate with clients to analyze business requirements.

Skills

Python

Java

Big Data technologies

REST APIs

Agile methodologies

Tools

Hadoop

Spark

PySpark

CI/CD pipelines

Docker

Kubernetes

Job description

A major financial institution in Halifax is seeking a skilled professional to join their Compliance Technology team. The role involves designing and maintaining data pipelines using Python and PySpark while ensuring compliance with trading systems. The ideal candidate has extensive experience in data management and cloud technologies. This position offers a comprehensive rewards program and flexible work options.