Big Data Automation Lead

RBC; Toronto; Hybrid; CAD 100,000 - 130,000; Full time

Job summary

RBC is looking for a highly skilled Big Data Automation Lead to spearhead automation initiatives on its data platforms. The ideal candidate will leverage experience in Python, Pandas, and PySpark to streamline big data processes and ensure the robustness of automation solutions.

Benefits

Comprehensive Total Rewards Program

Flexible work/life balance options

World-class training program

Opportunities to make a difference

Qualifications

• 4-5+ years experience as a Big Data Automation Lead.
• Strong programming skills in Python with knowledge of Pandas and PySpark.
• Hands-on experience with automation frameworks such as Robot Framework.

Responsibilities

• Lead the design and implementation of automation solutions.
• Develop and maintain automation scripts using Python and PySpark.
• Implement automated API testing and manage data processes on HDFS and Hadoop.

Skills

Python

Pandas

PySpark

SQL

Critical Thinking

Decision Making

Test Automation

Job description

Job Summary

Job Description

WHAT IS THE OPPORTUNITY?

We are seeking a highly skilled Big Data Automation Lead with 4-5+ years of experience to drive automation initiatives across our big data platforms. The ideal candidate will have a strong background in Python, Pandas, PySpark, and automation frameworks, with hands-on experience in big data technologies and data engineering best practices.

WHAT WILL YOU DO?

• Lead the design, development, and implementation of automation solutions for big data pipelines and workflows.

• Develop and maintain robust automation scripts using Python, Pandas, and PySpark.

• Implement and manage test automation using Robot Framework and BDD frameworks.

• Design and execute automated API testing and validation.

• Write and optimize complex SQL queries for data validation and transformation.

• Work with UNIX-based systems for scripting, automation, and job scheduling.

• Manage and automate data processes on HDFS, Hive, Hadoop, and Spark environments.

• Collaborate with data engineers, QA, and business stakeholders to ensure data quality and process efficiency.

• Troubleshoot and resolve issues in data pipelines and automation frameworks.

WHAT DO YOU NEED TO SUCCEED?

Must have:

• Strong programming skills in Python, with experience in Pandas and PySpark.

• Hands-on experience with Robot Framework and BDD (Behavior Driven Development) frameworks.

• Experience with API testing and automation.

• Advanced SQL skills for data manipulation and validation.

• Experience working in UNIX environments (shell scripting, process automation).

• Solid understanding of HDFS, Hive, Hadoop, and Spark.

• Strong problem-solving and analytical skills.

• Excellent communication and leadership abilities.

Nice-to-have:

• Experience in leading automation teams or projects.

• Exposure to CI/CD tools and cloud-based big data platforms.

• Experience with GitHub Actions for CI/CD automation.

• Experience with Ansible Automation Platform for infrastructure and workflow automation.

WHAT's IN FOR YOU?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

• A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable

• Leaders who support your development through coaching and managing opportunities

• Ability to make a difference and lasting impact

• Work in a dynamic, collaborative, progressive, and high-performing team

• A world-class training program in financial services

• Flexible work/life balance options

• Opportunities to do challenging work

#LI-HYBRID

#LI-POST

Job Skills

Critical Thinking, Decision Making, Detail-Oriented, DevOps, IT Quality Assurance, Long Term Planning, Process Management, Risk Management, Software Product Testing, Test Automation

Additional Job Details

Address:

RBC CENTRE, 155 WELLINGTON ST W:TORONTO

City:

TORONTO

Country:

Canada

Work hours/week:

37.5

Employment Type:

Full time

Platform:

TECHNOLOGY AND OPERATIONS

Job Type:

Regular

Pay Type:

Salaried

Posted Date:

2025-06-09

Application Deadline:

2025-06-30

Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above

Inclusion and Equal Opportunity Employment

At RBC, we believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.

Join our Talent Community

Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.

Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.