Lead Software Engineer (Python | Databricks)

Scotiabank; Toronto; On-site; CAD 90,000 - 140,000; Full time

Benefits offered by this job

Diversity & Inclusion programs

Accessibility & accommodations

Upskilling through courses & tuition

Competitive rewards including bonus &

Community engagement

Job summary

Scotiabank's Global Banking and Markets Engineering team is seeking Software Engineers with a strong engineering mindset to enhance capital markets analytics platforms. You will collaborate with business partners, architect solutions, and own end-to-end delivery from data modeling to production readiness.

You will work across data pipelines, ETL, and large-scale applications, applying agile methods and modern DevOps practices in a fast-paced financial tech environment in Canada.

Qualifications

• 4+ years Python development including PySpark and Pandas.
• 4+ years in designing scalable ETL pipelines and data modeling.
• 4+ years building, deploying, and maintaining large-scale apps with complex integration.
• 4+ years reading/writing SQL and managing databases (Postgres, MSSQL, Oracle).
• Experience with data tech and large datasets; familiar with Spark, Kafka, Elasticsearch.
• Hands-on with DataBricks or Snowflake; cloud platforms (Azure/GCP).
• Experience with Airflow or Data Factory; CI/CD with Jenkins and Git; API/Web Services.
• Agile environment participation; knowledge of Test-Driven Development; Bachelor's in CS/Engineering.

Responsibilities

• Collaborate with end users and business teams to capture needs and assess requirements.
• Partner with Architecture on design and integration initiatives.
• Deliver and improve solutions, covering full project life cycle and specs.
• Design data models, schemas, and data pipelines; ensure scalable architectures.
• Create test sets and automate functional and non-functional testing; performance tests.
• Write deployment scripts; utilize CI/CD pipelines to promote code.
• Support production systems and address issues; optimize end-user experience.
• Refactor legacy code when needed; mentor end users and participate in SCRUM.

Skills

Python

ETL pipelines

Data modeling

SQL

Big data

Spark

Kafka

DevOps

Agile

REST APIs

Education

Bachelor's degree in CS/Engineering

Tools

Postgres

MS SQL

Oracle DB

Apache Spark

Apache Kafka

Elasticsearch

DataBricks

Snowflake

Azure

GCP

Airflow

Data Factory

Jenkins

Git

Docker

Kubernetes

REST

Job description

Requisition ID: 258694

Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.

The Team

Global Banking and Markets Engineering (GBME) is the fast-moving, award-winning technology engine that powers Scotiabank’s Corporate, Investment Banking and Capital Markets businesses.

The Role

GBME is searching for Software Engineers who are continuous learners and are eager to boost capabilities of capital markets products and analytics platforms.

The successful candidate will have a strong technical and engineering mindset. You will work closely with business teams and cross functional engineering leaders to develop and implement market leading solutions for the various projects within GBME.

Is this role right for you? In this role you will:

• Work closely with end users, and business teams to understand business needs, conduct gap analysis and requirements assessments as needed
• Partner with Architecture on the design and integration
• Deliver and enhance new and current solutions and applications
• Be involved in the full life cycle of a project, responsible for the design, producing actionable and complete specifications, writing production quality code, and implementation
• Design, deliver and implement data models, database schema and data pipelines
• Design test sets and automate where possible, including functional and non functional testing, regression testing as well as performance testing
• Write deployment scripts and leverage tools such as the Accelerator pipeline to promote code
• Support the solution in production (as needed)
• Improve the end user experience by finding scalable solutions and addressing complex ad hoc queries.
• Effectively communicate with technical and non-technical audiences
• Able to work with and refactor legacy code (as needed).
• Coach and train end users (as needed).
• Participate and run SCRUM Meetings (as needed).

Skills

Do you have the skills that will enable you to succeed in this role? We’d love to work with you if you have:

• 4+ years’ experience with Python (PySpark, Pandas),
• 4+ years’ experience with designing scalable ETL pipelines & data modeling
• 4+ years’ experience developing, deploying, and maintaining large scale applications with complex integration
• 4 + years’ experience reading, writing, and running SQL scripts.
• Working experience with data technologies and databases, preferably in one or more of: Postgres, MSSQL, Oracle DB, Apache Spark, Apache Kafka, Apache ElasticSearch, comfortable working with large datasets
• Working experience with data platforms: DataBricks or Snowflake
• Working experience with cloud platforms (Azure/GCP)
• Working experience with workflow orchestration tool such as Airflow or Data Factory
• Working experience with Web Services & API
• Hands-on experience with DevOps: CI/CD (Jenkins, Git)
• Experience working in an Agile environment, with experience and understanding of Test-Driven Development methodology.
• Bachelor’s degree in computer science, Engineering or relevant experience.

Preferred Qualifications (Nice To Have)

• Understanding of networking protocols and security principles
• Capital Markets product knowledge
• Dockers, Kubernetes experience
• Experience working with real-time, high availability and low latency systems
• Experience in multi-threaded applications

What's in it for you?

• Diversity, Equity, Inclusion & Allyship - We strive to create an inclusive culture where every employee is empowered to reach their fullest potential, respected for who they are, and are embraced through bias-free practices and inclusive values across Scotiabank. We embrace diversity and provide opportunities for all employee to learn, grow & participate through our various Employee Resource Groups (ERGs) that span across diverse gender identities, ethnicity, race, age, ability & veterans.
• Accessibility and Workplace Accommodations - We value the unique skills and experiences each individual brings to the Bank and are committed to creating and maintaining an inclusive and accessible environment for everyone. Scotiabank continues to locate, remove, and prevent barriers so that we can build a diverse and inclusive environment while meeting accessibility requirements.
• Upskilling through online courses, cross-functional development opportunities, and tuition assistance.
• Competitive Rewards program including bonus, flexible vacation, personal, sick days, and benefits will start on day one.
• Community Engagement - no matter where you choose to work from; we offer opportunities for community engagement & belonging with our various programs.

Location(s): Canada : Ontario : Toronto

Scotiabank is a leading bank in the Americas. Guided by our purpose: "for every future", we help our customers, their families and their communities achieve success through a broad range of advice, products and services, including personal and commercial banking, wealth management and private banking, corporate and investment banking, and capital markets.

At Scotiabank, we value the unique skills and experiences each individual brings to the Bank, and are committed to creating and maintaining an inclusive and accessible environment for everyone.