Sr. Python/Pyspark Developer

PCRecruiter - Recruitment Software & Applicant Tracking System; Toronto; Hybrid; CAD 90,000 - 120,000; Full time

Job summary

A large IT services firm seeks a Sr. Python/Pyspark Developer for a permanent full-time role involving heavy data engineering. The position emphasizes architecting scalable data solutions while collaborating with agile teams, requiring extensive experience in Python, Pyspark, SQL, and data integration workflows.

Qualifications

• At least 10 years of experience in Python development.
• 5+ years of hands-on experience with Python and Pyspark.
• Experience in agile development and software architecture.

Responsibilities

• Design and maintain robust Python and Pyspark applications.
• Implement unit testing and code quality checks.
• Collaborate with cross-functional teams for data solutions.

Skills

Python

Pyspark

SQL

Data Engineering

ETL

Agile Development

Tools

Pytest

Pylint

Django

Flask

Tornado

SSIS

Git

Job description

Sr. Python/Pyspark Developer

This is with a large IT services firm for a major bank.
Candidate must have legal work status for Canada.
Client Location is Mississauga, Ontario.
Hybrid - In-Office 3 days per week.
Perm/FT role, salary and benefits

Python, Pyspark, SQL, Data Engineering.

Job Description:
Note: This role is heavy on data engineering and not app dev
As a senior Python and PySpark developer, your challenge is to architect and deliver scalable, high-performance data processing solutions that empower data-driven decision-making. You will tackle complex data integration, transformation, and analytics workloads by designing robust applications, optimizing SQL queries, and streamlining workflows. Your expertise will enable the organization to leverage cloud-scale data platforms effectively, maintain high code quality through rigorous testing, and foster collaboration within agile teams to innovate and maintain cutting-edge data solutions.

Responsibilities:
• Design, develop, and maintain robust Python and PySpark applications to support large-scale data processing, transformation, and analytics initiatives.
• Implement and execute unit testing and code quality checks using tools like Pytest and Pylint to ensure reliable and maintainable codebases.
• Design, optimize, and troubleshoot complex SQL queries, primarily in tSQL, to support data extraction, transformation, and loading processes.
• Develop and manage data integration workflows utilizing ETL/ELT frameworks, enterprise schedulers, and scripting in Python and other relevant tools.
• Collaborate with cross-functional teams to gather requirements, create high-level system specifications, and produce detailed design documents for data solutions.
• Implement RESTful API integrations to connect various data sources and applications, ensuring seamless data flow and accessibility.
• Lead and contribute to agile development processes, participating in sprint planning, stand-ups, and code reviews to ensure timely delivery of features.
• Leverage frameworks such as Django, Flask, Pyramid, or Tornado to build scalable web applications or APIs as needed.
• Utilize version control systems like Git, and manage development pipelines to support continuous integration and deployment.
• Apply best practices in software architecture, data structures, and algorithms to optimize application performance and scalability.
• Coordinate with enterprise teams on scheduling and automation tasks using enterprise schedulers and SSIS, enhancing operational efficiency.
• Lead technical discussions and influence stakeholders to adopt best practices and innovative solutions for data processing and application development.
• Maintain thorough documentation of system architecture, design specifications, and coding standards to support ongoing maintenance and knowledge sharing.
• Stay updated on emerging technologies and industry trends to recommend improvements and ensure the solutions remain cutting-edge.

Requirements:
• At least 10 years of Experience in Python development
• Must have hands-on experience in Python and Pyspark
• Experience in Python testing and code analysis tools like Pytest and Pylint
• At least 5 years of fulltime development experience using Python.
• At least 5 years working with tSQL, XML and enterprise schedulers
• Proficient understanding of algorithms, data structures, architectural design patterns and best practices
• Extensive experience in software development and the entire SDLC.
• Solid understanding of a variety of programming tools and development platforms.
• Excellent organizational and leadership abilities.
• Highly analytical mindset, with an ability to see both the big picture and the details.
• Experience in creating high level product specifications and design documents.
• Proven collaboration and influencing skills
• Experience in writing Python applications with the benefit of frameworks like Django, Flask,
• Pyramid, or Tornado.
• Integration experience (ETL, ELT) with Python
• Familiarity with SSIS would be helpful
• General development expertise, use of version control, ticketing, and continuous integration systems.
• Experience in an Agile Development environment
• Experience in consuming API's (i.e. Rest, etc.)

Preferred, but not required:
• Detail-oriented, ability to turn deliverables around quickly with a high degree of accuracy
• Strong analytical skills, ability to interpret business requirements and produce functional and technical design documents.
• Good time management skills – Ability to prioritize and multi-task, handling multiple efforts at once.
• Strong desire to understand and learn domain.
• Experience in a financial services/banking industry
• Ability to work in a fast-paced environment; to be flexible and learn quickly.
• Ability to multi-task with attention to detail/ prioritize tasks.