Sr Data Engineer

MPA Recruitment

Toronto

On-site

CAD 120,000 - 180,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

MPA Recruitment seeks a Senior Data Engineer to design and develop data processing and persistence software for large-scale data systems. You will lead development in agile teams, shaping significant data components with a focus on quality and maintainability.

The role emphasizes hands-on coding in Java/Scala/Python, building on AWS/Azure/GCP and Databricks, with strong emphasis on ETL/ELT pipelines, CI/CD, and production readiness. Docker/Kubernetes experience is desirable.

Qualifications

  • Strong software development experience in one of Java, Scala, or Python.
  • Experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
  • Experience developing substantial components for large-scale data processing solutions and deploying into production.
  • Proficient in SQL and SQL extensions for analytical queries.
  • Solid understanding of ETL/ELT data processing pipelines and design patterns.
  • Aware of features and pitfalls of distributed data processing frameworks, data stores and formats.
  • Able to write quality, testable code and has experience of automated testing.
  • Experience with CI/CD techniques.

Responsibilities

  • Design and develop data processing software components for large-scale data systems.
  • Provide development leadership and own significant technical components.
  • Work with Architects and Lead Engineers to ensure non-functional needs are met.
  • Collaborate with teams to resolve challenges and maintain code quality and maintainability.
  • Coach and mentor team members and ensure operational readiness.

Skills

Java
Scala
Python
SQL
ETL/ELT
Distributed systems
Automated testing
CI/CD

Tools

Docker
Kubernetes

Job description

As a Senior Data Engineer, you will be responsible or designing and developing data processing and data persistence software components for solutions which handle data at scale. Working in agile teams, Senior Data Engineers provide strong development leadership and take responsibility for significant technical components of data systems. You will work within a multi-skilled agile team to design and develop large-scale data processing software to meet user needs in demanding production environments.

Your responsibilities will include working to develop data processing software primarily for deployment in Big Data technologies. The role encompasses the full software lifecycle including design, code, test and defect resolution. Working with Architects and Lead Engineers to ensure the software supports non-functional needs. Collaborating with colleagues to resolve implementation challenges and ensure code quality and maintainability remains high. Leads by example in code quality. Working with operations teams to ensure operational readiness. Advising customers and managers on the estimated effort and technical implications of user stories and user journeys. Coaching and mentoring team members

MINIMUM (ESSENTIAL) REQUIREMENTS
  • Strong software development experience in one of Java, Scala, or Python
  • Software development experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
  • Experience of developing substantial components for large-scale data processing solutions and deploying into a production environment
  • Proficient in SQL and SQL extensions for analytical queries
  • Solid understanding of ETL/ELT data processing pipelines and design patterns
  • Aware of key features and pitfalls of distributed data processing frameworks, data stores and data serialisation formats
  • Able to write quality, testable code and has experience of automated testing
  • Experience with Continuous Integration and Continuous Deployment techniques
DESIRABLE:
  • Experience of data pipeline development
  • Experience of Docker and Kubernetes
  • Experience of performance tuning
  • Experience of data visualisation and complex data transformations
  • Experience with steaming and event-processing architectures including technologies such as Kafka and change-data-capture (CDC) products
  • Expertise in continuous improvement and sharing input on data best practice
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior data engineer
Senior data engineer

Akkodis • Montreal

On-site
CAD 90,000 - 120,000
Senior Software Engineer – Core Java, Apache Spark, Python, SQL, Data Processing
Senior Software Engineer – Core Java, Apache Spark, Python, SQL, Data Processing

Astra-North Infoteck Inc. • Montreal

On-site
CAD 90,000 - 130,000
Senior Data Engineer: Lead Scalable Data Pipelines
Senior Data Engineer: Lead Scalable Data Pipelines

MPA Recruitment • Toronto

On-site
CAD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

Luxoft • Toronto

On-site
CAD 90,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Aarorn Technologies Inc • Markham

On-site
CAD 68,880 - 89,544
Senior Software Developer - ETL JP241
Senior Software Developer - ETL JP241

Pathlion • Toronto

On-site
CAD 90,000 - 120,000
Senior Software Developer - ETL JP241
Senior Software Developer - ETL JP241

P@thlion Staffing Careers • Toronto

On-site
CAD 110,000 - 150,000
Sr. Data Engineer
Sr. Data Engineer

HRBrain Inc. • Toronto

On-site
CAD 130,000 - 180,000
Lead/Senior Data Engineer
Lead/Senior Data Engineer

Celebal Technologies • Ottawa

On-site
CAD 110,000 - 160,000
Senior Data Engineer
Senior Data Engineer

Cpus Engineering Staffing Solutions Inc. • Pickering

Hybrid
CAD 90,000 - 120,000