Data Engineer

TalentOla; Mississauga; On-site; CAD 90,000 - 110,000; Full time

Job summary

TalentOla is seeking a Data Engineer/Python Developer in Mississauga with a minimum of 9 years experience. The role includes designing, developing, and maintaining software solutions, along with developing Python and Pyspark programs for data analysis. Candidates should have hands-on experience with HBase and optimizing applications for performance and security. The employment type is full-time with a requirement for on-site work five days a week.

Qualifications

• Minimum 9 years of experience in data engineering and Python development.
• Strong knowledge of data analysis techniques using Python and Pyspark.

Responsibilities

• Design, develop, and maintain software solutions using Python.
• Develop Python and Pyspark programs for data analysis.
• Gather data from HBase and implement solutions using Pyspark.
• Develop back-end services and APIs.
• Optimize applications for performance.

Skills

Python

Spark

Pyspark

Job description

Job Title: Data Engineer/ Python Developer

Experience: Minimum 9 years

Employment: Fulltime, 5 days onsite

Mandatory Skills: Python, Spark, Pyspark

Responsibilities

• Design, develop, and maintain software solutions using Python.
• Develop Python and pyspark programs for data analysis. Good working experience with Python to develop a custom framework for generating rules (similar to a rules engine).
• Develop Python code to gather data from HBase and design a solution to implement using Pyspark. Apache Spark DataFrames/RDDs were used to apply business transformations and utilize Hive Context objects to perform read/write operations.
• Develop back‑end services, APIs, and integrate databases.
• Optimize applications for performance, security, and maintainability.
• Troubleshoot and resolve software defects and issues.