data engineer

The Wolf Works; Mississauga; On-site; CAD 110,000 - 120,000; Full time

Job summary

A technology firm in Mississauga is seeking an experienced Data Engineer to design and develop software solutions. The role requires strong proficiency in Python and Spark, with responsibilities including developing data analysis programs and optimizing applications. The ideal candidate has at least 8 years of experience and is expected to work on-site. This full-time position offers a competitive salary range of $110,000 to $120,000 annually.

Qualifications

• 8 years of experience as a Data Engineer is required.
• Strong proficiency in Python and Spark.

Responsibilities

• Design, develop, and maintain software solutions using Python.
• Develop Python and PySpark programs for data analysis.
• Build custom frameworks in Python for generating rules.
• Gather data from HBase and design solutions using PySpark.
• Work with Apache Spark DataFrames/RDDs to apply business transformations.
• Utilize Hive Context objects for read/write operations.
• Develop back-end services and APIs.
• Optimize applications for performance, security, and maintainability.
• Troubleshoot and resolve software defects.

Skills

Python

Spark

Education

Formal UX Qualification

Job description

Job details

• Job Title: Data Engineer
• Location: Mississauga, ON, Canada
• Employment Type: Full-time
• Salary: $110,000.00-$120,000.00 per year
• Experience: Data Engineer: 8 years (required)
• Work location: On site
• Starts: As soon as possible
• Source: Indeed.com

Responsibilities

• Design, develop, and maintain software solutions using Python.
• Develop Python and PySpark programs for data analysis.
• Build custom frameworks in Python for generating rules (similar to a rules engine).
• Gather data from HBase and design solutions using PySpark.
• Work with Apache Spark DataFrames/RDDs to apply business transformations.
• Utilize Hive Context objects for read/write operations.
• Develop back-end services, APIs, and integrate databases.
• Optimize applications for performance, security, and maintainability.
• Troubleshoot and resolve software defects and issues.

Qualifications

• Strong proficiency in Python and Spark.
• At least one formal UX Qualification.
• Application questions: This is a Full-Time position; confirm continued interest. Please share your LinkedIn profile. (Mandatory requirement)
• Location: Mississauga, ON
• Work location: On site

Notes

Important notice: This job posting has been provided by a partner site. Job Bank is not responsible for this content.