Databricks Data Quality & Automation Engineer

The Citco Group Limited; Toronto; On-site; CAD 70,000 - 90,000; Full time

Job summary

A leading global financial services provider is seeking a Data & Databricks Test Automation Engineer to develop and implement automated testing frameworks for Databricks solutions. The ideal candidate will have a strong background in Python (PySpark), SQL, and data testing frameworks, with 2+ years of experience in Databricks. This role focuses on ensuring the accuracy and reliability of data solutions, delivering quality assurance for financial services operations.

Benefits

Flexible working arrangements

Training and education support

Inclusive work culture

Qualifications

• 2+ years hands-on experience with Databricks (Spark).
• Relevant certifications in Databricks or data testing are a plus.
• Understanding of Delta Lake and Lakehouse architecture.

Responsibilities

• Develop and implement automated testing frameworks for Databricks solutions.
• Design and execute data quality test strategies.
• Implement automated data reconciliation processes.

Skills

Programming skills in Python (PySpark)

SQL knowledge

Analytical skills

Problem-solving abilities

Experience with large-scale data processing

Education

Bachelor's degree in Computer Science, Data Science, or related field

Tools

Databricks

AWS (S3, Glue, Lambda)

Data testing frameworks and tools

Version control systems (Git)

Job description

A leading global financial services provider is seeking a Data & Databricks Test Automation Engineer to develop and implement automated testing frameworks for Databricks solutions. The ideal candidate will have a strong background in Python (PySpark), SQL, and data testing frameworks, with 2+ years of experience in Databricks. This role focuses on ensuring the accuracy and reliability of data solutions, delivering quality assurance for financial services operations.