Data QA Engineer

Insight Global; Vancouver; On-site; CAD 80,000 - 100,000; Full time

Job description

Job Description

Day to Day:

We are seeking a qualified candidate for the role of QA Engineer, who will be responsible for developing and executing automated test scripts in a big data environment. The Senior Data Quality Engineer will support the development of secure, scalable data integration and analytics solutions within the People Enablement Technology team. This role focuses on validating data pipelines built using Databricks, Azure Data Factory, Snowflake, and related technologies. The ideal candidate will have a strong background in ETL testing, data validation, and automation, with a deep understanding of data governance, encryption, and automation practices.

You will collaborate closely with data engineers, architects, analysts, and business stakeholders to ensure that data solutions meet business requirements and maintain high standards of quality and compliance. This role focuses on validating data pipelines built using Databricks, Azure Data Factory, Snowflake, and related technologies. The ideal candidate will have a strong background in ETL testing, data validation, and automation, with a deep understanding of data governance, encryption, and DevSecOps practices.

You will collaborate closely with data engineers, architects, analysts, and business stakeholders to ensure that data solutions meet business requirements and maintain high standards of quality and compliance.

Principal Duties and Responsibilities:

Design and implement test strategies for complex data pipelines and workflows using Databricks, Azure Data Factory, and Snowflake

Develop and maintain automated test suites for data validation, transformation logic, and data quality checks using Python, SQL, and PySpark

Collaborate with cross-functional teams to define test cases, acceptance criteria, and quality benchmarks

Perform root cause analysis and support troubleshooting of data issues in production and non-production environments

Maintain detailed test documentation, prepare test data sets, and contribute to knowledge sharing across the QA and engineering teams

We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form (https://airtable.com/app21VjYyxLDIX0ez/shrOg4IQS1J6dRiMo) . The EEOC "Know Your Rights" Poster is available here (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12ScreenRdr.pdf) .

To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .

Skills and Requirements

Must Haves:

5+ years in Data Engineering with a strong understanding of ETL QA testing

Proficiency in Python and PySpark for developing and executing automated test scripts in a big data environment, along with experience in performance testing.

Strong SQL skills to perform data validation and verification during testing.

Expertise in Selenium

Familiarity with cloud platforms such as Azure, and hands-on experience with Databricks for data processing and testing. Plusses:

Understanding of data encryption, handling sensitive data, and governance using Databricks Unity Catalog.

• Experience with People data

  Power BI for visualizing and analyzing data, aiding in the identification of data quality issues and anomalies.

  Experience in a retail environment null

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [e-mail].