Data QA Engineer (Databricks)

Kutir Technologies; Ciudad de México; Presencial; MXN 1,249,000 - 1,607,000; Jornada completa

Descripción de la vacante

A technology solutions provider based in Mexico City is seeking an experienced Data QA Engineer. The role involves supporting Databricks development and migration projects on AWS and Azure. The ideal candidate will have over 5 years of hands-on experience, strong expertise in data quality assurance methodologies, and proficiency in SQL, PySpark, and automation frameworks. Join a team focused on delivering high-quality data solutions and improving QA processes.

Formación

• 5 years of experience working with Databricks on cloud platforms (AWS and/or Azure).
• Strong expertise in data quality assurance testing methodologies and test automation for big data and cloud-based environments.
• Hands-on experience in writing and executing SQL queries for data validation and analysis.

Responsabilidades

• Design, develop and execute comprehensive test strategies for Databricks-based data pipelines.
• Develop and maintain automated test scripts to validate data accuracy within Databricks environments.
• Work closely with developers to define test requirements and data validation protocols.

Conocimientos

Data quality assurance testing methodologies

SQL queries for data validation

PySpark

Test automation for big data

Data integration patterns

Educación

Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or related field

Herramientas

Jira

Descripción del empleo

Data QA Engineer Databricks Development & Migration Projects

Supporting Databricks Initiatives on AWS and Azure Cloud Platforms

Position Overview

We are seeking an experienced Data QA Engineer to join our team and support Databricks development and migration projects. The ideal candidate will have a minimum of 5 years of hands‑on experience working with Databricks on cloud platforms such as AWS and this role you will collaborate closely with data engineering analytics and migration teams to ensure the highest quality standards for our data solutions and migration processes.

Key Responsibilities

• Design develop and execute comprehensive test strategies for Databricks‑based data pipelines ETL processes and migration solutions on AWS and Azure.
• Develop and maintain automated test scripts to validate data accuracy transformation logic and system performance within Databricks environments.
• Work closely with developers data engineers and business analysts to define test requirements acceptance criteria and data validation protocols.
• Analyze data migration results identify discrepancies and drive resolution of data quality issues.
• Perform root cause analysis of defects and provide detailed reports to stakeholders.
• Contribute to the continuous improvement of QA processes test automation frameworks and best practices for Databricks on cloud platforms.
• Participate in code reviews design discussions and sprint planning activities to advocate for quality standards.

Must‑Have Skills and Experience

• 5 years of experience working with Databricks on cloud platforms (AWS and/or Azure).
• Strong expertise in data quality assurance testing methodologies and test automation for big data and cloud‑based environments.
• Hands‑on experience in writing and executing SQL queries for data validation and analysis.
• Proficiency with PySpark Python and Databricks notebooks for test automation and data validation.
• Experience in validating ETL pipelines data transformations and large‑scale data migrations.
• Solid understanding of cloud storage solutions (e.g. AWS S3 Azure Data Lake) and data integration patterns.
• Proven ability to troubleshoot and resolve data quality issues in complex distributed environments.
• Excellent communication and collaboration skills.

Good‑to‑Have Skills

• Experience with CI/CD tools for data pipeline deployment and automated testing (e.g. Jenkins Azure DevOps).
• Familiarity with other data platforms and tools such as Snowflake Redshift or Synapse Analytics.
• Knowledge of data governance data lineage and metadata management.
• Experience with performance testing and optimization of big data workloads.
• Exposure to scripting languages such as Scala or Shell scripting.
• Relevant certifications (e.g. Databricks Certified Data Engineer AWS Certified Solutions Architect Azure Data Engineer Associate).

Educational Qualifications

• Bachelors or Masters degree in Computer Science Information Systems Engineering or a related field.

Key Skills

• Corporate Communications
• Apache Commons
• Community Support
• Jira
• Java

Employment Type: Full Time

Experience: years

Vacancy: 1