Databricks Automation Engineer

Veriipro; Seattle (WA); On-site; USD 110,000 - 145,000; Full time

Job summary

Veriipro is seeking a Databricks Automation Engineer in Seattle, WA, to design and implement automated testing frameworks for their data lakehouse platform. This role requires strong expertise in Databricks/Spark and a solid background in test automation.

The ideal candidate will have extensive experience leading QA initiatives, enhancing data pipeline quality, and integrating testing in CI/CD workflows. Join our high-performing team to ensure the reliability and performance of our data systems.

Qualifications

• 8+ years in software engineering/testing with QA ownership.
• 3+ years leading or mentoring engineering/QA teams.
• Strong understanding of data lakehouse patterns and big data platforms.

Responsibilities

• Architect and maintain automated test frameworks for ETL/ELT and APIs.
• Design data validation, quality checks, and anomaly detection strategies.
• Integrate testing into CI/CD pipelines for pre-deploy gating and regression testing.

Skills

Databricks/Spark expertise

Data pipeline management

Test automation in Python, Java, or Scala

CI/CD integration

Performance testing

Job description

We are looking for an experienced Databricks Automation Engineer to design, implement, and scale automated testing frameworks for our data lakehouse platform. This role blends data engineering expertise with test automation to ensure high-quality, reliable, and performant data pipelines.

Responsibilities

• Architect and maintain automated test frameworks for ETL/ELT, Databricks/Spark, APIs, and UIs across multiple teams.
• Design data validation, quality checks, and anomaly detection strategies for large-scale telemetry and batch/streaming pipelines.
• Integrate testing into CI/CD pipelines (Azure DevOps / GitHub Actions) for pre-deploy gating, contract testing, and regression protection.
• Drive performance and scalability testing for Spark jobs and microservices; identify and remediate bottlenecks.
• Lead synthetic test data generation, mocking/stubbing, and secure environment-aware test harnesses.
• Implement test observability, metrics dashboards, and automated verification SLAs.
• Own test documentation, runbooks, failure triage, and production handovers.
• Mentor and grow a high-performing QA/test automation team; promote standards and best practices.

Qualifications

• 8+ years in software engineering/testing, with QA ownership across the SDLC.
• 3+ years leading or mentoring engineering/QA teams.
• Strong expertise in Databricks/Spark/Delta Lake, data pipelines, and lakehouse patterns (CDC, schema evolution, delta tables, time-travel, lineage).
• Hands‑on experience building scalable automated test frameworks integrated with CI/CD.
• Proficient in Python, Java, or Scala for test automation; experience with big‑data platforms at scale.