Quality Engineer, Tools and Automation

Socket.dev; Seattle (WA); On-site; USD 140,000 - 210,000; Full time

Job summary

Socket.dev in Seattle seeks a Senior Automation & AI Engineer to join the Quality Engineering team, focusing on backend and services testing with deep expertise in LLMs, AI agents, and intelligent pipelines.

Day-to-day involves bug reporting, test plan documentation, and writing test case automation, in close collaboration with engineering and program management. You will drive CI/CD testing for high-volume releases.

Qualifications

• Bachelor's degree in Computer Science or equivalent experience.
• 5+ years of experience in test automation.
• Experience with API/server-side automation.
• Experience programming in a high-level language such as Java.
• Experience with databases and SQL.
• Experience creating test plans and QA processes.
• Experience with prompt engineering, model fine-tuning, or integrating LLM APIs (e.g., OpenAI, Gemini, or on-device models).

Responsibilities

• Bug reporting and isolation, test plan documentation, and writing test case automation.
• Work spans CI/CD pipelines and web service testing to qualify enterprise-grade releases.
• Collaborate with engineering and program management to drive quality outcomes.

Skills

Test automation

API testing

Java

SQL

CI/CD pipelines

LLM integration

QA tooling

Education

Bachelor's degree in Computer Science or equivalent

Tools

Gradle

Maven

JUnit

Selenium

Playwright

Docker

Kubernetes

Job description

We are looking for a highly skilled Senior Automation & AI Engineer to join our Quality Engineering team, bringing deep expertise in backend and services testing with cutting-edge knowledge of LLMs, AI agents, and intelligent pipelines. This dual-impact role requires a strong individual contributor who can independently architect world-class test automation systems while collaborating cross-functionally with engineering and project management teams. The ideal candidate will have strong analytical and bug reporting skills, with the ability to manage a high volume of changes and reduce manual testing effort through intelligent automation.

DESCRIPTION

Day to day, this role involves bug reporting and isolation, test plan documentation, and writing test case automation — done in close collaboration with engineering and program management. Work spans CI/CD pipelines and web service testing to qualify enterprise-grade, customer-facing releases across a high volume of changes.

MINIMUM QUALIFICATIONS

• Bachelor\'s degree in Computer Science or equivalent experience
• 5+ years of experience in test automation
• Experience with API/server-side automation
• Experience programming in a high-level language such as Java
• Experience with databases and SQL
• Experience creating test plans and QA processes
• Experience with prompt engineering, model fine-tuning, or integrating LLM APIs (e.g., OpenAI, Gemini, or on-device models)
• Experience with automation tools/test frameworks (Gradle, Maven, JUnit)

PREFERRED QUALIFICATIONS

• Experience with UI automation frameworks such as Selenium, Playwright, or Appium
• Experience building RAG pipelines and integrating vector databases
• Hands-on experience integrating LLM APIs and AI/ML models into QA workflows
• Experience with Kubernetes/Docker
• Contributions to open-source test automation or AI projects
• 2+ years of experience in AI engineering or QE tooling
• Experience qualifying enterprise-grade, customer-facing releases under CI/CD
• Experience collaborating cross-functionally with engineering and program management to drive quality outcomes