Senior Software Developer in Test (AI Portfolio)

Vena Solutions

United States

Remote

USD 82,000 - 111,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Vena Solutions is seeking a Senior Software Developer in Test to build and maintain automated tests for AI-powered solutions. You will shape testing strategies, validate model outputs, and enhance automation frameworks while collaborating with product and development teams.

Your role emphasizes AI testing, integration into CI/CD, and robust test coverage across UI and API layers. A strong Agile background and scripting skills are essential.

Qualifications

  • 5+ years in QA/QE and test automation in Agile environments.
  • AI/ML product testing experience including prompt evaluation or model testing.
  • Experience integrating AI tools into CI/CD to speed test execution.
  • Strong knowledge of UI and API automation design patterns.
  • Proficiency with SQL or similar databases for data validation.

Responsibilities

  • Define comprehensive test strategies and automation needs from requirements.
  • Design, write, and maintain automated UI and API tests.
  • Contribute to scalable automation frameworks (Playwright, Rest-Assured).
  • Execute exploratory, functional, integration, performance, and regression tests.
  • Collaborate with teams to optimize coverage and validation.
  • Promote best practices for data management, CI/CD, and code quality.
  • Monitor and optimize automation pipelines for reliability and speed.
  • Perform root-cause analysis and assist in defect prevention.

Skills

QA/QE experience
AI testing
Programming languages
CI/CD
Analytical skills

Tools

Playwright
Cypress
Selenium
RestAssured
RestSharp
Pytest
GitHub
GitLab
JIRA
Xray
Zephyr
Qase

Job description

As a Senior Software Developer in Test, you will be responsible for the development, execution, and maintenance of automated tests to support the Software Test Life Cycle for Vena’s agentic AI solutions. You will collaborate closely with development and product teams to define and execute functional, integration, and regression testing strategies, while continuously improving the scalability, stability, and reliability of testing processes.

You will play a key role in defining and implementing testing approaches for AI-powered products, validating model accuracy, output consistency, performance, edge cases, and overall product quality. Additionally, you will leverage Generative AI and modern AI tools to enhance test automation, streamline test case development, and improve testing efficiency and coverage.

How You'll Make an Impact
  • Analyze functional and technical requirements to define comprehensive test strategies and automation needs
  • Design, write, maintain, and manage automated UI and API test suites
  • Contribute to building and enhancing scalable automation frameworks (e.g., Playwright, Rest-Assured)
  • Identify, plan, and execute exploratory, functional, integration, performance, and regression tests
  • Collaborate with development and product teams to validate testing requirements and optimize test coverage
  • Champion best practices for automation, test data management, CI/CD integration, and code quality
  • Monitor, troubleshoot, and optimize automation pipelines and test frameworks for dependability, scalability, and stability
  • Perform root cause analysis and support teams in identifying and preventing defects
We'd Love to See
  • 5+ years’ experience in QA/QE and test automation development roles, within a fast-paced Agile environment
  • AI Testing Expertise: Proven track record testing AI/ML products (including prompt evaluation, bias detection, or model boundary testing)
  • Hands-on experience integrating AI tools or LLM APIs into CI/CD pipelines to accelerate test execution and automatically detect flaky tests.
  • Strong understanding of software testing methodologies, automation design patterns, and the fundamentals of high-quality test case development
  • 3+ years of hands-on experience with at least one automation framework for both UI and API test automation (Playwright, Cypress, Selenium, RestAssured, RestSharp, Pytest, etc.)
  • Hands-on experience with version control, issue tracking, and test case management tools (GitHub, Gitlab, JIRA, Xray, Zephyr, Qase, etc.)
  • Proficiency with relational databases and data validation (SQL, MS SQL, Oracle, GraphQL, or DB2)
  • Proficiency in one or more programming languages (Java, JavaScript, Typescript, C#, or Python)
  • Ability to think critically about system architecture and assess the impact of product changes across components
  • Strong troubleshooting, analytical, and problem-solving skills
  • Exceptional written and verbal communication skills, with the ability to articulate both technical and non-technical concepts
  • Interest in AI and willingness to explore AI-driven solutions to enhance performance and drive efficiencies

The base salary range for this role is $114,750-155,250 CAD.

*Our salaries are tailored to roles, levels and locations. Your individual pay within this range is influenced by factors like work location, skills, experience and education. As you progress in your role, your compensation may adapt, offering flexibility for growth beyond initial levels. For specifics, your recruiter will provide details and address any questions during the hiring process.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI QA Engineer - Test Automation & AI Validation
Senior AI QA Engineer - Test Automation & AI Validation

Vena Solutions • United States

Remote
USD 82,000 - 111,000
Agentic Ai Automation Engineer
Agentic Ai Automation Engineer

Tata Consultancy Services Limited • Menlo Park (CA)

On-site
USD 120,000 - 130,000
Discretionary annual incentive
Comprehensive medical coverage
Parental leaves
+2
Senior Software Development Engineer in Test
Senior Software Development Engineer in Test

Varicent • United States

Hybrid
USD 72,000 - 95,000
Health & Wellness
Time Off
Volunteer Days
+5
Principal Test Engineer – Agentic AI
Principal Test Engineer – Agentic AI

Businessolver • United States

Remote
USD 93,000 - 140,000
Quarterly bonus
Engineer
Engineer

Tata Consultancy Services Limited • Menlo Park (CA)

On-site
USD 100,000 - 130,000
Annual incentive
Medical coverage
Parental leaves
+4
Test Automation Specialist, AI & Quality Engineering
Test Automation Specialist, AI & Quality Engineering

PLATO • United States

Remote
USD 120,000 - 160,000
Senior Test Engineer / QA Automation Engineer
Senior Test Engineer / QA Automation Engineer

Site • United States

On-site
EUR 55,000 - 85,000
Flexible engagement model
Challenging projects
Security compliance
+1
Automation Test Engineer
Automation Test Engineer

Apt • Birmingham (AL)

On-site
USD 90,000 - 130,000
Lead Software Engineer in Test – Agentic AI (Remote - US)
Lead Software Engineer in Test – Agentic AI (Remote - US)

vibehackers • United States

Remote
USD 150,000 - 190,000
Generous annual bonus opportunity
401(k) with employer match
Medical Insurance
+2
Lead AI Automation
Lead AI Automation

Tata Consultancy Services Limited • Plano (TX)

On-site
USD 100,000 - 105,000
Discretionary Annual Incentive
Medical Coverage: Health, Dental & VIs
Parental Leaves
+3