Senior Software Engineer, Quality

Lever, Inc.; Town of Italy (NY); Remote; USD 120,000 - 231,000; Full time

Benefits offered by this job

Remote work

Home office budget

Flexible time off

Equity via VSOP

Job summary

Lever, Inc. is seeking a Senior Software Engineer, Quality for a partner-led, fully remote role based in Italy.

You will design, build, and operate AI-powered testing agents spanning test creation, execution, maintenance, evaluation, and failure triage, blending hands-on software engineering with test infrastructure and CI at scale. You'll shape testing patterns, tooling, and enablement practices, enable teams to own quality, and influence technical direction across a distributed engineering

Qualifications

• 5+ years of professional software engineering experience including production systems.
• Hands-on experience with test infrastructure, developer tooling, or internal platforms.
• Practical experience with LLMs/agent frameworks and evaluating AI outputs.
• Distributed systems and CI/CD at scale, with attention to concurrency and reliability.
• Ability to influence teams and drive tool adoption without direct authority.
• Deep product understanding to build quality solutions in collaboration with engineers.
• Strong judgment balancing quality, maintainability, and delivery speed.
• Excellent collaboration in a distributed, cross-functional environment.
• Experience with LLM evaluation, open-source tooling, or platform engineering preferred.
• Experience with Java, TypeScript, or Python; Kubernetes and Helm advantageous.

Responsibilities

• Build and operate AI agents across testing lifecycle: design, generation, maintenance, execution, triage.
• Own infrastructure for dependable agent-driven testing: environments, data, interfaces, guardrails.
• Develop reliable evaluation methods for agent output to reduce manual review.
• Apply AI-driven evaluation to non-deterministic AI capabilities of the platform.
• Collaborate with product engineering to keep testing tools relevant to new features.
• Develop reusable testing patterns, tooling, and enablement practices for engineers.
• Drive quality tooling adoption across teams without direct reporting lines.
• Contribute to system design and continuous improvement of testing infrastructure.

Tools

Kubernetes

Helm

Java

TypeScript

Python

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer, Quality based in Italy.

This role focuses on building the next generation of quality engineering through AI-powered testing and intelligent automation.
You will design, build, and operate agents that support test creation, execution, maintenance, evaluation, and failure triage.
The role combines hands-on software engineering with test infrastructure, distributed systems, CI at scale, and emerging LLM technologies.
You will help product teams take greater ownership of quality by providing reliable tooling, patterns, and practices rather than acting as a traditional QA safety net.
You will also shape how AI-driven testing is evaluated and apply those methods to non-deterministic AI capabilities.
Working across a fully remote, global engineering organization, you will have substantial ownership and influence over technical direction.
The environment emphasizes autonomy, collaboration, experimentation, and engineering quality, with opportunities to solve complex problems at scale.

Accountabilities

• Build and operate AI agents across the testing lifecycle, including test design, generation, maintenance, execution, and failure triage.

• Own the infrastructure required to make agent-driven testing dependable, including test environments, test data, tool interfaces, guardrails, runtime management, and cost controls.

• Establish reliable methods for evaluating agent output quality so engineering teams can trust automated results without extensive manual review.

• Apply AI-driven evaluation practices to testing the platform's own non-deterministic and AI-powered capabilities.

• Work closely with product engineering teams as new features are developed to ensure testing tools remain relevant, effective, and grounded in real product needs.

• Develop reusable testing patterns, tooling, and enablement practices that allow engineers to own quality independently.

• Drive adoption of quality engineering capabilities across teams that do not report directly to you.

• Contribute to technical decisions, system design, and continuous improvement of testing infrastructure and engineering practices.

Requirements

• 5+ years of professional software engineering experience, with a strong track record of writing and shipping production software.

• Hands-on experience building and owning test infrastructure, developer tooling, internal platforms, or comparable systems used by other engineers.

• Practical experience with LLMs or agent frameworks, including prompt and context design, tool use, non-deterministic behavior, and evaluation of AI-generated outputs.

• Strong understanding of distributed systems and continuous integration at scale, including reasoning about concurrency, state, timing, and system reliability.

• Ability to influence engineering teams without direct management authority and demonstrate that you have successfully driven adoption of tools or practices across teams.

• Ability and willingness to use and understand the product deeply enough to build effective quality solutions.

• Strong engineering judgment and an ability to balance quality, maintainability, delivery speed, and practical business needs.

• Excellent collaboration and communication skills, with the ability to work effectively in a distributed, cross-functional environment.

• Preferred experience with LLM evaluation or benchmarking, open-source software development, or developer tooling and platform engineering.

• Experience with Java, TypeScript, or Python, as well as Kubernetes and Helm, is advantageous.

Benefits

• U.S. annual total target cash compensation ranging from $143,800 to $231,900, depending on market, location, skills, experience, and qualifications.

• Equity through a Virtual Stock Option Plan (VSOP), where applicable.

• Fully remote and flexible work environment with the ability to work from anywhere, subject to applicable employment arrangements.

• Home office budget and co-working space support.

• Flexible time off designed to support meaningful rest and recharge.

• Locally tailored healthcare benefits and global mental wellbeing support.

• Live Well Lifestyle Spending Account, providing flexible support for wellbeing, family, personal interests, experiences, and financial wellbeing.

• Retirement and pension plans, often including company contributions, plus life and disability insurance where applicable.

• Up to $1,000 per year for self-directed professional development, including courses, certifications, and books.

• Annual company gatherings, team offsites, local meetups, and budgets supporting in-person connection.

• Inclusive, global workplace committed to welcoming people across diverse cultures and backgrounds.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.