AI Software Developer in Test - Vice President

Icapitalnetwork; Salt Lake City (UT); Hybrid; USD 120,000 - 150,000; Full time

Benefits offered by this job

Employer matched retirement plan

100% employer paid healthcare

Unlimited paid time off (PTO)

Job summary

iCapital is seeking a Software Developer in Test to enhance its platform. You will build automated test suites for API and backend services, focusing on AI/ML system evaluations. Ideal candidates have 10+ years in QA with strong API testing skills.

The role requires collaboration with engineers and product managers to ensure quality across distributed systems. iCapital promotes an inclusive workplace with flexible working hours, comprehensive benefits, and a strong commitment to employee wellbeing.

Qualifications

• 10+ years of QA engineering delivery experience focused on backend or API testing.
• 6+ years working in Agile SDLC teams, ideally with CI/CD.
• Experience testing AI/ML systems including model validation and regression testing.
• Strong experience testing REST APIs and distributed systems.

Responsibilities

• Build and implement automated test suites for API and backend services.
• Establish automated evaluation frameworks for AI systems.
• Partner with AI/ML Engineers for responsible AI practices.
• Create and manage defect workflows integrated with tools like JIRA.

Skills

QA engineering delivery

API testing

Agile SDLC

AI/ML testing

REST APIs

Blockchain technologies

CI/CD tooling

Python

Tools

GitLab

Docker

Jenkins

Postman

JMeter

Job description

About the Role

iCapital is looking to hire a Software Developer in Test to work on the iCapital platform. This individual will bring practical knowledge of agile development methodologies and engineering best practices. This role will work closely with the Development and Testing team. The ideal candidate is a combination of a hands‑on test engineer who has some managerial experience, and who can understand existing automation test frameworks and has an ability to enhance and implement the same and should also be able to work with other engineers and product managers to support feature‑based testing.

Responsibilities

• Build and implement robust automated test suites for API and backend services.
• Design and implement test strategies for AI/ML models and AI-enabled in‑house applications, including validation of model outputs, prompt behavior, determinism, and regression across releases.
• Establish automated evaluation frameworks for AI systems (i.e. accuracy, consistency, hallucination detection, bias, and drift) and integrate them into CI/CD pipelines.
• Partner with AI/ML Engineers, Platform teams, and Governance stakeholders to ensure responsible AI practices, including testability, traceability, and production readiness of AI features.
• Integrate automated test suites into existing GitLab CI/CD pipelines and deployment workflows.
• Maintain and improve automation frameworks to accommodate evolving system architecture, including blockchain based components.
• Collaborate with developers, product managers, and DevOps engineers to ensure testability and quality across distributed systems.
• Create and manage defect workflows integrated with tools such as JIRA, ensuring clear communication and traceability.
• Define, track, and publish meaningful QA metrics to support automated test development, execution, and overall quality reporting.
• Build and maintain performance and load testing suites for critical backend services and smart contract interactions where applicable.
• Perform all types of testing as needed like Functional, API, Automation, Performance, and Integration testing.
• Contribute to continuous improvement of QA processes, tooling, and best practices within an Agile environment.

Qualifications

• 10+ years of experience with QA engineering delivery experience with strong focus on backend or API testing
• 6+ years of delivery within Agile SDLC teams ideally with CI/CD
• Experience testing AI/ML or LLM‑powered systems, including model validation, prompt testing, regression testing of AI behavior, and evaluation of non‑deterministic outputs
• Familiar with AI testing techniques and metrics such as precision and recall, drift detection, bias assessment, and offline vs online evaluation approaches
• Working knowledge of AI‑enabled development tools or platforms (i.e. AI coding assistants, test‑generation tools, or internal AI platforms) and their impact on SDLC and quality governance
• Strong experience testing REST APIs, backend microservices, and distributed systems
• Experience working with blockchain technologies (i.e. smart contracts, distributed ledgers, blockchain node interaction) is preferred
• Solid understanding of CI/CD tooling and pipelines, especially GitLab CI/CD
• Experience with Python, Behave, PyTest, Postman/REST Tools, JMeter or similar performance tools
• Experience with Gitlab, Git, Jenkins, and Docker
• AI Integration experience is preferred
• Exposure to banking or finance environments is preferred

Benefits

The base salary range for this role is $120,000 to $150,000. iCapital offers a compensation package which includes salary, equity for all full‑time employees, and an annual performance bonus. Employees also receive a comprehensive benefits package that includes an employer matched retirement plan, generously subsidized healthcare with 100% employer paid dental, vision, telemedicine, and virtual mental health counseling, parental leave, and unlimited paid time off (PTO).

Employees in this role will work in the office Monday-Thursday, with the flexibility to work remotely on Friday.

iCapital is proud to be an Equal Employment Opportunity and Affinitive Action employer. We do not discriminate based upon race, religion, color, national origin, gender, sexual orientation, gender identity, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.