AI/ML Software Test Engineer (GenAI & Chatbot Solutions)

System One; Columbia (MD); Hybrid; USD 70,000 - 100,000; Part time

Benefits offered by this job

Health and welfare benefits

401(k) plan

Job summary

System One is seeking an AI/ML Software Test Engineer to support GenAI and chatbot initiatives in Columbia, MD. This contract role blends software testing with ML concepts and focuses on validating AI-powered solutions for quality and reliability.

You will collaborate with engineering and business teams to evaluate AI apps, analyze outputs, and drive improvements through automated testing, prompt engineering, and data-pipeline familiarity.

Qualifications

• Bachelor's degree in CS/IT/Engineering or equivalent experience.
• 3+ years of software engineering experience.
• 3+ years on cloud platforms (AWS, Azure, or GCP).
• Experience applying ML/GenAI concepts in a professional environment.
• Experience testing AI/ML solutions, models, or chatbots.
• Experience with automation testing (Selenium, BDD).
• Understanding of data pipelines and databases.

Responsibilities

• Support ML/GenAI initiatives, tests and enhancements.
• Collaborate with engineering to evaluate and improve AI apps.
• Analyze model outputs and provide feedback.
• Apply ML/GenAI concepts to assess model behavior and optimization.
• Work with stakeholders to ensure AI outputs meet business expectations.
• Test chatbot solutions and conversational AI platforms.
• Validate AI-generated responses for accuracy and relevance.
• Utilize prompt engineering to improve chatbot interactions.
• Develop and execute system, integration, and automation testing.
• Test data pipelines and validate outputs throughout the AI lifecycle.
• Document findings and testing results using collaboration tools.

Skills

Software testing

GenAI concepts

AI/ML concepts

Problem solving

Communication

Education

Bachelor's degree in CS/IT/Engineering

Tools

Selenium

BDD

Jira

Confluence

Job description

Job Title: AI/ML Software Test Engineer (GenAI & Chatbot Solutions)
Location: Columbia, MD
Type: Contract
Compensation: $62 /hr
Work Model: Hybrid – onsite and remote

Position Summary

Seeking a Software Engineer to support innovative Machine Learning (ML) and Generative AI (GenAI) initiatives across the organization. This role will work closely with engineering leadership and cross-functional teams to evaluate, test, and optimize AI-powered solutions, including machine learning models, chatbot technologies, and automation capabilities.

The ideal candidate has a strong foundation in software engineering and testing, combined with knowledge of ML and GenAI concepts. This individual will help validate AI solution performance, analyze outputs, identify opportunities for improvement, and collaborate with technical and business stakeholders to ensure solutions deliver accurate and meaningful results.

This is an excellent opportunity for someone who enjoys working at the intersection of software engineering, AI innovation, data analysis, automation, and quality assurance.

Key Responsibilities

• Support existing Machine Learning and Generative AI initiatives while contributing to new projects and enhancements.
• Collaborate with engineering teams to evaluate, test, and improve AI-powered applications and services.
• Analyze model outputs and provide feedback to improve performance, efficiency, and response quality.
• Apply ML and GenAI concepts to assess model behavior, validate outcomes, and identify opportunities for optimization.
• Work closely with business stakeholders to gather feedback and ensure AI-generated outputs meet business expectations.
• Participate in the evaluation and testing of chatbot solutions and conversational AI platforms.
• Validate AI-generated responses for accuracy, consistency, relevance, and effectiveness.
• Utilize prompt engineering techniques to improve chatbot interactions and response quality.
• Support efforts to fine-tune machine learning models and AI-powered solutions.
• Develop and execute system, integration, and automation testing activities.
• Test data pipelines and validate intermediate and final outputs throughout the AI lifecycle.
• Work with cloud-based environments and AI services to support development and testing activities.
• Document findings, recommendations, defects, and testing results using collaboration and project management tools.
• Partner with developers, engineers, and business teams to drive continuous improvement and successful project outcomes.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent experience.
• 3+ years of software engineering experience.
• 3+ years of experience working with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP).
• Experience applying Machine Learning (ML) concepts and Generative AI (GenAI) concepts in a professional environment.
• Experience evaluating, testing, analyzing, or supporting AI/ML solutions, models, chatbots, or related technologies.
• Experience with system testing, software testing, or quality assurance methodologies.
• Experience with automation testing using Selenium, BDD, or similar frameworks.
• Understanding of data pipelines and data-driven systems.
• Experience working with relational and non-relational databases.
• Strong analytical, troubleshooting, and problem-solving skills.
• Ability to interpret AI-generated outputs and provide actionable feedback.
• Strong verbal and written communication skills with the ability to work effectively across technical and business teams.
• Experience using collaboration tools such as Jira, Confluence, or similar platforms.

Preferred Qualifications

• Experience with AWS Bedrock, Amazon SageMaker, or similar AI services.
• Experience testing or validating chatbot and conversational AI solutions.
• Knowledge of prompt engineering principles and best practices.
• Experience working with Snowflake and/or Oracle databases.
• Experience with emerging AI testing frameworks and methodologies.
• Healthcare industry experience.
• Experience working in Agile development environments.
• Familiarity with Salesforce platforms and integrations.

Preferred Skills

• Understanding of machine learning model evaluation and optimization concepts.
• Ability to analyze complex datasets and AI-generated outputs.
• Strong attention to detail and a quality-focused mindset.
• Ability to communicate technical findings to both technical and non-technical audiences.
• Interest in emerging AI technologies and industry trends.
• Strong collaboration and stakeholder engagement skills.

Why Join Us?

This role offers the opportunity to work on cutting-edge AI and machine learning initiatives that directly impact business operations and user experience. You will collaborate with experienced engineers, business leaders, and technology teams while helping shape the future of Generative AI and Machine Learning solutions within the organization.

System One, and its subsidiaries including Joulé and Mountain Ltd., are leaders in delivering outsourced services and workforce solutions across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan.

System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.