Automation Test Engineer (AI-Focused)
Job Description
Position Overview
We are seeking an experienced Automation Test Engineer with solid expertise in AI/ML testing to join our Quality Assurance team. The ideal candidate will have 4-5years of professional experience in test automation and a strong understanding of AI-powered applications, machine learning model testing, and emerging AI technologies.
Experience Level - 4 to 5 yrs
Job Summary
As an Automation Test Engineer, you will be responsible for designing, developing, and maintaining comprehensive automated test suites for web, mobile, and AI-powered applications. You will work closely with development, product, and QA teams to ensure software quality, identify defects early, and optimize testing processes. Your expertise in AI/ML testing will be critical in validating intelligent systems, model performance, and data accuracy.
Key Responsibilities
Test Automation & Development
- Design and develop automated test cases for web, mobile, and API-based applications
- Create and maintain comprehensive test automation frameworks using industry-standard tools
- Develop reusable test libraries and utilities to improve testing efficiency
- Execute regression, smoke, functional, and integration testing
AI/ML Testing Expertise
- Design and execute test cases specifically for AI/ML models and features
- Validate data quality, model accuracy, and prediction correctness
- Test AI-driven features (chatbots, recommendation engines, NLP models, computer vision)
- Identify edge cases and potential failures in intelligent systems
- Monitor and validate AI model performance metrics and KPIs
- Test integration of AI/ML models with application workflows
Test Planning & Strategy
- Collaborate with product managers and developers to understand requirements and test scenarios
- Develop comprehensive test strategies and test plans for new features
- Identify, document, and track defects with clear severity and reproduction steps
- Estimate testing effort and timelines for assigned projects
- Participate in test case reviews and quality assurance discussions
Quality Assurance & Reporting
- Generate detailed test reports with metrics, coverage analysis, and recommendations
- Monitor test execution and provide status updates to stakeholders
- Identify opportunities for process improvements and automation optimization
- Maintain test documentation and keep test cases up-to-date
- Participate in root cause analysis for critical defects
Collaboration & Communication
- Work with cross-functional teams including developers, product managers, and DevOps engineers
- Participate in sprint planning, standups, and retrospectives
- Mentor junior QA team members and share testing best practice
Present findings and recommendations to technical and non-technical stakeholders
Required Qualifications
Experience
- 3-4 years of hands-on experience in test automation
- 1-2 years of experience testing AI/ML-powered applications or features
- Proven experience working in Agile/Scrum development environments
- Experience with building and maintaining test automation frameworks
Technical Skills
Programming Languages:
- Python (preferred for AI/ML testing)
- Java
- JavaScript/TypeScript
Test Automation Tools:
- Selenium, , Appium.
- API testing tools (Postman, REST Assured, or similar)
- Test management tools (Zephyr)
- CI/CD tools (Jenkins, GitHub Actions, GitLab CI, or similar)
AI/ML Testing Knowledge:
- Understanding of machine learning concepts (supervised/unsupervised learning, classification, regression)
- Experience testing AI/ML models, APIs, and integrations
- Familiarity with AI testing frameworks or tools (e.g., , PyTorch testing utilities)
- Knowledge of data validation and quality assessment techniques
- Database & SQL: Ability to write queries for data validation and test data management
- Version Control: Git or similar version control systems
Education
- Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or related field
- OR equivalent professional experience in QA/test automation
Soft Skills
- Strong analytical and problem-solving abilities
- Excellent attention to detail and quality focus
- Clear communication and documentation skills
- Ability to work independently and as part of a team
- Time management and ability to handle multiple priorities
- Proactive approach to learning new tools and technologies
Preferred Qualifications
- Experience with machine learning operations (MLOps) and model monitoring
- Familiarity with cloud platforms (AWS, GCP, Azure) and their testing services
- Knowledge of performance and load testing tools (JMeter, LoadRunner)
- Experience with containerization technologies (Docker, Kubernetes)
- Experience with BDD frameworks (Cucumber)
- Participation in open-source QA projects or communities
- Relevant certifications (ISTQB, AWS Certified, etc.)
- Experience with GenAI testing (LLMs, prompt engineering validation)
Tools & Technologies
Required
- Selenium
- Python or Java
- Git
- Zephyr, or Jira
Preferred
- Docker
- Jenkins or GitHub Actions
- PyTorch
- AWS/GCP/Azure
- Appium