GenAI Prompt Engineer

Moonfire; Toronto; On-site; CAD 90,000 - 130,000; Full time

Job summary

Moonfire in Toronto seeks an experienced AI/software QA professional to drive prompt design, validate AI outputs, and build robust test cases for GPT-based systems. You will collaborate with data scientists, engineers, and product managers to implement improvements in AI workflows, ensuring high-quality user experiences.

Required 5-8 years in software or automation testing, plus at least 1 year with generative AI/LLMs.

Qualifications

• 5-8 years of professional experience in software development or automation testing.
• Minimum 1 year hands-on experience with generative AI technologies, large language models (LLMs), or GPT-based systems.
• Strong understanding of natural language processing (NLP) principles.
• Proficiency in programming languages such as Python, JavaScript, or similar tools.
• Experience with automation frameworks and test scripting.
• Familiarity with AI and ML workflows is a plus.
• Strong analytical and problem-solving skills.

Responsibilities

• Prompt Writing: Design and refine prompts for generative AI systems to maximize performance and accuracy.
• AI Response Validation: Analyze and validate AI-generated outputs for quality and contextual relevance.
• Test Case Development: Create and execute comprehensive test cases for AI responses.
• Iteration and Feedback: Provide structured feedback to improve AI model behavior.
• Collaboration: Work with data scientists, engineers, and product managers to implement and test improvements.

Skills

Analytical thinking

NLP principles

Prompt engineering

LLMs

Automation testing

Strong problem-solving

Tools

Python

JavaScript

Automation frameworks

Job description

Key Responsibilities:

• -Prompt Writing: Design and refine creative, effective, and contextually relevant prompts for generative AI systems to maximize performance and accuracy across various use cases.
• -AI Response Validation: Analyze and validate AI-generated outputs to ensure they meet quality standards, are contextually appropriate, and align with user needs.
• -Test Case Development: Create and execute comprehensive test cases for AI generated responses, assessing robustness, accuracy, and user experience.
• -Iteration and Feedback: Provide structured feedback to improve AI model behavior, performance, and overall functionality.
• -Collaboration: Work closely with cross-functional teams including data scientists, engineers, and product managers to implement and test improvements.

Experience:

• -5-8 years of professional experience in software development or automation testing.
• -Minimum 1 year of hands-on experience with generative AI technologies, large language models (LLMs), or GPT-based systems.
• -Strong understanding of natural language processing (NLP) principles.
• -Proficiency in programming languages such as Python, JavaScript, or similar tools.
• -Experience with automation frameworks and test scripting.
• -Familiarity with AI and ML workflows is a plus.
• -Strong analytical and problem-solving skills.