AI Automation Engineer

Zoomcar; Sunnyvale (CA); On-site; USD 110,000 - 160,000; Full time

Job summary

Fortinet in Sunnyvale, CA seeks a junior engineer to own and build our technical interview pipeline—from the question bank to AI-assisted scoring and workflow automation. You’ll design systems to add/calibrate interview questions, prompt and structure AI models for consistency, and measure results against ground truth to catch and correct errors.

You will grow this role into a broader automation function within Fortinet, collaborating with the team to align interview content with real job

Qualifications

• Experience designing and maintaining AI-driven interview pipelines.
• Ability to design systems for adding/calibrating interview questions and evaluating candidate answers.
• Skill in prompting and structuring AI models for consistency and reliability.
• Ability to measure output quality against ground truth and catch model errors.
• Willingness to grow the role into broader automation initiatives over time.

Responsibilities

• Architect scoring workflows for evaluating candidate answers against rubrics.
• Develop validation checks to prevent incorrect or biased outputs.
• Create automation for updating question sets and metadata.
• Review and update questions to stay aligned with real job requirements.
• Collaborate with teams to expand automation capabilities within Fortinet.

Skills

Technical interview pipeline

AI-assisted scoring

Prompt engineering

System design

Quality assurance

Job description

We're looking for a junior engineer to be the primary engineer building and maintaining our technical interview pipeline - from the question bank to AI-assisted scoring to full workflow automation - working alongside the team to grow this role into a broader automation function over time. This isn't a "call an API and ship it" job: you'll be designing the systems that add/calibrate interview questions and evaluate candidate answers, which means understanding how to prompt and structure AI models for consistency, how to measure and validate output quality against ground truth, and how to catch and correct the ways these systems get things wrong. This is a role we expect to grow - the shape of that growth is still being defined.

Fortinet (NASDAQ: FTNT) secures the largest enterprise, service provider, and government organizations around the world. Fortinet empowers its customers with intelligent, seamless protection across the expanding attack surface and the power to take on ever-increasing performance requirements of the borderless network - today and into the future. Only the Fortinet Security Fabric architecture can deliver security without compromise to address the most critical security challenges, whether in networked, application, cloud or mobile environments. Fortinet ranks number one in the most security appliances shipped worldwide and more than 500,000 customers trust Fortinet to protect their businesses. We are committed to providing reasonable accommodations for all qualified individuals with disabilities. If you require assistance or accommodation due to a disability, please contact us at [e-mail]. Fortinet is an equal opportunity employer. We value diversity in our company, and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, military/veteran status or any other applicable legally protected characteristics in the location in which the candidate is applying.

We're looking for a junior engineer to be the primary engineer building and maintaining our technical interview pipeline - from the question bank to AI-assisted scoring to full workflow automation - working alongside the team to grow this role into a broader automation function over time. This isn't a "call an API and ship it" job: you'll be designing the systems that add/calibrate interview questions and evaluate candidate answers, which means understanding how to prompt and structure AI models for consistency, how to measure and validate output quality against ground truth, and how to catch and correct the ways these systems get things wrong. This is a role we expect to grow - the shape of that growth is still being defined.

KEY RESPONSIBILITIES

Assessment Repository Management

• Add, revise, and retire technical interview questions across roles/levels/skill areas
• Maintain question metadata (difficulty, topic tags, expected answer criteria, role relevance)
• Create variant/randomized versions of questions to reduce answer-sharing risk
• Build validation checks (automated and manual) to catch incorrect, ambiguous, or miscalibrated questions before they enter the repository
• Build automation to update problem sets and recalculate average difficulty ratings as new results come in
• Review and update questions periodically to prevent leakage/staleness and keep content aligned with actual job requirements

Scoring & Answer Evaluation

• Architect scoring workflows that evaluate and compare candidate answers against reference answers/rubrics, grounded in a clear methodology for what "correct" and "well-scored" mean
• Calibrate and benchmark scoring prompts/models against human-graded samples, measuring accuracy and identifying systematic errors or bias
• Understand and account for model failure modes (inconsistency, hallucination, prompt sensitivity) and design safeguards around them
• Flag edge cases or low-confidence scores for human review rather than fully automating high-stakes decisions

Process Automation