Junior AI Interview Automation Engineer

Fortinet; Sunnyvale (CA); On-site; USD 102,000 - 124,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Life and disability insurance

401(k) plan

11 paid holidays

Vacation time

Sick time

Comprehensive leave program

Job summary

Fortinet in Sunnyvale, CA is hiring a junior engineer to build and maintain our technical interview pipeline, from question bank to AI-assisted scoring and workflow automation. You will design systems to prompt and structure AI models, measure output against ground truth, catch errors, and grow the role into broader automation, working with the team to evolve the function.

The role requires Bachelor’s in CS or related field, strong programming fundamentals (Python, JS/TS), API experience, and

Qualifications

• Bachelor's degree in Computer Science or related field.
• Solid programming fundamentals (Python, JavaScript/TypeScript, or similar).
• Comfort working with APIs, including LLM/AI APIs (OpenAI, Anthropic, etc.).
• Demonstrated experience designing and evaluating prompts/pipelines for LLM-based tools (not just using AI products, but building with them).
• Proficient with Git and a hosted platform (GitHub or GitLab) - branching, PRs/MRs, code review, CI basics
• Basic understanding of databases (SQL or similar)
• Strong attention to detail and technical writing ability
• Ability to work independently on processes that are still being defined, and adapt as scope evolves

Responsibilities

• Architect scoring workflows that evaluate and compare candidate answers against reference answers/rubrics, grounded in a clear methodology for what 'correct' and 'well-scored' mean
• Calibrate and benchmark scoring prompts/models against human-graded samples, measuring accuracy and identifying systematic errors or bias
• Understand and account for model failure modes (inconsistency, hallucination, prompt sensitivity) and design safeguards around them
• Flag edge cases or low-confidence scores for human review rather than fully automating high-stakes decisions
• Automate the end-to-end workflow: question selection -> test delivery -> answer collection -> scoring -> reporting
• Help transition our source code and repositories to an internal Git platform
• Track test results to help establish scoring benchmarks over time
• Document the process, prompts, and scoring logic for auditability

Skills

Python

JavaScript/TypeScript

APIs (LLM/AI)

Prompt design

Git workflows

SQL basics

Technical writing

Independent work

Education

Bachelor's degree in Computer Science or related field

Tools

GitHub

GitLab

Job description

Fortinet in Sunnyvale, CA is hiring a junior engineer to build and maintain our technical interview pipeline, from question bank to AI-assisted scoring and workflow automation. You will design systems to prompt and structure AI models, measure output against ground truth, catch errors, and grow the role into broader automation, working with the team to evolve the function.

The role requires Bachelor’s in CS or related field, strong programming fundamentals (Python, JS/TS), API experience, and