Software Engineer III

Electronic Arts (EA); Vancouver; Hybrid; CAD 122,000 - 171,000; Full time

Benefits offered by this job

Vacation 3 weeks per year

Extended health/dental/vision coverage

Life insurance

Disability insurance

Retirement plan

Job summary

Electronic Arts in Burnaby, BC is seeking a Senior QA Engineer for FC Quality Verification Engineering. You will build automated testing frameworks and evaluation tools, applying AI and ML to expand test coverage, report to Engineering Manager, and work in a hybrid setup from EA's Burnaby office.

The role emphasizes developing LLM-based validation, integrating with CI/CD, and collaborating with engineering, design, and quality partners to clarify requirements and resolve blockers.

Qualifications

• 5+ years of professional software engineering experience, including 2+ years of hands-on AI/ML/LLM experience.
• Master of Science in CS/Math or equivalent experience.
• In-depth experience with LLMs, AI/ML, NLP concepts and failure modes.
• Experience developing evaluation frameworks for generative systems.
• Proficiency in Python, algorithms, data structures, modular design, automated testing, debugging, and version control.

Responsibilities

• Build automated testing frameworks, validation checks, and CI/CD integrations with reproducible diagnostics.
• Develop methods to explore complex software behaviour, identify coverage gaps, and detect defects.
• Build automated checks to validate software behaviour and data against requirements.
• Develop evaluation frameworks for generative AI and LLM-based systems with rubric-based scoring and calibration.
• Apply AI, NLP and ML techniques to solve testing problems.
• Integrate testing tools into development pipelines and produce reproducible diagnostics.
• Prototype ML/LLM capabilities and collaborate with engineering, design and quality teams to clarify requirements and resolve blockers.

Skills

Python

LLMs

NLP

AI/ML

Testing frameworks

CI/CD

Data structures

Debugging

Version control

Education

Master of Science in Computer Science or Mathematics

Equivalent professional engineering experience

Tools

Git

CI/CD tooling

Jenkins

Job description

Description & Requirements

Electronic Arts creates next-level entertainment experiences that inspire players and fans around the world. Here, everyone is part of the story. Part of a community that connects across the globe. A place where creativity thrives, new perspectives are invited, and ideas matter. A team where everyone makes play happen.

Description

In FC Quality Verification Engineering, our mission is to help our teams identify quality issues earlier. We build automation frameworks and developer tools that combine software development and applied AI/ML to improve how we test EA-SPORTS FC.

Role Summary

You will work with us to build automated testing frameworks and evaluation tools, applying AI and machine learning to expand our test coverage. You will report to our Engineering Manager. You will work hybrid, at least 3 days a week, from our office in Burnaby, BC.

Responsibilities

• You will build automated testing frameworks, validation checks, and CI/CD pipeline integrations with reproducible diagnostics.
• Develop methods that help us explore complex software behaviour, identify coverage gaps, and detect defects.
• Build automated checks to validate software behaviour and data against requirements.
• Develop evaluation frameworks for generative AI and LLMs-based systems (e.g., implementing LLMs-as-a-judge patterns, rubric-based scoring, and automated calibration) alongside deterministic validation checks.
• Apply generative AI, NLP, and machine learning, balancing heuristic, deterministic, and model-based approaches to solve complex software testing problems.
• Integrate testing tools into development pipelines, producing reproducible diagnostics, and support the software you deliver.
• Prototype latest machine learning and LLMs capabilities, and collaborate with engineering, design, and quality partners to clarify requirements, review code, and resolve blockers.

Required Qualifications

• 5+ years of professional software engineering experience, including 2+ years of hands-on experience building, integrating, or evaluating AI/ML and LLMs-based systems.
• A Master of Science in Computer Science, Mathematics or a related field, or equivalent professional engineering experience.
• In-depth experience with LLMs, generative AI, and NLP concepts, architectures, and failure modes.
• Experience developing evaluation frameworks for generative systems, including dataset curation, model-as-judge design, validation criteria, error analysis, and quantitative/qualitative metrics.
• Proficiency in Python, algorithms, data structures, modular design, automated testing, debugging, and version control.
• Experience preparing datasets, defining validation criteria, and performing error analysis using quantitative ML metrics.
• Experience with natural language processing or large language models, such as prompting, fine-tuning, or retrieval-augmented generation.

Pay Transparency - North America

The ranges listed below are what EA in good faith expects to pay applicants for this role in these locations at the time of this posting. If you reside in a different location, a recruiter will advise on the applicable range and benefits. Pay offered will be determined based on a number of relevant business and candidate factors (e.g. education, qualifications, certifications, experience, skills, geographic location, or business needs).

Compensation And Benefits

Pay is just one part of the overall compensation at EA.

For Canada, we offer a package of benefits including vacation (3 weeks per year to start), 10 days per year of sick time, paid top-up to EI/QPIP benefits up to 100% of base salary when you welcome a new child (12 weeks for maternity, and 4 weeks for parental/adoption leave), extended health/dental/vision coverage, life insurance, disability insurance, retirement plan to regular full-time employees. Certain roles may also be eligible for bonus and other incentive programs.

We adopt a holistic approach to our benefits programs, emphasizing physical, emotional, financial, career, and community wellness to support a balanced life. Our packages are tailored to meet local needs and may include healthcare coverage, mental well-being support, retirement savings, paid time off, family leaves, complimentary games, and more. We nurture environments where our teams can always bring their best to what they do.

PAY RANGES

• British Columbia (depending on location e.g. Vancouver vs. Victoria) *$122,300 - $170,700 CAD

About Electronic Arts

We're proud to have an extensive portfolio of games and experiences, locations around the world, and opportunities across EA. We value adaptability, resilience, creativity, and curiosity. From leadership that brings out your potential, to creating space for learning and experimenting, we empower you to do great work and pursue opportunities for growth.

Electronic Arts is an equal opportunity employer. All employment decisions are made without regard to race, color, national origin, ancestry, sex, gender, gender identity or expression, sexual orientation, age, genetic information, religion, disability, medical condition, pregnancy, marital status, family status, veteran status, or any other characteristic protected by law. We will also consider employment qualified applicants with criminal records in accordance with applicable law. EA also makes workplace accommodations for qualified individuals with disabilities as required by applicable law.