Software Development Engineer in Test - Agentic Automation (Remote, CAN)

CrowdStrike Canada ULC; Alberta; Hybrid; CAD 120,000 - 170,000; Full time

Benefits offered by this job

Competitive compensation

Equity awards

Medical and wellness programs

Parental leave

Professional development

Job summary

CrowdStrike is seeking a hands-on Software Development Engineer in Test to shape how AI agents are tested in production. You’ll build and maintain automated test suites for our cloud‑based platform and help define eval‑driven testing workflows.

You’ll design agentic test generation, maintain datasets and scoring pipelines, and enforce CI eval gates so every change is measured before release. This role offers collaboration across distributed teams, competitive compensation, equity and

Qualifications

• Degree in Computer Science or equivalent experience.
• 5+ years in software quality; 3+ years SDET/test automation.
• Strong programming in Java, Node.js or Python; Python preferred.
• Experience designing, building, and maintaining test frameworks integrated with CI/CD pipelines (e.g., Jenkins, GitLab CI/CD).
• Experience testing web services and APIs (REST, gRPC, JSON).
• Hands-on performance and load testing with Locust or k6.

Responsibilities

• Design, develop and maintain automated test suites for distributed, cloud-based systems.
• Maintain and extend agentic test generation and analysis tooling.
• Build AI-powered workflows and agent skill files for test automation.
• Maintain eval suites, datasets and scoring pipelines for AI agents.
• Enforce CI eval gates before release and feed production failures back into eval sets.
• Detect and mitigate unreliable AI output and flaky tests.
• Design and run performance/load tests to validate scalability.
• Validate integration of platform systems end-to-end.
• Review designs for testability and evaluability.
• Share knowledge and contribute to quality standards.

Skills

Java

Node.js

Python

Go

Distributed systems

Test automation

CI/CD

Education

Bachelor's degree in Computer Science

Tools

Jenkins

GitLab CI/CD

Locust

k6

Docker

Kafka

MongoDB

Terraform

Nomad

Job description

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We're proud to work for a mission-driven company leveraging AI to transform the way we work. CrowdStrikers drive their careers through flexibility and autonomy while also being expected to contribute to a culture of responsible AI adoption, experimentation, and innovation. We use an AI-first mindset as a force multiplier to proactively and continuously accelerate execution, build expertise, uncover insights, and solve complex problems. We’re always looking to add talented CrowdStrikers to the team who have limitless passion, a relentless focus on innovation and a fanatical commitment to our customers, our community and each other. Ready to join a mission that matters? The future of cybersecurity starts with you.

About the Role

Are you an SDET who wants to define how AI agents are tested, evaluated and trusted in production? The CrowdStrike Detonations and Threat Analysis Platform Engineering team is hiring a hands‑on Software Development Engineer in Test. You'll build and maintain the functional, integration and performance automation that keeps our large‑scale, cloud‑based cybersecurity platform reliable. You'll also help shape how we test AI. Our core testing frameworks include agentic test generation and analysis, and we use eval‑driven development (EDD) to validate the AI agents we build.

What You’ll Do

• Design, develop and maintain automated test suites and frameworks for distributed, cloud-based systems
• Maintain and extend the agentic test generation and analysis tooling built into our major testing frameworks
• Build AI-powered workflows and agent skill files that automate manual test toil, such as test data setup, failure triage and release validation
• Maintain eval suites, golden datasets and scoring pipelines for our AI agents, scoring correctness, tool use, trajectory, faithfulness, safety, latency and cost
• Enforce CI eval gates so that every change to prompts, models, tools or orchestration is measured before release, and feed production failures back into the eval sets
• Detect and mitigate unreliable AI output, including hallucinated assertions, flaky or redundant tests, incorrect failure triage and judge bias
• Design, run and analyze performance, load and stress tests to find bottlenecks and validate scalability across distributed services
• Validate the integration of platform systems, services, data and applications, as well as customer-facing features end-to-end
• Review product, system and agent designs for testability, evaluability, flaws and improvements
• Share knowledge and contribute to the team's quality and evaluation standards

What You’ll Use

Languages: Java, Node.js, Python, Go Data & Messaging: MongoDB, MinIO, RabbitMQ, Kafka Cloud & Infrastructure: AWS (EC2, S3, DynamoDB), Docker, Nomad, Terraform, Unix/Linux CI/CD & Tooling: Git, Jenkins, GitLab CI/CD, Jira, Bitbucket Performance Testing: Locust, k6 AI/Agentic & Evals: LLM APIs, agent frameworks and LLM eval tooling

What You’ll Need

• Degree in Computer Science or equivalent experience (OOP, data structures, algorithms, distributed systems)
• 5+ years in software quality, including 3+ years of hands‑on SDET or test automation development
• Strong programming skills in at least one of Java, Node.js or Python, with Python preferred for eval and agent work
• Proven experience designing, building and maintaining test frameworks integrated with CI/CD pipelines (e.g., Jenkins, GitLab CI/CD)
• Experience testing web services and APIs (REST, gRPC, JSON)
• Hands‑on experience with performance and load testing using tools such as Locust or k6, including designing realistic workloads and interpreting results (latency percentiles, throughput, error rates)
• A skeptical, verification‑first mindset toward AI output, with a clear sense of when to use deterministic assertions and when to use probabilistic evaluation
• Proven experience utilizing AI technologies to enhance decision‑making, streamline workflows and processes, improve efficiency and drive business outcomes

Bonus Points

• End‑to‑end eval‑driven development experience, including release gating, trajectory and tool‑use evals, and calibrating LLM‑as‑judge against human review
• Prompt engineering and context or tool design for agents
• Integrating performance tests into CI/CD and using observability tools (e.g., Grafana, Prometheus, CloudWatch)
• Background in cybersecurity, threat intelligence, malware analysis or sandboxing/detonation environments

Benefits of Working at CrowdStrike

• Market leader in compensation and equity awards
• Comprehensive physical and mental wellness programs
• Competitive vacation and holidays for recharge
• Paid parental and adoption leaves
• Professional development opportunities for all employees regardless of level or role
• Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
• Vibrant office culture with world class amenities
• Great Place to Work Certified™ across the globe

CrowdStrike is proud to be an equal opportunity employer. We are committed to fostering a culture of belonging where everyone is valued for who they are and empowered to succeed. We support veterans and individuals with disabilities through our affirmative action program. CrowdStrike is committed to providing equal employment opportunity for all employees and applicants for employment. The Company does not discriminate in employment opportunities or practices on the basis of race, color, creed, ethnicity, religion, sex (including pregnancy or pregnancy‑related medical conditions), sexual orientation, gender identity, marital or family status, veteran status, age, national origin, ancestry, physical disability (including HIV and AIDS), mental disability, medical condition, genetic information, membership or activity in a local human rights commission, status with regard to public assistance, or any other characteristic protected by law. We base all employment decisions–including recruitment, selection, training, compensation, benefits, discipline, promotions, transfers, lay‑offs, return from lay‑off, terminations and social/recreational programs–on valid job requirements. If you need assistance accessing or reviewing the information on this website or need help submitting an application for employment or requesting an accommodation, please contact us at [e-mail] for further assistance.

The base salary range for this position in Canada is $120,000 - $170,000 CAD per year + variable/incentive compensation + equity + benefits. This is Canadian-based employment, and it is expected that all employees maintain legal entitlement to work in Canada. Applicants selected to move forward in the hiring process are subject to background checks, including but not limited to criminal record, credit, and/or reference checks.

CrowdStrike was founded in 2011 to fix a fundamental problem: The sophisticated attacks that were forcing the world’s leading businesses into the headlines could not be solved with existing malware‑based defenses. Founder George Kurtz realized that a brand new approach was needed — one that combines the most advanced endpoint protection with expert intelligence to pinpoint the adversaries perpetrating the attacks, not just the malware. There’s much more to the story of how Falcon has redefined endpoint protection but there’s only one thing to remember about CrowdStrike: We stop breaches.