Software Test Engineer

BigBear.ai; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

BigBear.ai is seeking a Software Test Engineer to strengthen automated validation across our AI platform. The role focuses on scalable regression coverage, backend test automation, and reliable release signaling within a distributed system environment.

You will collaborate with engineers to define acceptance tests, improve test infrastructure, and validate APIs, authentication, and AI workflow paths while minimizing test cost and latency.

Qualifications

• 5-8 years of experience in software testing.
• Strong software engineering background with automated test development.
• Experience testing backend services, APIs, distributed systems, and database-backed applications.
• Experience designing smoke, unit, integration, and end-to-end testing strategies.
• Strong judgment around edge cases, non-happy paths, adversarial inputs, regression risk, and failure isolation.
• Experience improving CI test reliability, execution time, parallelization, test isolation, and failure observability.

Responsibilities

• Develop and maintain smoke tests, unit tests, integration tests, and end-to-end tests across backend and platform workflows.
• Validate backend changes for new PRs through automated test coverage, direct API testing, and user-facing workflow verification where appropriate.
• Exercise backend behavior through the UI when useful, without owning visual design, frontend UX validation, or manual UI regression testing.
• Build regression coverage for edge cases, malformed inputs, authorization boundaries, concurrency issues, failure modes, and other non-happy-path scenarios.
• Improve the scalability, determinism, execution performance, maintainability, and diagnostic quality of the test suite.
• Strengthen test fixtures, mocks, test data management, failure analysis, and CI feedback loops.
• Design end-to-end tests for AI platform workflows while minimizing unnecessary token usage, external provider calls, latency, and test cost.
• Apply AI-assisted development and analysis tools to accelerate test design, test generation, triage, and maintenance while preserving reliability, reviewability, performance, and deterministic validation standards.
• Validate backend APIs, authentication flows, billing and token behavior, model routing, AI workflow execution, file parsing, MCP/tool execution, and passthrough APIs.
• Validate AI platform E2E paths involving prompts, model responses, streaming behavior, tool calls, agents, workflow orchestration, and provider-facing API compatibility.
• Validate agentic harnesses, MCP integrations, and workflow automation systems, including Power Automate, Zapier, Make, n8n.

Skills

Automated test development

Backend testing

Distributed systems

CI/CD testing

API validation

Tools

Docker

Kubernetes

PostgreSQL

GitHub Actions

Observability tooling

Job description

Residency

All applicants must currently reside in the United States

OverviewAsk Sage,

ResidencyAll applicants must currently reside in the United StatesOverviewAsk Sage, a BigBear.ai company, is hiring a Software Test Engineer in to strengthen automated validation across our AI platform. We have a growing backlog of backend and platform work that requires deeper regression coverage, stronger system-level validation, and more scalable test execution. This is not a manual QA role, and it is not a DevSec role. We are looking for a software engineer who specializes in quality engineering: someone who can understand backend code, reason through distributed system behavior, write high-quality automated tests, and improve the infrastructure that gives engineers reliable release signal. You will partner with engineers to define acceptance tests that prove a fix works and fail reliably if the fix is reverted, and your work will directly improve release velocity, platform reliability, and confidence in complex AI workflows.

What You Will Do

• Develop and maintain smoke tests, unit tests, integration tests, and end-to-end tests across backend and platform workflows.
• Validate backend changes for new PRs through automated test coverage, direct API testing, and user-facing workflow verification where appropriate.
• Exercise backend behavior through the UI when useful, without owning visual design, frontend UX validation, or manual UI regression testing.
• Build regression coverage for edge cases, malformed inputs, authorization boundaries, concurrency issues, failure modes, and other non-happy-path scenarios.
• Improve the scalability, determinism, execution performance, maintainability, and diagnostic quality of the test suite.
• Strengthen test fixtures, mocks, test data management, failure analysis, and CI feedback loops.
• Design end-to-end tests for AI platform workflows while minimizing unnecessary token usage, external provider calls, latency, and test cost.
• Apply AI-assisted development and analysis tools to accelerate test design, test generation, triage, and maintenance while preserving reliability, reviewability, performance, and deterministic validation standards.
• Validate backend APIs, authentication flows, billing and token behavior, model routing, AI workflow execution, file parsing, MCP/tool execution, and passthrough APIs.
• Validate AI platform E2E paths involving prompts, model responses, streaming behavior, tool calls, agents, workflow orchestration, and provider-facing API compatibility.
• Validate agentic harnesses, MCP integrations, and workflow automation systems, including concepts common to no-code and low-code workflow builders such as Power Automate, Zapier, Make, n8n, and similar platforms.
• Validate security-sensitive product behavior such as user isolation, permission boundaries, validation, sanitization, rate limits, replay prevention, safe error handling, and layered control behavior.
• Build and maintain test infrastructure that must scale with a growing platform, expanding product surface area, and active engineering team.

What You Need To Have

• 5-8 years of experience with software testing
• Strong software engineering background with deep experience in automated test development.
• Experience testing backend services, APIs, distributed systems, and database-backed applications.
• Experience designing smoke, unit, integration, and end-to-end testing strategies.
• Strong judgment around edge cases, non-happy paths, adversarial inputs, regression risk, and failure isolation.
• Experience improving CI test reliability, execution time, parallelization, test isolation, and failure observability.
• Familiarity with testing Defense in Depth behavior: validating that multiple layers of checks work together, without this being a dedicated DevSec role.
• Strong troubleshooting, analytical, and communication skills.
• Ability to work independently and as part of a team.
• Ability to obtain clearance.

What We'd Like You To Have

• Experience writing E2E tests for AI platforms, LLM applications, model gateways, agents, MCP tools, or tool-using systems.
• Experience designing AI E2E tests that control token usage, external provider calls, latency, and cost.
• Experience testing workflow automation systems, agentic harnesses, or no-code/low-code automation platforms (e.g., Power Automate, Zapier, Make, n8n).
• Familiarity with flagship Generative AI provider APIs (Google VertexAI, AWS Bedrock, Microsoft Azure OpenAI) and models (OpenAI GPT, Anthropic Claude, Google Gemini).
• Experience with CI/CD pipelines (e.g., GitHub Actions), Docker, Kubernetes, and observability/monitoring tooling.
• Experience with PostgreSQL and SQL for validating stored data.
• Knowledge of government compliance frameworks (FedRAMP, NIST AI RMF, CMMC 2.0).

Pay transparencyPlease note the targeted compensation range is provided as an estimate, and any actual compensation offer may vary depending on the needs of the company, or an applicant's skillset, competencies, experience, education, certifications, location, or other factors. The estimated range does not include the value of any benefits offered.

About BigBear.aiBigBear.ai is a leading provider of AI-powered decision intelligence solutions for national security, supply chain management, and digital identity. Customers and partners rely on Bigbear.ai’s predictive analytics capabilities in highly complex, distributed, mission-based operating environments. Headquartered in McLean, Virginia, BigBear.ai is a public company traded on the NYSE under the symbol BBAI. For more information, visit https://bigbear.ai/ and follow BigBear.ai on LinkedIn: @BigBear.ai and X: @BigBearai. BigBear.ai is an Equal opportunity employer all protected groups, including protected veterans and individuals with disabilities.