Backend Engineer, Collection Platform

Vannevar; United States; Remote; USD 120,000 - 165,000; Full time

Benefits offered by this job

Health insurance

Remote-friendly with WeWork access

Unlimited PTO and year-end downtime

401(k) match

Wellbeing stipend

Military reserve duty pay top-up

Parental leave

Travel care reimbursement

Job summary

Vannevar builds AI systems for national security missions. As part of the Collection Platform team, you’ll own web scrapers, data pipelines, and third-party integrations to deliver complete, timely data even with unreliable upstream systems.

You’ll partner with mission teams, product managers, and engineers to balance data quality, reliability, scale, and cost while shaping system architecture and standards.

Qualifications

• 4+ years of backend engineering experience with production web scraping or data ingestion.
• Proficient in Python and relational databases; solid API design.
• Strong focus on data quality across large datasets.
• Experience designing systems tolerant of upstream failures and changing interfaces.

Responsibilities

• Build and operate production web scrapers, data pipelines, and integrations.
• Design for schema changes, rate limits, authentication, and partial failures.
• Implement retry, backfill, deduplication, idempotency, and replay mechanisms.
• Define data completeness, correctness, freshness, duplication, and provenance metrics.
• Develop APIs, databases, and shared services for collection workflows.
• Collaborate with mission teams, product managers, and engineers to evaluate sources and new data requirements.
• Influence architecture and engineering standards for collection systems.

Skills

Backend engineering

Python

API design

Data quality

Reliability & scalability

Communication

Tools

PostgreSQL

REST APIs

Job description

Vannevar builds AI systems for the Department of War’s most consequential missions. We have 125 deployments across every branch and combatant command spanning our core platform and five distinct products. An an example of how we work, when major combat operations started with Iran, we fielded a new product supporting 24/7 operations and 9,000+ users in three months.

How we build is our advantage: we deploy forward with the people who own the mission. Our engineers, product team, and CTO deploy forward to the point of friction, including visiting units in Ukraine. We focus on embedding directly with operational units supporting great power competition with China, combat operations with Iran, and counter-narcotics missions.

About the Role

At Vannevar, we build mission software for the national security, defense, and intelligence communities. The Collection Platform team builds the systems that acquire mission-critical data from websites, APIs, feeds, and third-party platforms.

In this role, you’ll own web scrapers, third-party integrations, and data pipelines that produce complete, correct, and timely data despite unreliable upstream systems. You’ll partner with mission teams, product managers, and engineers to define requirements and make tradeoffs among data quality, reliability, scale, and cost.

What you’ll do

• Build and operate production web scrapers, third-party integrations, and data pipelines.
• Design for schema changes, rate limits, authentication requirements, and partial failures.
• Build reliable retry, backfill, deduplication, idempotency, and replay mechanisms.
• Define and monitor data completeness, correctness, freshness, duplication, and provenance.
• Develop APIs, databases, and shared services for collection workflows.
• Partner with mission teams, platform teams, and product managers to evaluate sources and support new data requirements.
• Influence architecture, technology selection, and engineering standards for collection systems.

What we’re looking for

• You have 4+ years of backend engineering experience, including production web scraping, data ingestion, ETL, or third-party integrations.
• You write production software in Python and are familiar with relational databases and API design.
• You know how to maintain data quality across large, changing datasets.
• You have designed systems that tolerate upstream failures and changing external interfaces.
• You can make thoughtful tradeoffs among reliability, scalability, performance, maintainability, and cost.
• You communicate technical decisions clearly and work well across engineering, product, and mission teams.
• U.S. Person status is required, as this role requires access to U.S.-only data systems and the ability to obtain a security clearance.

Nice to Have

• Experience with distributed task processing, workflow orchestration, or large-scale data pipelines.
• Experience monitoring data quality and investigating production regressions.

What we offer

We’re proud to offer competitive benefits that support our employees. Some key highlights of our benefits package include:

• Health, dental, and vision insurance
• Remote friendly with WeWork access
• Unlimited PTO, shared downtime during the federal holiday calendar, and company-wide off time at the end of each year
• 401(k) match
• Lifestyle & wellbeing stipends
• Salary top-up during military reserve duty
• Fully paid parental leave
• Child and pet care reimbursement during travel

Vannevar is an equal opportunity employer, and qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status.