Senior Data Engineer

Venturatalents; Virginia (MN); Remote; USD 103,000 - 165,000; Full time

Job summary

Venturatalents is seeking a Senior Data Engineer for a 1099 contract. The role focuses on building and operating Spark pipelines, data models, and production REST APIs within an Ad-Media Measurement data platform.

You will work on the Ad Optimization team, handling attribution, impressions, and partner integrations from a remote US-based location. The ideal candidate has 7+ years in software/data engineering, strong Spark with Java and PySpark, SQL expertise, and hands-on AWS/Airflow experience.

Qualifications

• :7+ years of professional experience in software or data engineering
• 4+ years building, optimizing, and maintaining production Spark pipelines with Java and Python
• Advanced SQL, relational modeling and large-scale data processing; production experience with Snowflake, MySQL, Iceberg
• 3+ years operating AWS data workloads using S3 and Airflow; container orchestration a plus
• 3+ years building, changing, and operating production REST APIs
• Production full-stack experience with PHP, Symfony, Doctrine, Twig, and JavaScript
• Demonstrated AI-assisted development with AI tools for code generation, review, and documentation
• Ability to deliver secure, backward-compatible API changes
• Experience with external provider/client integrations: schemas, APIs, file delivery, identity fields, SLAs

Responsibilities

• Build, optimize, and operate production Apache Spark pipelines for attribution and optimization feeds
• Design and evolve data processing and models in data lake/warehouse stack
• Build and operate partner optimization feeds and integrations including schemas and IDs
• Develop and maintain production REST APIs (PHP) end-to-end
• Trace and modify application code paths and tests
• Diagnose production issues and manage releases/rollbacks
• Write unit and integration tests for pipelines, APIs, and feeds

Skills

Senior level experience

Java (JVM Spark)

Python (PySpark)

Advanced SQL

Snowflake

MySQL

Iceberg

AWS (S3)

Airflow

REST APIs development

PHP / Symfony

AI-assisted development

Tools

Spark pipelines

AWS

REST APIs

PHP

Symfony

Doctrine

Twig

JavaScript

Snowflake

MySQL

Iceberg

Airflow

Job description

Type: Contract - 1099 | 6 Months + Extendable
Work Schedule: 40 Hours / Week
Location: Remote - US Based Only
Work Authorization: Must be authorized to work in the US legally | US Citizen / Green Card Only

Salary will be discussed on call once we have your details.

About The Role

We are hiring for a Senior Data Engineer for an end-client in the Ad-Media Measurement space. This is for the Ad Optimization team - the revenue-facing edge of the data platform that turns attributed conversions and impressions into optimization feeds that external partners use to make their advertising models smarter.

This is a Contractor (1099) role and includes availability for occasional off-hours release / incident support. Formal on-call is not required unless separately established.

Responsibilities

• Build, optimize, and operate production Apache Spark pipelines that generate attributed conversion and optimization feeds from large-scale impression and conversion data
• Design and evolve data processing and models across modern data lake and warehouse technologies, orchestrated with a production workflow scheduler on AWS
• Build and operate partner optimization feeds and integrations with external partners, including schemas, identity fields and unique IDs, file delivery, reconciliation, and SLAs
• Build, change, and operate production REST APIs, including PHP APIs, delivering secure, backward-compatible changes end-to-end: implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics
• Trace and modify behavior across controllers, database queries, templates, JavaScript, and automated tests
• Diagnose production failures, reconcile partner-facing data, execute backfills, and manage staged releases and rollbacks
• Write and maintain unit and integration tests across pipelines, APIs, and feeds

Required Qualifications - Must Haves

This is a strict filter. Please do not submit without this.

• 7+ years of professional experience in software or data engineering
• 4+ years building, optimizing, and maintaining production Apache Spark pipelines, with strong Java (JVM Spark) AND Python (PySpark) skills
• Advanced SQL, relational data modeling, and large-scale data processing; production experience with Snowflake, MySQL, and Iceberg
• 3+ years operating AWS data workloads using S3 and Airflow (or equivalent production orchestration); container orchestration and data-catalog experience a plus
• 3+ years building, changing, and operating production REST APIs
• Production full-stack experience with PHP, Symfony (or comparable PHP MVC), Doctrine, Twig, and JavaScript
• Demonstrated AI-assisted development - Must be comfortable using AI tools for code generation, code review, documentation, investigation, working with agentic code harnesses, and spec-driven development, with judgment on when NOT to trust AI output. This is an AI-forward engineering org.
• Ability to independently deliver secure, backward-compatible API changes
• Experience with external provider / client integrations involving schemas, APIs, file delivery, identity fields, reconciliation, and SLAs

Preferred / Nice to Have

• Direct experience building partner optimization or advertising feeds with external platforms (DSPs, publishers, measurement partners)
• Experience in AdTech / Advertising measurement - attribution, conversions, audience/impression data, identity matching
• Experience with deterministic and probabilistic identity matching, unique IDs, device IDs, identity graphs
• Familiarity with edge/log delivery infrastructure and pixel/impression tracking
• Experience with privacy and compliance-driven workflows (deletion, opt-out, suppression, retention)
• Experience with data lake table formats and query engines at large scale

Interview Process

Three rounds. Coding interview is possible.

Screening Checklist for Submittal:

1. 7+ yrs SWE/Data Eng?
2. 4+ yrs Java (JVM Spark) + Python (PySpark) prod?
3. 3+ yrs S3 + Airflow/orchestration?
4. 3+ yrs building/operating prod REST APIs?
5. AI-assisted dev tools in professional capacity?
6. Prod experience with Snowflake, MySQL, Iceberg?
7. PHP/Symfony, Doctrine, Twig, JS prod experience?
8. US Citizen / Green Card?
9. Open to 1099, 40hrs/week, 6-month contract?
10. Notice period / How soon can join?