Senior Backend Data Pipeline Engineer

Sunset; Northern, New York (KY, NY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Sunset is seeking a backend engineer to build the de-identification pipeline for enterprise data, ensuring replayability, operability, and safe changes. You will own backend systems and contracts that surface failure modes, enabling auditability and safe releases.

You will work across asynchronous orchestration, storage, databases, and model-backed stages, with a product and delivery focus. You will collaborate with ML, data, platform, security, and product teams to deliver reliable data

Qualifications

• You have at least three years of professional software engineering experience, including personal ownership of production backend systems.
• Strong in asynchronous or distributed systems and can reason precisely about queues, concurrency, state, storage, idempotency, partial failure, and recovery.
• Experience with systems where output could be wrong even when services appear healthy.
• Define invariants and use reconciliation, diffs, replay, or independent sources to verify correctness.
• Design versioned data and artifact contracts and migrate them safely in a live system.
• Debug from evidence across system boundaries and turn incidents into durable improvements.
• You evaluate technical work by operator and customer impact, not architecture alone.
• Proficient with AI engineering tools, with model-backed checks and guardrails.

Responsibilities

• Design and ship backend systems for multi-stage, high-volume data processing.
• Define authoritative, versioned contracts for manifests, artifacts, lineage, and state transitions.
• Make retries, checkpoints, partial failures, replay, backfills, migrations, and rollbacks safe and understandable.
• Build independent reconciliation and verification instead of treating job success as proof of correct output.
• Turn escaped and recurring failures into fixtures, regression coverage, release gates, and durable recovery paths.
• Expose trustworthy run state and safe controls to products teams for data investigation and release.
• Diagnose production behavior across code, queues, stores, artifacts, data formats, and deployed versions.
• Improve correctness, throughput, and operating leverage without weakening privacy or security.
• Use AI deeply in development and in bounded verification systems with checks.

Job description

Sunset is seeking a backend engineer to build the de-identification pipeline for enterprise data, ensuring replayability, operability, and safe changes. You will own backend systems and contracts that surface failure modes, enabling auditability and safe releases.

You will work across asynchronous orchestration, storage, databases, and model-backed stages, with a product and delivery focus. You will collaborate with ML, data, platform, security, and product teams to deliver reliable data