Remote Senior Data Engineer — End-to-End Pipelines

Revelio Labs, Inc.; New York (NY); Vor Ort; USD 165.000 - 215.000; Vollzeit

Zusammenfassung

Revelio Labs, Inc. is seeking a Senior Data Engineer to design, build, operate, and continually improve Revelio’s core data flow systems.

You will own production pipelines end-to-end, from ingestion and transformation to enrichment, validation, publication, and monitoring, tackling data quality and governance challenges in a messy real-world environment. You will collaborate with Data Acquisition, Infrastructure, Data Science, Economics, and client-facing teams to design scalable patterns, tune

Qualifikationen

• 4+ years of professional experience in production software or data engineering.
• Strong programming ability in Python, Go, and/or Rust.
• Strong SQL skills and ability to debug complex data quality issues.
• Experience with building or operating large production data pipelines.
• Experience with incremental processing, idempotency, data versioning, schema evolution, and operational recovery.
• Comfort working across the full data system stack.
• Experience with Flyte, Argo Workflows, Snowflake, ClickHouse, Terraform, AWS.
• Experience with large-scale web-derived datasets and data normalization problems.

Aufgaben

• Own and operate production data flow pipelines end to end: ingestion, transformation, enrichment, validation, publication, and monitoring.
• Build and improve reusable flow patterns for high-volume datasets such as profiles, companies, job postings, and enrichments.
• Debug production failures across data sources, code, orchestration, storage, and downstream layers.
• Investigate data correctness issues and discrepancies.
• Design and implement guardrails: validation checks, monitors, data contracts, runbooks, and safer release patterns.
• Improve pipeline reliability, performance, and cost efficiency by tuning resources and patterns.
• Collaborate with cross-functional teams to understand dependencies and communicate impact.
• Raise engineering bar by simplifying brittle systems and building reusable patterns.
• Apply judgment to patch, rollback, rerun, escalate, or redesign when needed.

Kenntnisse

Python

Go

Rust

SQL

Data pipelines

Data versioning

Kubernetes

Snowflake

ClickHouse

Terraform

AWS

Flyte

Argo Workflows

NLP models integration

Entity resolution

Tools

Flyte

Argo Workflows

Snowflake

ClickHouse

Terraform

AWS

Kubernetes

Jobbeschreibung

Revelio Labs, Inc. is seeking a Senior Data Engineer to design, build, operate, and continually improve Revelio’s core data flow systems.

You will own production pipelines end-to-end, from ingestion and transformation to enrichment, validation, publication, and monitoring, tackling data quality and governance challenges in a messy real-world environment. You will collaborate with Data Acquisition, Infrastructure, Data Science, Economics, and client-facing teams to design scalable patterns, tune