Senior Data Platform Engineer: Build Reliable Data Pipelines

Alchemy; United States; Vor Ort; USD 120.000 - 180.000; Vollzeit

Benefits dieser Stelle

Health insurance

401k plan

Relocation bonus

Gym membership

Commuting benefits

Company retreats

Zusammenfassung

Alchemy is seeking a Data Engineer to build and operate the internal data platform supporting product usage, revenue insights, and cost-of-service metrics. You will own ingestion, schema changes, and data-quality checks, collaborating with analytics engineers and data scientists to define datasets for reporting and AI tools.

You will develop batch and streaming pipelines connecting operational systems to the data platform, ensuring recoverability, testing, and safe backfills.

Qualifikationen

• Strong SQL and Python skills, including testing, debugging, and maintainable software design.
• Understanding of data modeling, incremental processing, schema evolution, and data-quality testing.
• Good judgment about stakeholder needs: turn recurring requests into reusable datasets or tooling, and make trade-offs clear.
• Experience independently building and operating production data pipelines and the infrastructure around them.
• The ability to trace a data discrepancy from a downstream report back to its source, explain the cause, and fix it.
• Experience with workflow orchestration and transformation tools such as Airflow and dbt, or equivalent systems.
• Experience with Snowflake or a comparable warehouse, including performance tuning and access management.
• Kafka, database replication, or change-data-capture systems.
• Apache Iceberg, Spark, Trino, or other lakehouse and query technologies.
• Usage metering, billing and revenue data, or semantic layers for self-service analytics.
• Infrastructure as code, data observability, lineage, and disaster recovery

Aufgaben

• Build and run the internal data platform that helps Alchemy understand product usage, revenue, and the cost of serving customers.
• Own ingestion, schema changes, and checks that show whether shared datasets are current and correct.
• Work with analytics engineers and data scientists to define datasets for reporting, analysis, and AI-assisted tools.
• Build batch and streaming pipelines that connect operational databases, product events, billing systems, and external sources to the internal data platform.
• Make ingestion recoverable through change-data capture, schema-change handling, idempotent processing, and safe backfills.
• Operate and improve Snowflake, including access controls, workload management, query performance, and compute and storage costs.
• Improve orchestration and deployment workflows so engineers can test, ship, and debug pipelines safely.
• Build data-quality checks that detect stale, missing, duplicated, or inconsistent records before they reach downstream consumers.
• Work with analytics partners on shared data models, source definitions, and documentation. Make dataset ownership and upstream dependencies clear.
• Reduce recurring manual work and build reusable tools for self-service data access.
• Participate in production support and incident response for pipelines and the data platform. Restore data flow, reconcile affected datasets, and fix the causes of recurring failures.
• Within three months: You own a production pipeline and have shipped an improvement to its freshness, reliability, or operating cost.
• Downstream users can see whether its data is current, recovery procedures have been tested, and a recurring manual request has become a reusable dataset or tool

Kenntnisse

SQL

Python

Data modeling

Data quality testing

Stakeholder management

Production data pipelines

Airflow

dbt

Snowflake

Kafka

Spark

Trino

Iceberg

Data observability

Tools

Snowflake

Airflow

dbt

Kafka

Iceberg

Spark

Trino

Jobbeschreibung

Alchemy is seeking a Data Engineer to build and operate the internal data platform supporting product usage, revenue insights, and cost-of-service metrics. You will own ingestion, schema changes, and data-quality checks, collaborating with analytics engineers and data scientists to define datasets for reporting and AI tools.

You will develop batch and streaming pipelines connecting operational systems to the data platform, ensuring recoverability, testing, and safe backfills.