Senior Data Engineer — Low-Latency AI Pipelines (Hybrid NYC)

Promise & Proof; New York, Northern (NY, KY); Hybrid; USD 180,000 - 250,000; Full time

Job summary

Promise & Proof is seeking a Senior Data Engineer in New York to design and build data pipelines and models for AI systems. You will work on both batch and streaming data, creating low-latency services and ensuring data quality and governance.

You’ll collaborate with forward deployed engineers and client teams, with in-person presence in New York at least 25% of the time. Ideal candidates have 5+ years of production data experience, strong SQL and Python, and hands-on with

Qualifications

• 5+ years of experience building production data pipelines and platforms.
• Strong SQL and Python proficiency with production-grade systems.
• Experience with Snowflake, BigQuery, or Databricks and orchestration tools.
• Ability to explain data trade-offs to non-technical stakeholders.

Responsibilities

• Design and build data pipelines and models on client platforms, batch and streaming.
• Build retrieval and embedding pipelines and choose vector stores.
• Create low-latency data services for milliseconds-level decisions.
• Ensure data quality, lineage, and access controls for trusted AI.
• Turn client data into data products with clear ownership and docs.
• Document patterns as reusable reference implementations.
• Work in person with clients and the team in New York at least 25% of the time.

Skills

SQL

Python

Snowflake

BigQuery

Databricks

Kafka

Embeddings

LLM prep

Education

Bachelor's degree

Tools

Airflow

dbt

Kafka

Job description

Promise & Proof is seeking a Senior Data Engineer in New York to design and build data pipelines and models for AI systems. You will work on both batch and streaming data, creating low-latency services and ensuring data quality and governance.

You’ll collaborate with forward deployed engineers and client teams, with in-person presence in New York at least 25% of the time. Ideal candidates have 5+ years of production data experience, strong SQL and Python, and hands-on with