Senior Data Engineer

Promise & Proof; New York, Northern (NY, KY); Ibrido; USD 180.000 - 250.000; Tempo pieno

Descrizione del lavoro

Promise & Proof is seeking a Senior Data Engineer in New York to design and build data pipelines and models for AI systems. You will work on both batch and streaming data, creating low-latency services and ensuring data quality and governance.

You’ll collaborate with forward deployed engineers and client teams, with in-person presence in New York at least 25% of the time. Ideal candidates have 5+ years of production data experience, strong SQL and Python, and hands-on with

Competenze

• 5+ years of experience building production data pipelines and platforms.
• Strong SQL and Python proficiency with production-grade systems.
• Experience with Snowflake, BigQuery, or Databricks and orchestration tools.
• Ability to explain data trade-offs to non-technical stakeholders.

Mansioni

• Design and build data pipelines and models on client platforms, batch and streaming.
• Build retrieval and embedding pipelines and choose vector stores.
• Create low-latency data services for milliseconds-level decisions.
• Ensure data quality, lineage, and access controls for trusted AI.
• Turn client data into data products with clear ownership and docs.
• Document patterns as reusable reference implementations.
• Work in person with clients and the team in New York at least 25% of the time.

Conoscenze

SQL

Python

Snowflake

BigQuery

Databricks

Kafka

Embeddings

LLM prep

Formazione

Bachelor's degree

Strumenti

Airflow

dbt

Kafka

Descrizione del lavoro

About Promise & Proof

Promise & Proof is an applied AI consulting and implementation firm in New York. We find where frontier AI belongs in a business, ship it to production, and tie the result to the P&L. We are a senior team of operators and forward deployed engineers who have run data and AI inside global enterprises and built it hands-on in startups. We work in any industry, including across private equity portfolios; our published case studies come from media, streaming, sport, and ad tech.

About the role

As a Senior Data Engineer, you will build the data foundations our AI systems run on: the pipelines, models, and serving layers that turn a client’s data into something a model can use and a business can trust. Some of our builds serve decisions in milliseconds; others turn raw data into products clients sell. You will work inside client teams alongside forward deployed engineers, from the scoped brief through rollout.

Responsibilities

• Design and build data pipelines and models on client platforms, batch and streaming
• Build retrieval and embedding pipelines, and choose vector stores with the forward deployed engineers
• Build low-latency data services where decisions need to happen in milliseconds
• Put data quality, lineage, and access controls in place so AI systems can be trusted and governed
• Turn client data into data products with clear owners, documentation, and measurable use
• Document patterns as reusable reference implementations
• Work in person with clients and the team in New York at least 25% of the time, more when an engagement needs it

You may be a good fit if you

• Have 5+ years of experience building production data pipelines and platforms
• Write excellent SQL and Python
• Have worked with modern data platforms such as Snowflake, BigQuery, or Databricks, and orchestration tools such as Airflow or dbt
• Can explain data trade-offs to non-technical stakeholders
• Take ownership of quality and reliability, not only delivery

Strong candidates may also have

• Geospatial data, or high-throughput, low-latency serving at thousands of queries a second
• Audience, identity, or advertising data
• Streaming systems such as Kafka
• Experience preparing data for LLM applications: retrieval, embeddings, and evaluation sets

Compensation

The annual compensation range for this role is listed below. Annual salary: $180,000 to $250,000 USD.

Logistics

Education requirements: we require at least a bachelor’s degree in a related field, or an equivalent combination of education, training, and experience.

Location-based hybrid policy: We expect everyone to be in person in New York, with the team or at client sites, at least 25% of the time. Some engagements require more.

Visa sponsorship: we are not able to sponsor visas for this role at this time.

How we are different

We are small on purpose. The people who scope the work are the people who build it, and success is defined before the first line of code: revenue, margin, hours reclaimed, adoption. We build with whichever frontier model fits the job, and every win becomes a playbook, so what you learn on one engagement compounds on the next.

Benefits

Competitive compensation and benefits, generous vacation and parental leave, and flexible working hours.

Promise & Proof is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, veteran status, or any other characteristic protected by law.