Senior Data Engineer

Foresight Works Ltd; New York (NY); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health insurance

Dental

Vision

401(k)

PTO

Job summary

Foresight is seeking a Senior Data Engineer in New York to build the data foundation behind our AI-powered platform. You will transform complex project data from customer tools into clean, well-modeled data powering predictions, analytics and reporting.

This hands-on role treats data engineering as product engineering: every customer-facing number starts with trusted data. You will ingest industry data, preserve update histories, enable ML features, and build dashboards while enforcing data

Qualifications

• 5+ years of experience building and running production data pipelines (source to consumer).
• Strong Python and advanced SQL skills, including complex queries.
• Deep data modeling experience in PostgreSQL or similar relational DBs: schema design, indexing, migrations.
• Experience parsing and normalizing messy real-world data (large files, inconsistent formats).
• Hands-on with multi-step workflow orchestration and dependency management.

Responsibilities

• Own pipelines from design through production: define data contracts, freshness targets, and quality metrics.
• Collaborate with ML engineers and full-stack engineers on data behind product features.
• Design for failure: safe retries, idempotent re-runs, and clean recovery from bad inputs.
• Make data observable with monitors to catch issues before customers notice.
• Control costs as data volumes grow and choose simple, effective tools.
• Help define data standards and shape the platform as one of the first data engineers.

Skills

Python

Advanced SQL

PostgreSQL

Data modeling

AWS data services

Data pipelines

Workflow orchestration

Tools

AWS S3

PostgreSQL

Job description

About Foresight

Foresight's mission is to bring the world's critical infrastructure into service faster by better enabling teams to deliver complex capital projects.

The AI buildout runs on physical things: data centers, power, and the supply chains behind them. A single hyperscale campus involves an owner, a general contractor, dozens of trades, long-lead equipment suppliers, lenders and commissioning teams. Each works from its own schedule and its own spreadsheets. When a project slips, the people who could fix it usually find out last. For an AI data center, a month late can mean tens of millions in lost revenue.

Foresight is the control tower for these projects. We connect schedules, field progress and execution data across every party, forecast where delivery is heading, and help teams act before a delay becomes a missed date. On one 15-project portfolio, Foresight found a project 245 days behind, cut reporting overhead by 90%, and helped prevent more than $100M in lost revenue.

We work with top data-center owners, developers and general contractors across the world, including Crusoe, Applied Digital, and NTT. We raised a $25M Series A led by Macquarie Capital and are headquartered in New York.

The role

We're looking for a Senior Data Engineer to build the data foundation behind Foresight.

You will build the pipelines that take complex project data from the tools our customers already use and turn it into clean, trusted, well-modeled data. That data powers our predictions, AI recommendations, analytics and customer reporting. This is a hands-on role on a new team, for someone who cares about data being correct, explainable and available when it matters.

At Foresight, data engineering is product engineering. Every number a customer sees, every forecast and every recommendation starts with data you made trustworthy. This could translate to work on:

• Ingestion from industry tools: Build robust pipelines that bring in project data from the scheduling and planning tools the industry runs on, and handle every messy, inconsistent file along the way.

• A reliable history of every update: Model data so each update is preserved exactly as it arrived, and teams can compare any two points in time with confidence.

• Data that powers ML and AI: Build the features, training datasets and search indexes behind our predictions and AI recommendations, and keep them fresh as new data lands.

• Analytics and reporting: Build the models and pipelines behind customer dashboards, stakeholder reports and exports, plus the internal analytics we run the business on.

• Quality, isolation and governance: Build automated data-quality checks, lineage and access controls, so every customer's data stays accurate, isolated and in the region it belongs.

What you'll do

• Own pipelines from design through production: define data contracts, set freshness and quality targets, and keep them met.

• Partner with ML engineers on the features and datasets their models need, and with full stack engineers on the data behind each product feature.

• Design for failure: pipelines that retry safely, can be re-run without duplicating data, and recover cleanly from bad inputs.

• Make data observable, so problems are caught by monitors before a customer notices them.

• Keep cost in check as data volumes grow, choosing the simplest tool that does the job well.

• Help set the team's data standards and shape the platform as one of its first data engineers.

You might be a fit if you

• Have 5+ years of experience building and running production data pipelines, with ownership from source to consumer.

• Are strong in Python and advanced SQL, including writing and tuning complex queries.

• Have deep data modeling experience in Postgres or a similar relational database: schema design, indexing, partitioning and migrations.

• Have experience parsing and normalizing messy real-world data: large files, inconsistent formats, missing or contradictory values.

• Are hands‑on with workflow orchestration for multi-step pipelines with dependencies, retries and backfills.

• Have experience with AWS data services such as S3, managed databases, streaming and serverless compute.

• Havea testing mindset for data: validation, quality checks, and reproducible, idempotent pipelines.

• Are clear in communication and comfortable making pragmatic trade-offs on a small team.

Bonus points

• Experience preparing data for ML: feature pipelines, training datasets, and keeping offline and online data consistent.

• Experience with embeddings and vector search (pgvector or a search engine) for AI features.

• Event-driven data: change data capture, streaming platforms, and the outbox pattern.

• Familiarity with project scheduling data or tools from construction, engineering or energy.

• Working with regulated enterprise or public-sector data: SOC 2, GDPR, data residency.

• Infrastructure as code and CI/CD for data pipelines.

• Early-stage or founding-team experience.

None of these are required.

Why join now

• Real stakes, real customers. Real stakes, real customers. You won't be building for a hypothetical buyer. Owners and contractors are building now, on deadlines measured in revenue per day.

• Blank canvas, solid ground. You get first-engineer level scope on a product that already has paying customers, real data and proven outcomes to tell.

• Defining a category.Agentic software for physical infrastructure is a new category. What you build will become the standard.

• Small, senior team. Flat, direct, low on process. We debate openly, decide quickly and ship.

Compensation and location

• Total Comp: $180,000-240,000, based on experience, level, and location.

• Benefits: Health, Dental, Vision, 401(k), PTO .

• Location: New York, San Francisco, and Montreal are preferred. Occasional travel to may be required.

Foresight is an equal opportunity employer. We welcome applicants of every background and do not discriminate on any protected basis.