Senior Data Engineer

Foresight Works; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health

Dental

Vision

401(k)

PTO

Job summary

Foresight is seeking a Senior Data Engineer to build the data foundation behind our platform. You will construct pipelines from customer data sources to clean, trusted, modeled data powering predictions, analytics, and reporting.

This hands-on role treats data engineering as product engineering, ensuring every customer-facing number is accurate, explainable, and available when it matters. You will own core data flows and contribute to data standards.

Qualifications

• 5+ years of experience building and running production data pipelines from source to consumer.
• Strong in Python and advanced SQL, including writing and tuning complex queries.
• Deep data modeling experience in Postgres or a similar relational database: schema design, indexing, partitioning and migrations.
• Experience parsing and normalizing messy real-world data: large files, inconsistent formats, missing or contradictory values.
• Hands-on with workflow orchestration for multi-step pipelines with dependencies, retries and backfills.
• Experience with AWS data services such as S3, managed databases, streaming and serverless compute.
• Testing mindset for data: validation, quality checks, and reproducible, idempotent pipelines.
• Clear in communication and comfortable making pragmatic trade-offs on a small team.

Responsibilities

• Own pipelines from design through production: define data contracts, set freshness and quality targets, and keep them met.
• Partner with ML engineers on the features and datasets their models need, and with full stack engineers on the data behind each product feature.
• Design for failure: pipelines that retry safely, can be re-run without duplicating data, and recover cleanly from bad inputs.
• Make data observable, so problems are caught by monitors before a customer notices them.
• Keep cost in check as data volumes grow, choosing the simplest tool that does the job well.
• Help set the team's data standards and shape the platform as one of its first data engineers.

Skills

Python

Advanced SQL

Data modeling

Data pipelines

Communication

Tools

PostgreSQL

AWS

S3

ETL

Job description

About Foresight

Foresight's mission is to bring the world's critical infrastructure into service faster by better enabling teams to deliver complex capital projects.

The AI buildout runs on physical things: data centers, power, and the supply chains behind them. A single hyperscale campus involves an owner, a general contractor, dozens of trades, long-lead equipment suppliers, lenders and commissioning teams. Each works from its own schedule and its own spreadsheets. When a project slips, the people who could fix it usually find out last. For an AI data center, a month late can mean tens of millions in lost revenue.

Foresight is the control tower for these projects. We connect schedules, field progress and execution data across every party, forecast where delivery is heading, and help teams act before a delay becomes a missed date. On one 15-project portfolio, Foresight found a project 245 days behind, cut reporting overhead by 90%, and helped prevent more than $100M in lost revenue.

We work with top data-center owners, developers and general contractors across the world, including Crusoe, Applied Digital, and NTT. We raised a $25M Series A led by Macquarie Capital and are headquartered in New York.

The role

We're looking for a Senior Data Engineer to build the data foundation behind Foresight.

You will build the pipelines that take complex project data from the tools our customers already use and turn it into clean, trusted, well-modeled data. That data powers our predictions, AI recommendations, analytics and customer reporting. This is a hands-on role on a new team, for someone who cares about data being correct, explainable and available when it matters.

At Foresight, data engineering is product engineering. Every number a customer sees, every forecast and every recommendation starts with data you made trustworthy. This could translate to work on:

• Ingestion from industry tools: Build robust pipelines that bring in project data from the scheduling and planning tools the industry runs on, and handle every messy, inconsistent file along the way.

• A reliable history of every update: Model data so each update is preserved exactly as it arrived, and teams can compare any two points in time with confidence.

• Data that powers ML and AI: Build the features, training datasets and search indexes behind our predictions and AI recommendations, and keep them fresh as new data lands.

• Analytics and reporting: Build the models and pipelines behind customer dashboards, stakeholder reports and exports, plus the internal analytics we run the business on.

• Quality, isolation and governance: Build automated data-quality checks, lineage and access controls, so every customer's data stays accurate, isolated and in the region it belongs.

What you'll do

• Own pipelines from design through production: define data contracts, set freshness and quality targets, and keep them met.

• Partner with ML engineers on the features and datasets their models need, and with full stack engineers on the data behind each product feature.

• Design for failure: pipelines that retry safely, can be re-run without duplicating data, and recover cleanly from bad inputs.

• Make data observable, so problems are caught by monitors before a customer notices them.

• Keep cost in check as data volumes grow, choosing the simplest tool that does the job well.

• Help set the team's data standards and shape the platform as one of its first data engineers.

You might be a fit if you

• Have 5+ years of experience building and running production data pipelines, with ownership from source to consumer.

• Are strong in Python and advanced SQL, including writing and tuning complex queries.

• Have deep data modeling experience in Postgres or a similar relational database: schema design, indexing, partitioning and migrations.

• Have experience parsing and normalizing messy real-world data: large files, inconsistent formats, missing or contradictory values.

• Are hands‑on with workflow orchestration for multi-step pipelines with dependencies, retries and backfills.

• Have experience with AWS data services such as S3, managed databases, streaming and serverless compute.

• Havea testing mindset for data: validation, quality checks, and reproducible, idempotent pipelines.

• Are clear in communication and comfortable making pragmatic trade-offs on a small team.

Bonus points

• Experience preparing data for ML: feature pipelines, training datasets, and keeping offline and online data consistent.

• Experience with embeddings and vector search (pgvector or a search engine) for AI features.

• Event-driven data: change data capture, streaming platforms, and the outbox pattern.

• Familiarity with project scheduling data or tools from construction, engineering or energy.

• Working with regulated enterprise or public-sector data: SOC 2, GDPR, data residency.

• Infrastructure as code and CI/CD for data pipelines.

• Early-stage or founding-team experience.

None of these are required.

Why join now

• Real stakes, real customers. Real stakes, real customers. You won't be building for a hypothetical buyer. Owners and contractors are building now, on deadlines measured in revenue per day.

• Blank canvas, solid ground. You get first-engineer level scope on a product that already has paying customers, real data and proven outcomes to tell.

• Defining a category.Agentic software for physical infrastructure is a new category. What you build will become the standard.

• Small, senior team. Flat, direct, low on process. We debate openly, decide quickly and ship.

Compensation and location

• Total Comp: $180,000-240,000, based on experience, level, and location.

• Benefits: Health, Dental, Vision, 401(k), PTO .

• Location: New York, San Francisco, and Montreal are preferred. Occasional travel to may be required.

Foresight is an equal opportunity employer. We welcome applicants of every background and do not discriminate on any protected basis.