AI‑Ready Real-Time Data Platform Engineer

Harbor Compliance; United States; On-site; USD 172,000 - 215,000; Full time

Benefits offered by this job

Health benefits

Flexible paid time off

Parental leave

Fertility and adoption assistance

401(k)

Educational reimbursement

Job summary

Harbor Compliance is seeking a Staff Data Engineer to help build its first end-to-end data platform. You will design and implement real-time data pipelines and AI-ready embeddings from scratch, collaborating with the Sr. Director of Data Platform & Analytics.

In this hands-on role, you will own the stack from CDC events to vector databases, create dbt transformations, and partner across Finance, Marketing, and Ops to deliver reliable, low-latency data products for executive reporting.

Qualifications

• 7+ years of hands-on data engineering experience in streaming systems.
• Proven experience designing near real-time pipelines from scratch (Kafka, Kinesis, Flink, Debezium/CDC).
• Hands-on production experience with vector databases and embeddings (e.g., Pinecone, Weaviate, pgvector, Milvus).
• Advanced proficiency in SQL and Python.
• Working knowledge of cloud warehouse/l lakehouse platforms (Snowflake, BigQuery, Databricks) and dbt.
• Proven experience building end-to-end production data environments in early-stage teams.
• Familiarity with B2B SaaS and recurring revenue data models.
• Working knowledge of BI tools (Looker, Tableau, Power BI).
• Ability to work independently in a lean, fast-moving environment.

Responsibilities

• Design, build, and own near real-time data pipelines as the backbone of the platform's data flow.
• Evaluate, implement, and maintain vector database infrastructure and embedding pipelines for AI-augmented use cases.
• Design and build ELT/ETL pipelines ingesting data from our platform, CRM (HubSpot) and HRIS.
• Architect the underlying warehouse/lakehouse as a system of record for streaming and AI infrastructure.
• Build lightweight transformation layers (dbt) to deliver business-ready datasets.
• Own pipeline reliability and observability across streaming and batch pipelines.
• Enable self-service and AI-powered reporting with stakeholders on recurring reports.
• Implement data governance with proper documentation and access controls.
• Collaborate with Finance, Marketing, Customer Success, and Operations to translate data needs into reliable data products.
• Leverage AI-augmented development tools to accelerate engineering and documentation.

Skills

Streaming data pipelines

Vector databases

SQL

Python

Cloud data warehouse

dbt

End-to-end data platform

BI tooling (Looker/Tableau/Power BI)

B2B SaaS data models

Event streaming (Kafka/Kinesis)

Tools

Kafka

Kinesis

Flink

Debezium/CDC

Snowflake

BigQuery

Databricks

Job description

Harbor Compliance is seeking a Staff Data Engineer to help build its first end-to-end data platform. You will design and implement real-time data pipelines and AI-ready embeddings from scratch, collaborating with the Sr. Director of Data Platform & Analytics.

In this hands-on role, you will own the stack from CDC events to vector databases, create dbt transformations, and partner across Finance, Marketing, and Ops to deliver reliable, low-latency data products for executive reporting.