Remote Senior Data Engineer - Lakehouse & Pipelines

Geomatik; New York (NY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Octave is seeking a Data Engineer to own the reliable flow of data from Hexagon sources into the Octave platform's Fabric lakehouse. You will build connectors, pipelines, and staging infra to ensure high-quality data for ontological transformation and knowledge graph construction.

You will implement CDC, near-real-time mirroring, and middleware layers (Databridge Pro), while ensuring strong data quality, lineage, and observability across ingestion and staging.

Qualifications

• 7+ years in data engineering, with hands-on production experience building and operating ETL/ELT pipelines at scale.
• Strong proficiency with Microsoft Fabric or Azure Synapse Analytics — Delta Lake, lakehouse architecture, and Spark-based transformation.
• Experience with database mirroring or CDC for SQL Server (e.g. Fabric Database Mirroring, Debezium, SQL Server replication) and Oracle.
• Hands-on experience with REST API integration and OData/OpenAPI-based data extraction.
• Solid understanding of medallion/lakehouse architecture patterns (Bronze/Silver/Gold or equivalent staging discipline).
• Experience with pipeline orchestration tools (Azure Data Factory, Fabric Data Workflows, Airflow, or comparable).
• Proficiency in Python and/or PySpark for transformation logic and notebook-based ETL.
• Strong data quality mindset — schema validation, null handling, deduplication, and monitoring at ingestion.
• Familiarity with enterprise authentication patterns (OAuth2 client credentials, service principals, managed identities).

Responsibilities

• Design, build, and operate data integration pipelines connecting Hexagon source products to the Fabric lakehouse
• Implement and maintain database mirroring configurations for SQL Server and Oracle source systems with near-real-time replication
• Configure and operate Databridge Pro as a middleware integration layer with API adapters, event streams, and scheduled extracts
• Enforce the staging discipline: land data in its original form in the lakehouse Bronze layer before any transformation
• Build and maintain the change-data-capture (CDC) infrastructure for temporal queries and Delivery Genome
• Own pipeline observability: monitoring, alerting, lineage tracking, and data quality checks at ingestion
• Collaborate with the Graph Engineer on schema mapping to ontological transformation requirements
• Work with Governance to implement data classification, access controls, and audit-ready lineage
• Contribute to no-code pipeline capability within Platform Studio's data engineering tool

Skills

ETL/ELT pipelines

Microsoft Fabric

Azure Synapse

Python / PySpark

SQL Server / Oracle

REST API integration

CDC / change data capture

Data governance

Tools

Databridge Pro

Airflow

Fabric Data Workflows

Job description

Octave is seeking a Data Engineer to own the reliable flow of data from Hexagon sources into the Octave platform's Fabric lakehouse. You will build connectors, pipelines, and staging infra to ensure high-quality data for ontological transformation and knowledge graph construction.

You will implement CDC, near-real-time mirroring, and middleware layers (Databridge Pro), while ensuring strong data quality, lineage, and observability across ingestion and staging.