Remote Senior Databricks Engineer - Ingestion Lead

Lumenalta (formerly Clevertech); Northern (KY); Hybrid; USD 90,000 - 130,000; Full time

Benefits offered by this job

Health insurance

Remote work environment

Professional development fund

Paid time off 15 days

Job summary

Lumenalta is seeking a Senior Databricks Engineer to own the ingestion layer for a US defense client on a fully remote basis. You will run source discovery across PLM, ERP, and MBSE sources, design ingestion patterns, and implement Lakeflow Connect, JDBC-based ingestion, and S3 Auto Loader patterns with strong data quality and governance.

This is a hands-on senior role with end-to-end ownership. Applicants should bring 7+ years of Databricks data engineering, fluent Python/PySpark/SQL,

Qualifications

• 7+ years of production data engineering on Databricks.
• Fluent in Python, PySpark, and SQL.
• Experience with Lakeflow Connect or similar CDC ingestion.
• Experience building JDBC-based ingestion against Oracle E-Business Suite.

Responsibilities

• Run source discovery across each system in scope, documenting data structures, change patterns, extraction constraints, data quality, and current-state integration topology.
• Assess extraction options per source, including change data capture feasibility from PLM, JDBC access to Oracle E-Business Suite under its row-level security model, SQL Server change patterns, and file-based ingestion from S3.
• Design the ingestion pattern for each source (Lakeflow Connect, custom JDBC, Auto Loader, or federation) with tradeoffs written down and defensible.
• Model bill-of-materials data across PLM, ERP, and MBSE sources, keyed on part number plus revision, including reconciliation rules.

Skills

Databricks

Python

PySpark

SQL

Delta Lake

Lakeflow Connect

JDBC ingestion

Oracle E-Business Suite

SQL Server

Git CI/CD

PyTest

AWS S3

IAM roles

US citizenship

Tools

Terraform

Databricks Asset Bundles

Job description

Lumenalta is seeking a Senior Databricks Engineer to own the ingestion layer for a US defense client on a fully remote basis. You will run source discovery across PLM, ERP, and MBSE sources, design ingestion patterns, and implement Lakeflow Connect, JDBC-based ingestion, and S3 Auto Loader patterns with strong data quality and governance.

This is a hands-on senior role with end-to-end ownership. Applicants should bring 7+ years of Databricks data engineering, fluent Python/PySpark/SQL,