Remote Senior Databricks Data Engineer, ERP & Logistics

Lumenalta; United States; Remote; USD 90,000 - 132,000; Full time

Benefits offered by this job

Health insurance

401K Contribution

Professional development fund

15 days of paid time off annually

Soft-skill development courses

Job summary

Lumenalta seeks a Senior Databricks Data Engineer to architect and implement large-scale ingestion of ERP and logistics data into a unified Databricks platform. You will connect Oracle E-Business Suite, SLICwave, and other sources to part/revision data from engineering systems, enabling secure, scalable analytics.

You will collaborate with security reviewers, data stewards, and SMEs, and you will orchestrate end-to-end pipelines with Databricks Workflows while ensuring proper lineage and

Qualifications

• 6+ years in data engineering, including production experience building ingestion pipelines on Databricks.
• Working knowledge of Oracle E-Business Suite data structures, including cost, supplier, item, and BOM tables.
• Experience with relational ingestion into Databricks from Oracle, SQL Server, or comparable databases.
• Understanding of Oracle E-Business Suite's multi-org security model and how it affects returns.
• Experience with Auto Loader and file-based ingestion from S3.
• Experience with Databricks declarative pipelines or Delta Live Tables, including schema evolution configuration.
• Experience modeling ERP or logistics data so it joins to engineering or product data.

Responsibilities

• Ingest cost and supplier data from Oracle E-Business Suite into Databricks, using file-based ingestion from S3 extracts or direct read access.
• Ingest SLICwave logistics support analysis data into Databricks, working with its relational backend and file exports.
• Implement incremental and batch refresh patterns for both sources, starting from representative extracts where direct access is still pending.
• Configure schema evolution and baseline data quality checks in Databricks declarative pipelines.
• Model cost, supplier, and logistics data so it links to parts, revisions, and BOM structures from Teamcenter and other engineering sources.
• Support the architect in presenting the Oracle integration approach to enterprise security reviewers, including documentation of access patterns and controls.
• Orchestrate pipelines with Databricks Workflows and ensure lineage is captured for every pipeline.
• Work with logistics and ERP SMEs to understand data structures, business rules, and extraction constraints.

Skills

PySpark

SQL

Data modeling

Delta Lake

ETL design

Tools

Databricks

Oracle E-Business Suite

Delta Lake internals

S3 ingestion

Job description

Lumenalta seeks a Senior Databricks Data Engineer to architect and implement large-scale ingestion of ERP and logistics data into a unified Databricks platform. You will connect Oracle E-Business Suite, SLICwave, and other sources to part/revision data from engineering systems, enabling secure, scalable analytics.

You will collaborate with security reviewers, data stewards, and SMEs, and you will orchestrate end-to-end pipelines with Databricks Workflows while ensuring proper lineage and