Data Engineer (SAP)

Lumenalta; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Lumenalta is seeking a senior data engineer to build and optimize SAP data ingestion pipelines on Databricks. You will transform landed data into clean Delta tables using PySpark and SQL, implement incremental and CDC patterns, and ensure data reconciliation between SAP and Databricks.

You will collaborate with client teams and document reusable patterns. The role requires strong communication, deep experience with SAP data, and a remote US-facing environment with a focus on scalable data

Qualifications

• 5+ years of data engineering experience in production pipelines.
• Hands-on experience with SAP data extraction and modeling from ECC or S/4HANA.
• Experience with Databricks, PySpark, SQL, and Delta Lake.
• Familiarity with incremental and CDC ingestion patterns.
• Knowledge of SAP extraction approaches (ODP, BW extractors, SLT, SAP Datasphere).
• Strong written communication and ability to document patterns for reuse.

Responsibilities

• Build ingestion pipelines for SAP data landing in Databricks following client architecture.
• Land SAP data into Delta tables using PySpark and SQL.
• Implement incremental and CDC patterns and validate deltas.
• Reconcile Databricks outputs with SAP data for accuracy.
• Identify performance issues and propose improvements to ingestion architecture.
• Document reusable patterns, configurations, and runbooks.

Skills

5+ years data engineering

Databricks

PySpark

SQL

CDC ingestion

Delta Lake

Delta Live Tables

SAP data modeling

Tools

ODP

BW extractors

SLT

SAP Datasphere

Unity Catalog

Job description

About the Role

We partner with global enterprises to design and build large-scale data platforms that power the products and operations at the center of their business. This engagement supports a major US consumer goods company that is moving its SAP data into Databricks.

The client has defined an ingestion architecture for SAP source data landing in Databricks. The question now is whether that architecture holds up as a repeatable delivery method for the SAP data domains that come next. Your job is to put it to work against real SAP data and find out.

What You’ll Do

• Build ingestion pipelines for SAP source data landing in Databricks, following the client-defined ingestion architecture.
• Work with SAP table structures, relationships, and business objects to land data correctly and completely.
• Implement incremental and change-data-capture patterns for SAP sources, and validate that deltas are applied accurately.
• Transform landed SAP data into clean, usable Delta tables using PySpark, SQL, and Databricks declarative pipelines.
• Reconcile Databricks outputs against SAP to confirm completeness and accuracy.
• Identify gaps, performance issues, and failure points in the ingestion architecture, and propose practical improvements.
• Document reusable patterns, configurations, and runbooks so the approach can be repeated for subsequent SAP data domains.
• Work directly with the client’s engineering and data teams in their tools and environments, adapting to their priorities as they shift.

What We’re Looking For

• 5+ years in data engineering, including production pipelines on Databricks.
• Hands-on experience extracting and modeling SAP data from ECC or S/4HANA, including familiarity with core SAP tables and modules such as finance, sales, materials, or supply chain.
• Experience with SAP extraction approaches, such as ODP or BW extractors, SLT, SAP Datasphere, or third-party replication tools.
• Experience implementing incremental and CDC ingestion patterns.
• Strong PySpark and SQL, and working knowledge of Delta Lake.
• Experience with Databricks declarative pipelines or Delta Live Tables, Auto Loader, and Databricks Workflows.
• Comfortable working in a client-directed, time-and-materials model, where priorities are set by the client and can change week to week.
• Clear written communication, with the discipline to document patterns so others can reuse them.
• Fluent English, both written and spoken.

Nice to Have

• Experience with SAP Business Data Cloud or SAP Databricks.
• Experience with Unity Catalog, including organizing and governing SAP-derived data.
• Prior SAP-to-cloud migration work in consumer goods, manufacturing, or distribution.
• Databricks Data Engineer Professional certification.

Why Lumenalta is an amazing place to work at

• Be 100% dedicated to one project at a time so that you can innovate and grow.
• Be a part of a team of talented and friendly senior-level developers.
• Work on projects that allow you to use leading tech.

For Databricks practitioners specifically, we are a Databricks partner with Champions and MVPs on our team, and that carries three concrete things:

• Delivery Partner Program qualification. Databricks co-delivers its Professional Services engagements through a short list of approved partner organizations, and practitioners have to be individually qualified before they can work on them. The qualification is granted once at the individual level and does not expire, so it stays with you. Partner organizations are the route in, so the qualified pool stays small. That scarcity is most of the value, and we sponsor it.
• Champion path. Databricks Champion is an individual recognition with its own badge, a dedicated Databricks solutions architect as your mentor, access to product teams, confidential roadmap briefings, and an invitation to the internal Tech Summit. Nomination has to come from a partner organization, which puts it out of reach for independents and most contractors. We nominate actively and regularly, and the Champions already on our team mentor candidates through it.
• Certifications covered. We pay the exam fee for any Databricks certification you want to take.

Our Process

A screening call, a technical interview, and a HackerRank assessment.

The technical interview is a conversation rather than a whiteboard exam. We will ask you to walk through an SAP-to-cloud pipeline you built, how you extracted and modeled the SAP data, how you handled deltas and reconciliation, and what you would change. Relevant project experience matters more to us than certifications.

Location

This is a fully remote position. Availability to work overlapping U.S. Central time zone business hours.

Application Deadline

Applications will be accepted until October 4th, 2026. Candidates can expect feedback by October 12th, 2026.