Senior Data Engineer – GenAI & Automation (Databricks)

Lemon.io; United States; Remoto; USD 120.000 - 180.000; Tempo pieno

Descrizione del lavoro

Lemon.io seeks a Senior Data Engineer to own GenAI-enabled data pipelines within a mature finance data platform. Remote from Europe or LATAM, the role is full‑time for 3 months with strong extension potential, balancing hands-on work (80%) and strategy (20%).

The position requires 6+ years in data engineering, deep Databricks experience, SQL/dbt, Python/PySpark, and production use of GenAI tools to automate monitoring and development.

Competenze

• 6+ years of hands-on data engineering at senior or principal level.
• Strong Databricks experience at large scale.
• Strong SQL and dbt; Python / PySpark.
• Real, production use of GenAI and agentic tools — agents for monitoring, automation or dev workflows (not just experiments).
• Production experience with pipeline monitoring, observability and data quality.
• A proven before/after track record improving an existing, mature platform.
• Experience with large, high‑precision datasets.
• Finance data experience (month‑end close, P&L, sales and accounting data).
• Based in Europe or LATAM, able to work full CET hours; fluent English.

Mansioni

• Review pipelines, monitoring and releases with an as‑is → to‑be plan.
• Build agent‑based monitoring that detects failures and escalates automatically.
• Cut manual support effort from ~13–14 hours/day to ~2–3 hours/day.
• Speed up delivery on a complex data model with safer changes and testing.
• Bring agentic and AI‑assisted development into daily work and coach by example.
• Improve data quality visibility so issues are caught before business users see them.

Conoscenze

Databricks

SQL

Python/PySpark

GenAI tools

Data quality

Finance data

Strumenti

dbt

GitHub

CI/CD pipelines

Descrizione del lavoro

Senior Data Engineer – GenAI & Automation (Databricks, Finance Data) – full-time, 3 months + strong potential to extend

Remote: Europe or LATAM

A role for a Senior, hands-on Data Engineer who uses agentic AI to transform how a mature finance data team works. The client is one of the world's leading home appliance manufacturers.

The context

The Global Finance data platform serves thousands of users, with reports followed at CFO and CEO level. Pipelines run hourly around the clock across APAC, India, Europe and Oceania, and core models process around 3 billion records where every figure must match exactly. The team is strong but absorbed by operations, especially at month-end close. This is not a greenfield role — your job is to automate that load away and set a new standard by example. Roughly 80% hands‑on, 20% strategy.

Your first 3 months

• Review pipelines, monitoring and releases, and present an as‑is → to‑be plan with about 5 concrete deliverables
• Build agent‑based monitoring that detects failures and delays, applies rule‑based fixes and escalates automatically
• Cut manual support effort (today ~13–14 hours/day, target ~2–3 hours/day)
• Speed up delivery on a complex, interdependent data model with safer changes, better testing and impact analysis
• Bring agentic and AI‑assisted development (e.g. Databricks agents, GitHub Copilot) into the team's daily work and coach by example
• Improve data quality visibility so issues are caught before business users see them

Beyond 3 months

Grow into the technical go‑to person for the Finance innovation team, designing multi‑agent workflows in Databricks to automate finance business processes.

Must-haves

• 6+ years of hands‑on data engineering at senior or principal level
• Strong Databricks experience at large scale
• Strong SQL and dbt; Python / PySpark
• Real, production use of GenAI and agentic tools — agents for monitoring, automation or dev workflows (not just experiments)
• Production experience with pipeline monitoring, observability and data quality
• A proven before/after track record improving an existing, mature platform
• Experience with large, high‑precision datasets
• Finance data experience (month‑end close, P&L, sales and accounting data)
• Based in Europe or LATAM, able to work full CET hours; fluent English

Nice‑to‑haves

• SAP as a data source (FI/CO, sales data) — strongly preferred
• Databricks AI features (Mosaic AI / Agent Bricks, Unity Catalog)
• Medallion architecture / Data Mesh, CI/CD and GitHub
• Power BI / semantic models; large enterprise experience

Hiring process

• Call with the Head of Finance Data & Analytics
• Technical call with a senior data engineer from the team

The selected candidate goes through an insider list and legal onboarding, as the role involves confidential financial data.