Senior Data Engineer – GenAI & Automation (Databricks)

Lemon.io; España; Remote; EUR 90,000 - 130,000; Full time

Job summary

Lemon.io is seeking a Senior Data Engineer to own and accelerate the Finance data platform within a GenAI and automation context. You will work hands-on on Databricks pipelines, implement agent-based monitoring, and partner with a mature global finance team to raise data quality and reliability.

This is a 3‑month+ full-time contract with strong potential to extend, open to Europe or LATAM, and requires fluency in English and CET hours.

Qualifications

• 6+ years of hands-on data engineering at senior/principal level.
• Strong Databricks experience at large scale.
• Strong SQL and dbt; Python / PySpark.
• Experience with GenAI and agentic tools in production.
• Production experience with pipeline monitoring and data quality.
• Experience with large, high-precision datasets and finance data (month-end close, P&L).
• Based in Europe or LATAM and able to work CET hours; fluent English.

Responsibilities

• Review pipelines, monitoring and releases and present an as‑is → to‑be plan with about 5 concrete deliverables.
• Build agent‑based monitoring that detects failures and delays, applies rule‑based fixes and escalates automatically.
• Cut manual support effort (today ~13–14 hours/day, target ~2–3 hours/day).
• Speed up delivery on a complex, interdependent data model with safer changes, better testing and impact analysis.
• Bring agentic and AI‑assisted development (e.g. Databricks agents, GitHub Copilot) into the team's daily work and coach by example.
• Improve data quality visibility so issues are caught before business users see them.

Skills

Databricks

SQL

dbt

Python

PySpark

GenAI/agentic tools

Data quality/observability

Finance data

Tools

Databricks

dbt

GitHub Copilot

Power BI

Job description

Senior Data Engineer – GenAI & Automation (Databricks, Finance Data) – full-time, 3 months + strong potential to extend

Remote: Europe or LATAM

A role for a Senior, hands-on Data Engineer who uses agentic AI to transform how a mature finance data team works. The client is one of the world's leading home appliance manufacturers.

The context

The Global Finance data platform serves thousands of users, with reports followed at CFO and CEO level. Pipelines run hourly around the clock across APAC, India, Europe and Oceania, and core models process around 3 billion records where every figure must match exactly. The team is strong but absorbed by operations, especially at month-end close. This is not a greenfield role — your job is to automate that load away and set a new standard by example. Roughly 80% hands‑on, 20% strategy.

Your first 3 months

• Review pipelines, monitoring and releases, and present an as‑is → to‑be plan with about 5 concrete deliverables
• Build agent‑based monitoring that detects failures and delays, applies rule‑based fixes and escalates automatically
• Cut manual support effort (today ~13–14 hours/day, target ~2–3 hours/day)
• Speed up delivery on a complex, interdependent data model with safer changes, better testing and impact analysis
• Bring agentic and AI‑assisted development (e.g. Databricks agents, GitHub Copilot) into the team's daily work and coach by example
• Improve data quality visibility so issues are caught before business users see them

Beyond 3 months

Grow into the technical go‑to person for the Finance innovation team, designing multi‑agent workflows in Databricks to automate finance business processes.

Must-haves

• 6+ years of hands‑on data engineering at senior or principal level
• Strong Databricks experience at large scale
• Strong SQL and dbt; Python / PySpark
• Real, production use of GenAI and agentic tools — agents for monitoring, automation or dev workflows (not just experiments)
• Production experience with pipeline monitoring, observability and data quality
• A proven before/after track record improving an existing, mature platform
• Experience with large, high‑precision datasets
• Finance data experience (month‑end close, P&L, sales and accounting data)
• Based in Europe or LATAM, able to work full CET hours; fluent English

Nice‑to‑haves

• SAP as a data source (FI/CO, sales data) — strongly preferred
• Databricks AI features (Mosaic AI / Agent Bricks, Unity Catalog)
• Medallion architecture / Data Mesh, CI/CD and GitHub
• Power BI / semantic models; large enterprise experience

Hiring process

• Call with the Head of Finance Data & Analytics
• Technical call with a senior data engineer from the team

The selected candidate goes through an insider list and legal onboarding, as the role involves confidential financial data.