Senior Data Engineer Databricks

C&W Services (S); Lisboa; On-site; EUR 75,000 - 105,000; Full time

Job summary

Agentic AI in Portugal is seeking a Senior Data Engineer to advance a Databricks-native Lakehouse for EMEA/APAC. You will help shift the team toward AI orchestration, design semantic data models, and own automated ingestion pipelines across diverse sources.

The role emphasizes data quality, production troubleshooting, and governance of AI-enabled data workflows as part of an enterprise-scale data platform strategy.

Qualifications

• Around 6+ years of relevant experience.
• Degree in a quantitatively rigorous field such as computer science, data science, econometrics, mathematics, or physics.
• Driven and curious with an interest in solving not-yet figured-out problems.
• Strong written communication with async updates and documentation.

Responsibilities

• Driving the shift from a code writing team to an AI orchestration team, building agentic workflows that handle data engineering tasks and CI/CD, designing semantic models for downstream consumption, and implementing an AI Gateway for governance.
• Stewarding an AI amplified Databricks based framework, maintaining and improving it incrementally.
• Designing data models that bring sources from anywhere into a single coherent shape.
• Building data solutions for domains like custom MDM, geocoding, client mastering, and profiling.
• Designing and implementing stable, automated ingestion pipelines from diverse sources.
• Owning production troubleshooting and root cause analysis, diagnosing and resolving pipeline failures, performance issues and data quality problems.
• Reporting on data quality and shipping improvements that move the numbers.

Job description

Senior Data Engineer Databricks

Job Description Summary

Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC

Job Description

We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.

Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.

We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a Senior Data Engineer to help build what comes next.

Our Databricks-native Lakehouse is established and mature and already supports the business end to end. The work ahead is more ambitious: applying agentic capabilities at genuine enterprise scale, across a breadth of domains, and shaping how we win in the age of AI.

What you will work on

• Driving the shift from a code writing team to an AI orchestration team. This means building agentic workflows that handle data engineering tasks and CI/CD, designing semantic models for downstream consumption, and implementing an AI Gateway for governance.
• Stewarding our AI amplified Databricks based framework, maintaining it and improving it incrementally as we learn.
• Designing data models that bring sources from anywhere in the world into a single coherent shape.
• Building data solutions for domains like custom MDM, geocoding, client mastering, and profiling. Real estate context is a plus but not required.
• Designing and implementing stable, automated ingestion pipelines from diverse sources.
• Owning production troubleshooting and root cause analysis. When things go wrong, you diagnose and resolve pipeline failures, performance issues, and data quality problems by working through code, logs, and documentation systematically.
• Reporting on data quality and shipping the improvements that move the numbers.

What we are looking for

Core technical skills

• Systems thinking. You see the bigger picture and build for scale rather than for the ticket in front of you.
• Production troubleshooting and root cause analysis. You diagnose and resolve pipeline failures, performance issues, and data quality problems under pressure, working through code, logs, and documentation systematically rather than guessing.
• Data modeling. You design cross functional data products, establish data contracts, and handle complex business rules.
• Advanced SQL, including window functions, query optimization, and MERGE/UPSERT operations.
• Python and PySpark. You write reusable, parameterized functions and work comfortably with JSON, CSV, and Parquet.

Platform knowledge

• Deep experience with Databricks (Declarative Pipelines, DABS, Delta Lake, Spark optimization, job orchestration).
• Familiarity with the Azure ecosystem.
• Working understanding of Unity Catalog.

About you

• Around 6 or more years of relevant experience.
• A degree in a quantitatively rigorous field such as computer science, data science, econometrics, mathematics, or physics.
• Driven and curious. You want to work on things that have not been figured out yet.
• Strong written communication. Async updates and clear documentation are part of the job.

Nice to have

• Experience with agentic data engineering, using coding agents in real delivery and building the harness around them: repo conventions, reusable context, tests and guardrails that make agent output trustworthy.
• Geospatial data processing (H3 indexes, spatial SQL, point in polygon at scale).
• Recursive CTEs and complex SQL patterns.
• Infrastructure familiarity (Azure Portal, resource management, CLI).
• Strong Git workflows and code review habits.
• Real estate domain experience.

What we offer

• Autonomy. We set the high level roadmap. You decide how to get there.
• Dedicated learning time and a team that levels each other up. The frontier moves weekly. We give you the time to keep up with it, and you will be surrounded by people doing the same.
• A rare growth path. You will build leading capability in a field where very few organisations in our sector have yet begun, giving you a skill set you could not have assembled elsewhere in this industry.
• A solid technical foundation. The architecture, standards, and best practices are already in place. You build on top, not from