Databricks & Agentic AI Senior Data Engineer

Primus Connect; Greater London; Hybrid; GBP 80,000 - 90,000; Full time

Job summary

Primus Connect is seeking a Senior Data Engineer to advance its Databricks platform with a strong focus on agentic AI. This hands-on role involves building scalable data pipelines, driving AI-driven workflows, and shaping enterprise data engineering at scale.

You'll work in a hybrid London setup, collaborating across EMEA and APAC, owning architecture decisions, and delivering production-grade solutions with a proactive, analytical mindset.

Qualifications

• Strong Databricks expertise including Delta Lake, Declarative Pipelines, DABs and Spark optimisation.
• Advanced SQL and strong Python/PySpark skills.
• Experience building scalable data solutions and troubleshooting production issues.
• Proactive, analytical mindset with interest in AI-driven engineering.

Responsibilities

• Building agentic AI workflows to automate data engineering, code generation, testing and CI/CD.
• Developing scalable data pipelines, data models and ingestion frameworks.
• Working extensively with Databricks, Delta Lake, SQL, Python and PySpark.
• Improving data quality, platform performance and production reliability.
• Delivering solutions across master data management, client data and geospatial processing.

Skills

Databricks

Delta Lake

SQL

Python

PySpark

Spark optimisation

Unity Catalog

Geospatial data

Agentic AI interest

Job description

Primus Connect is seeking a Senior Data Engineer to advance its Databricks platform with a strong focus on agentic AI. This hands-on role involves building scalable data pipelines, driving AI-driven workflows, and shaping enterprise data engineering at scale.

You'll work in a hybrid London setup, collaborating across EMEA and APAC, owning architecture decisions, and delivering production-grade solutions with a proactive, analytical mindset.