Senior Databricks Architect - Data Mesh & Streaming Tech

Infosys Limited; Greater London; On-site; GBP 110,000 - 170,000; Full time

Job summary

Infosys Limited in London is seeking a Senior Databricks Architect to design, build, and operate scalable data pipelines and data products that enable a modern data mesh. The role focuses on integrating legacy systems with cloud-native platforms for analytics, reporting, and domain-owned data products, with hands-on work in Databricks, Kafka, and AWS S3.

You will implement batch and streaming pipelines, apply the Bronze-Silver-Gold medallion architecture, ensure data quality and governance, and

Qualifications

• Extensive experience with Databricks and data lake architectures.
• Hands-on streaming experience with Kafka.
• Experience integrating legacy sources like DB2 and SQL Server.
• Working knowledge of AWS data services, especially S3.
• Understanding of data mesh concepts and modern data engineering best practices.

Responsibilities

• Build and maintain batch and streaming data pipelines from legacy DB2 and on-prem SQL Server into an AWS data lake.
• Implement real-time ingestion using Confluent Kafka and land data in S3.
• Develop data transformations and analytics pipelines in Databricks using the medallion architecture (Bronze, Silver, Gold).
• Design and deliver high-quality, reusable data products aligned to data mesh principles.
• Ensure data quality, reliability, security, and governance across pipelines.
• Collaborate with domain teams, platform teams, and stakeholders to enable self-serve analytics and consumption.

Skills

Databricks

Spark

PySpark/SQL

Kafka

DB2

SQL Server

AWS S3

Data mesh

Data product thinking

Tools

Confluent Kafka

Amazon S3

Job description

Infosys Limited in London is seeking a Senior Databricks Architect to design, build, and operate scalable data pipelines and data products that enable a modern data mesh. The role focuses on integrating legacy systems with cloud-native platforms for analytics, reporting, and domain-owned data products, with hands-on work in Databricks, Kafka, and AWS S3.

You will implement batch and streaming pipelines, apply the Bronze-Silver-Gold medallion architecture, ensure data quality and governance, and