Senior Technology Architect_ Databricks

Infosys; Greater London; On-site; GBP 110,000 - 165,000; Full time

Job summary

Infosys in London, UK is seeking a Senior Databricks Architect to design, build, and operate scalable data pipelines and data products for a modern data mesh architecture.

The role focuses on integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Strong expertise in Databricks and streaming with Kafka is required.

Qualifications

• Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures.
• Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka.
• Experience integrating legacy data sources such as DB2 (mainframe) and SQL Server.
• Understanding of AWS data services, especially S3.
• Knowledge of data product thinking and data mesh concepts.

Responsibilities

• Build and maintain batch and streaming data pipelines from legacy sources into an AWS-based data lake.
• Implement real-time ingestion using Confluent Kafka and land data in S3.
• Develop data transformations and analytics pipelines in Databricks using medallion architecture (Bronze/Silver/Gold).
• Design and deliver high-quality, reusable data products aligned to data mesh principles.
• Ensure data quality, reliability, security, and governance across pipelines.
• Collaborate with domain and platform teams to enable self-serve analytics.

Skills

Databricks

Kafka

DB2/SQL

AWS S3

Data mesh

Job description

Senior Databricks Architect

Technology – Databricks (Spark, PySpark/SQL) and data lake architectures

Location – London, UK

Compensation – Competitive (including bonus)

Job Description

We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role focuses on integrating legacy systems with cloud‑native platforms to support analytics, reporting, and domain‑owned data products.

Primary Skills

• Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures.
• Hands‑on experience with streaming technologies, especially Kafka / Confluent Kafka.
• Experience integrating legacy data sources such as DB2 (mainframe) and SQL Server.
• Working knowledge of AWS data services, especially S3.
• Understanding of data product thinking, data mesh concepts, and modern data engineering best practices.

Key Responsibilities

• Build and maintain batch and streaming data pipelines from legacy mainframe DB2 and on‑prem SQL Server systems into an AWS‑based data lake.
• Implement real‑time ingestion using Confluent Kafka and land data in Amazon S3.
• Develop data transformations and analytics pipelines in Databricks using the medallion architecture (Bronze, Silver, Gold).
• Design and deliver high‑quality, reusable data products aligned to data mesh principles.
• Ensure data quality, reliability, security, and governance across pipelines.
• Collaborate with domain teams, platform teams, and stakeholders to enable self‑serve analytics and consumption.

Good to have

• Experience in banking or financial services data platforms.
• Familiarity with data governance, metadata management, and regulatory reporting.

Personal

• High analytical skills
• A high degree of initiative and flexibility
• High customer orientation
• High quality awareness
• Excellent verbal and written communication skills

Infosys is a global leader in next‑generation digital services and consulting. Infosys is proud to be an equal opportunity employer.