Solution Architect

HCL Technologies Limited; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

HCL Technologies Limited is seeking a skilled data engineer to design, develop and operate scalable ETL/ELT pipelines across structured and unstructured data in a GMP-like cloud environment. You will build batch and real-time processing solutions using BigQuery, Dataflow, Cloud Storage, Pub/Sub and related GCP services, while delivering robust data models and cost-aware processing patterns.

You will implement reusable data ingestion and transformation frameworks, ensure data quality and

Qualifications

• Degree in Computer Science, Software Engineering or a related discipline, or equivalent practical experience.
• Experience delivering software solutions in agile teams.
• Knowledge of modern development frameworks, tools and engineering practices.

Responsibilities

• Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
• Build batch and real-time processing solutions using BigQuery, Dataflow, Cloud Storage, Pub/Sub, and related GCP services.
• Develop data-processing components and integrations using Java or Python, SQL, REST APIs, Kafka, and containerized services on GKE.
• Implement reusable ingestion and transformation frameworks supporting full, incremental, and event-driven loads.
• Design data models and optimize BigQuery tables through partitioning, clustering, efficient SQL, and cost-aware processing patterns.
• Produce high-level and low-level technical designs, document assumptions, and contribute to architecture reviews.
• Implement data-quality checks, schema validation, reconciliation, audit logging, lineage, monitoring, alerting, and error-handling mechanisms.
• Troubleshoot complex production issues, perform root-cause analysis, and deliver sustainable corrective actions.
• Apply security, privacy, access-control, reliability, and operational-support standards across data solutions.
• Use source control, automated testing, CI/CD, and infrastructure-as-code practices to promote repeatable deployments.
• Participate in code reviews, improve engineering standards, and mentor less-experienced engineers.
• Collaborate with architects, analysts, application teams, platform teams, and business stakeholders to deliver reliable data products.

Skills

BigQuery

Dataflow

Pub/Sub

Cloud Storage

SQL

Java

Python

REST APIs

Kafka

Docker

Kubernetes

GKE

JSON

Parquet

Git

CI/CD

Observability

Release management

Education

Computer Science degree

Software Engineering degree

Equivalent practical experience

Tools

Cloud Composer

Apache Airflow

Dataproc

Spark

Dataform

dbt

Terraform

Job description

• Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
• Build batch and real-time processing solutions using BigQuery, Dataflow, Cloud Storage, Pub/Sub, and related GCP services.
• Develop data-processing components and integrations using Java or Python, SQL, REST APIs, Kafka, and containerized services on GKE.
• Implement reusable ingestion and transformation frameworks supporting full, incremental, and event-driven loads.
• Design data models and optimize BigQuery tables through partitioning, clustering, efficient SQL, and cost-aware processing patterns.
• Produce high-level and low-level technical designs, document assumptions, and contribute to architecture reviews.
• Implement data-quality checks, schema validation, reconciliation, audit logging, lineage, monitoring, alerting, and error-handling mechanisms.
• Troubleshoot complex production issues, perform root-cause analysis, and deliver sustainable corrective actions.
• Apply security, privacy, access-control, reliability, and operational-support standards across data solutions.
• Use source control, automated testing, CI/CD, and infrastructure-as-code practices to promote repeatable deployments.
• Participate in code reviews, improve engineering standards, and mentor less-experienced engineers.
• Collaborate with architects, analysts, application teams, platform teams, and business stakeholders to deliver reliable data products.

Required Technical Skills

• Strong hands-on experience with GCP data services, particularly BigQuery, Dataflow, Cloud Storage, and Pub/Sub.
• Proficiency in SQL, including complex transformations, query tuning, data validation, and analytical processing.
• Proficiency in Java or Python for data pipelines, automation, API integration, and production support.
• Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing.
• Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE.
• Experience with data formats such as JSON, CSV, Avro, and Parquet.
• Working knowledge of Git, automated testing, CI/CD pipelines, observability, and release management.
• Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency.

Preferred Skills

• Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks.
• Exposure to Terraform or another infrastructure-as-code tool.
• Knowledge of Dataplex, data governance, metadata management, lineage, and access-control practices.
• Experience modernizing legacy or on-premises data workloads to GCP.
• Google Cloud Professional Data Engineer certification or an equivalent cloud data certification.

Typical Qualifications
Degree in Computer Science, Software Engineering or a related discipline, or equivalent practical experience.
Experience delivering software solutions in agile teams.
Knowledge of modern development frameworks, tools and engineering practices.

Key Responsibilities

Key Responsibilities

• Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
• Build batch and real-time processing solutions using BigQuery, Dataflow, Cloud Storage, Pub/Sub, and related GCP services.
• Develop data-processing components and integrations using Java or Python, SQL, REST APIs, Kafka, and containerized services on GKE.
• Implement reusable ingestion and transformation frameworks supporting full, incremental, and event-driven loads.
• Design data models and optimize BigQuery tables through partitioning, clustering, efficient SQL, and cost-aware processing patterns.
• Produce high-level and low-level technical designs, document assumptions, and contribute to architecture reviews.
• Implement data-quality checks, schema validation, reconciliation, audit logging, lineage, monitoring, alerting, and error-handling mechanisms.
• Troubleshoot complex production issues, perform root-cause analysis, and deliver sustainable corrective actions.
• Apply security, privacy, access-control, reliability, and operational-support standards across data solutions.
• Use source control, automated testing, CI/CD, and infrastructure-as-code practices to promote repeatable deployments.
• Participate in code reviews, improve engineering standards, and mentor less-experienced engineers.
• Collaborate with architects, analysts, application teams, platform teams, and business stakeholders to deliver reliable data products.

Required Technical Skills

• Strong hands-on experience with GCP data services, particularly BigQuery, Dataflow, Cloud Storage, and Pub/Sub.
• Proficiency in SQL, including complex transformations, query tuning, data validation, and analytical processing.
• Proficiency in Java or Python for data pipelines, automation, API integration, and production support.
• Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing.
• Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE.
• Experience with data formats such as JSON, CSV, Avro, and Parquet.
• Working knowledge of Git, automated testing, CI/CD pipelines, observability, and release management.
• Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency.

Preferred Skills

• Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks.
• Exposure to Terraform or another infrastructure-as-code tool.
• Knowledge of Dataplex, data governance, metadata management, lineage, and access-control practices.
• Experience modernizing legacy or on-premises data workloads to GCP.
• Google Cloud Professional Data Engineer certification or an equivalent cloud data certification.

Typical Qualifications
Degree in Computer Science, Software Engineering or a related discipline, or equivalent practical experience.
Experience delivering software solutions in agile teams.
Knowledge of modern development frameworks, tools and engineering practices.

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 223,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending June 2026totaled $14.8billion.