Data Engineer

Lloyds Technology Centre; Hyderabad; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Lloyds Technology Centre is seeking a senior data engineer in India with 10–15 years of experience and strong GCP cloud expertise. You will design and implement data pipelines using Python/PySpark, SQL, and modern ETL tools, while leveraging BigQuery, Cloud SQL, and data services.

The role requires proficiency in Airflow/Composer for scheduling and familiarity with Kubernetes, GitHub, and CI/CD practices. Remote options are not specified; on-site Hyderabad is expected.

Qualifications

• 10–15 years of experience in data engineering.
• Data Engineer with mandatory GCP cloud expertise.
• Proficient in Python and PySpark; SQL knowledge.
• Experience building ETL pipelines using Data Flow, DBT/Data Fusion, or Data Proc.
• Knowledge of cloud SQL databases and BigQuery.
• Familiar with scheduling using Airflow/Composer or cloud schedulers.
• Experience with cloud storage: GCS/S3 and data services.
• CI/CD with Jenkins or Harness; IaC with Terraform/Harness.

Skills

Python

PySpark

SQL

GCP Cloud

Data Engineer

Data Flow

Airflow/Composer

Kubernetes

Terraform

Tools

BigQuery

Spanner/BigTable

Cloud SQL

AWS Lambda

DBT/Data Fusion

DataProc

GCS

S3

Kafka/PubSub

GitHub

Jenkins

Harness

Terraform

Kubernetes

Job description

Experience : 10 to 15 years

Job Description : Primary Skills: Data Engineer with GCP cloud Mandatory

Programming - Python/PySpark, SQL

ETL Pipelines (preferred)- Data Flow, DBT/Data Fusion, and/or Data Proc

(alternative) - AWS Lambda, AWS Glue, AWS EMR, Azure Data Factory

Databases - Big Query, Spanner/Big Table, Cloud SQL (preferred) Or RDS Oracle, Cockroach, Snowflake, Dynamo DB

Scheduling/Orchestration: Composer Airflow DAG or any cloud-based scheduler

Other Cloud services: GCS/S3, Kubernetes, KMS, Kafka/ PubSub, GitHub

CICD pipeline using Jenkins/Harness

IAC using Terraform/Harness