Data Engineer - Bangalore

DuoBridge; Bengaluru; Hybrid; INR 1,200,000 - 2,500,000; Full time

Job summary

DuoBridge in Bengaluru is seeking an experienced Data Engineer to design, develop, and manage scalable cloud-native data engineering solutions on AWS. The role involves building automated ETL/ELT pipelines, real-time data processing frameworks, and governed data platforms.

The position emphasizes building robust pipelines with AWS, Python, SQL, Databricks, Snowflake, and orchestration tools like Airflow and Kafka, along with Docker/Kubernetes for deployable containers and Terraform/Git/Jenkins

Qualifications

• Experience designing and building cloud-native data solutions on AWS.
• Knowledge of ETL/ELT pipelines, data processing, and data governance.

Responsibilities

• Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks.
• Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration.
• Implement containerisation and deployment using Docker and Kubernetes.
• Manage CI/CD pipelines and infrastructure automation using Terraform, Git, and Jenkins.
• Ensure data quality, governance, monitoring, and performance optimisation.

Skills

AWS

Python

SQL

Databricks

Snowflake

Apache Airflow

Docker

Kubernetes

Spark

Kafka

Terraform

Git

Jenkins

Job description

Job Description

We are looking for an experienced Data Engineer to design, develop, and manage scalable cloud-native data engineering solutions on AWS. The role involves building automated ETL/ELT pipelines, real-time data processing frameworks, and governed data platforms.

Key Responsibilities

• Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks.
• Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration.
• Implement containerisation and deployment using Docker and Kubernetes.
• Manage CI/CD pipelines and infrastructure automation using Terraform, Git, and Jenkins.
• Ensure data quality, governance, monitoring, and performance optimisation.

Required Skills

AWS, Python, SQL, Databricks, Snowflake, Apache Airflow, Docker, Kubernetes, Spark, Kafka, Terraform, Git, and Jenkins.

Skills: terraform,python,sql,aws kinesis,kafka,git,spark,docker,aws,snowflake,jenkins,ckad,databricks,airflow

Note: Candidates must be comfortable working from the Bangalore office 4 days a week.