DevOps Lead

Harmony Data Integration Technologies Pvt. Ltd.; Sahibzada Ajit Singh Nagar; Hybrid; INR 2,500,000 - 4,500,000; Full time

Job summary

Harmony Data Integration Technologies Pvt. Ltd. is seeking a DevOps Lead to design, build, and operate scalable cloud data platforms on GCP with AWS as a secondary environment.

The role blends infrastructure and data engineering, including Terraform, Kubernetes, CI/CD, and data pipelines using Python. Expect hybrid work and strong emphasis on security, observability, and cost management.

Qualifications

• 8+ years in software, data, or infrastructure engineering.
• 4+ years in a dedicated DevOps/SRE/Platform Engineering role.
• Strong data engineering background with pipelines, warehousing, and orchestration.
• Proficient in GCP and working knowledge of AWS.

Responsibilities

• Design, build, and operate scalable cloud data platforms on GCP with AWS as a secondary.
• Manage Terraform IaC and Kubernetes clusters; helm/ArgoCD for workloads.
• Create secure, scalable CI/CD pipelines with GitHub Actions, Jenkins, GitLab CI, ArgoCD.
• Implement container security, tests, and SonarQube integration in pipelines.
• Embed DevSecOps, IAM, VPCs, encryption, and vulnerability management.
• Develop production-grade ETL/ELT pipelines in Python for batch/streaming.
• Operate Airflow/Cloud Composer for workflow orchestration; data modeling in BigQuery.
• Collaborate with analytics/ML teams; ensure data quality, lineage, and observability.

Skills

GCP

AWS

Terraform

Kubernetes

CI/CD

ArgoCD

GitHub Actions

Jenkins

Docker

Helm

Airflow

BigQuery

Dataflow

Python

SQL

Cloud Composer

Tools

Terraform

Kubernetes

Docker

Helm

ArgoCD

GitHub Actions

Jenkins

GitLab CI

Airflow

Cloud Composer

BigQuery

Snowflake

Redshift

Job description

We are looking for a highly skilled and motivated DevOps Lead to design, build, and operate scalable, secure, and highly available cloud and data platforms, primarily on Google Cloud Platform (GCP) with Amazon Web Services (AWS) as a secondary environment.

This is a hybrid platform role. Alongside core DevOps ownership – Terraform, Kubernetes, CI/CD, cloud security, and observability – you will also contribute substantially to our data engineering function: building and operating production Python data pipelines, orchestrating batch and streaming workflows, and owning the infrastructure that sits underneath our data platform.

The ideal candidate has spent several years building data pipelines as a data engineer and has since moved into a DevOps / SRE / Platform Engineering role, or has run platform engineering for data-intensive products. We are looking for someone who can credibly wear both hats: infrastructure and automation first, data engineering a close second.

• Design, implement, and manage cloud infrastructure across GCP and AWS using Terraform and other Infrastructure-as-Code practices.
• Deploy and maintain Kubernetes clusters and manage application workloads using Helm and/or ArgoCD.
• Build and enhance secure, scalable CI/CD pipelines using GitHub Actions, Jenkins, GitLab CI, and ArgoCD.
• Implement best practices in containerization using Docker, including image hardening and container runtime security.
• Integrate quality gates in CI/CD, including automated testing, code coverage, and SonarQube configuration.
• Ensure secure CI/CD processes, including secrets management, least-privilege access, and hardened build agents.
• Implement and maintain cloud security best practices: IAM roles and policies, network segmentation, encryption, and vulnerability management.
• Collaborate with security teams to embed DevSecOps practices (shift-left security, scanning in pipelines, policy-as-code).
• Contribute to system architecture, monitoring, and observability using modern tools and practices.
• Participate in incident response, post-incident reviews, and root cause analysis.
• Design, build, and maintain production-grade ETL/ELT pipelines in Python for both batch and streaming workloads.
• Build and operate workflow orchestration using Airflow / Cloud Composer, including scheduling, dependency management, retries, and alerting.
• Develop and optimise data models and transformations in BigQuery (and equivalents such as Snowflake or Redshift), with attention to partitioning, clustering, and query cost.
• Own the infrastructure and deployment lifecycle of data services – provisioning, autoscaling, upgrades, and cost management.
• Implement data quality, validation, lineage, and observability checks so pipeline failures and data drift are caught early.
• Apply software engineering discipline to data workloads: version control, code review, automated testing, and CI/CD for pipelines and transformations.
• Work with analytics, ML, and product teams to make data reliably available, secure, and well governed.

Must have skills:

-8+ years of total professional experience in software, data, or infrastructure engineering.

-Minimum 4 years in a dedicated DevOps / SRE / Platform Engineering role.

-Substantial prior hands-on data engineering experience at a senior level (pipeline development, data warehousing, and orchestration).

-Strong proficiency in Google Cloud Platform (GCP) and solid working experience with AWS.

-Hands-on experience with Terraform and Infrastructure as Code (IaC).

-Strong experience with Kubernetes, Docker, and Helm in production.

-Expertise with CI/CD tools: Jenkins, GitLab CI, GitHub Actions, ArgoCD.

-Strong production-grade Python – not scripting only, but maintainable pipeline and tooling code – plus solid Bash.

-Advanced SQL and hands-on experience with a cloud data warehouse (BigQuery preferred; Snowflake or Redshift also considered).

-Experience with workflow orchestration (Airflow / Cloud Composer) and a distributed processing framework (Dataflow/Apache Beam, Spark, or dbt).

-Experience with streaming or event-driven data (Pub/Sub, Kafka, or Kinesis).

-Experience with SonarQube and static code analysis integration.

-Practical cloud security experience (IAM, VPC, firewalls/security groups).

-Experience with image and dependency scanning.

-Knowledge of secure software development practices.

-Excellent written and verbal communication skills.

Good to Have:

-Certifications: GCP Professional DevOps Engineer, GCP Professional Data Engineer, GCP Architect, AWS DevOps Engineer, etc.

-Experience with dbt and modern analytics engineering practices.

-Data modelling experience (dimensional modelling, lakehouse formats such as Iceberg or Delta Lake).

-Data quality and testing frameworks (Great Expectations, Soda, dbt tests).

-Knowledge of compliance frameworks (ISO 27001, CIS Benchmarks, NIST) and data governance / PII handling.

-Experience with monitoring tools (Prometheus, Grafana).

-Experience with logging stacks (ELK, Loki, etc.).

-Exposure to policy-as-code and security tools (OPA, scanners).

-Cloud cost optimisation / FinOps experience, particularly for data workloads.