Airflow Platform Engineer

Jade Global Software Pvt LTD; Hyderabad; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

Jade Global Software Pvt LTD is seeking an Airflow Platform Engineer in Hyderabad. In this role, you will design and optimize Airflow DAGs, support the Airflow platform on AKS, and enhance CI/CD processes.

The ideal candidate has 4-6 years of experience, strong Python skills, and expertise in Kubernetes. You will work to ensure reliable platform operations while enabling scalable data workflows.

Qualifications

• 4-6 years of experience as an Airflow Platform Engineer.
• Strong experience with Airflow in production environments.
• Expert-level Python knowledge for building and maintaining workflows.

Responsibilities

• Design, develop, and optimize Airflow DAGs for enterprise use cases.
• Support Airflow platform deployed on AKS and troubleshoot related issues.
• Implement CI/CD pipelines for DAG deployments.

Skills

Apache Airflow

Python

Kubernetes

CI/CD Tools (Git, Jenkins, Docker)

DBT

Tools

Azure

Monitoring tools (Splunk, Prometheus)

Job description

Airflow Platform Engineer

Experience: 4 - 6 years

Key Responsibilities

DAG Development & Developer Enablement

• Design, develop, and optimize Airflow DAGs for enterprise use cases
• Demonstrate strong experience with operators such as PythonOperator, SQL operators, API/HTTP operators, KubernetesPodOperator, etc.
• Apply advanced DAG patterns including dynamic DAGs, branching, sensors, retries, and error handling
• Troubleshoot DAG failures, performance issues, and integration challenges
• Enable orchestration of modern data workflows including DBT pipelines
• Provide reusable templates, best practices, and onboarding support to drive self‑service adoption

Platform Operations & Kubernetes Support

• Support Airflow platform deployed on AKS, covering scheduler, webserver/API, workers, and metadata services
• Troubleshoot issues related to task execution, scheduling delays, and resource bottlenecks
• Diagnose Kubernetes‑level issues involving pods, networking, storage, and RBAC
• Work with Azure services such as Key Vault, Storage, and networking
• Support CI/CD pipelines for DAG and container‑based deployments
• Improve platform stability, monitoring, and alerting

Required Skills

• Strong experience with Apache Airflow (3.x or later) in production environments
• Proven experience in DAG development using operators and orchestration patterns
• Working knowledge of Kubernetes (AKS preferred) to support Airflow runtime
• Expert‑level Python knowledge for building, debugging, and maintaining production‑grade workflows and supporting libraries
• Expertise in writing unit, integration, and DAG validation tests for Airflow workflows, including mocking operators, validating dependencies, and ensuring reliability in CI/CD pipelines
• Experience with DBT and modern data pipelines
• Experience with CI/CD tools (Git, Jenkins, Docker)
• Familiarity with Azure or other cloud platforms

Preferred Skills

• Experience with Astronomer / Astro Private Cloud
• Exposure to data platforms such as Snowflake, and APIs
• Experience with monitoring tools such as Splunk or Prometheus
• Knowledge of secrets management using Azure Key Vault

Summary

This role is critical to ensuring reliable platform operations and scalable adoption of the Airflow‑based orchestration platform.