Airflow Platform Engineer

Jade Global

Hyderabad

Vor Ort

INR 1.000.000 - 2.500.000

Vollzeit

14 Tage+

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Jade Global in India is seeking an Airflow Platform Engineer with 4–6 years of experience to design, optimize and operate Airflow DAGs and the platform on AKS. You will implement complex DAG patterns, templates, and onboarding to empower teams with self-service data workflows.

You will also support platform operations on AKS, troubleshoot scheduling and resource issues, and contribute to CI/CD pipelines for DAG deployments and container updates, ensuring reliability and scalability for

Qualifikationen

  • Strong Airflow experience (3.x or later) in production environments.
  • Proven DAG development experience using operators and orchestration patterns.
  • Kubernetes AKS knowledge to support Airflow runtime.
  • Expert Python for building and debugging production workflows.
  • Experience with DBT and modern data pipelines.
  • Familiarity with CI/CD tools (Git, Jenkins, Docker).
  • Experience with cloud platforms like Azure or others.

Aufgaben

  • Design, develop, and optimize Airflow DAGs for enterprise use cases.
  • Implement advanced DAG patterns including dynamic DAGs, branching, sensors, retries, and error handling.
  • Provide reusable templates and onboarding to drive self-service adoption.
  • Support Airflow platform on AKS: scheduler, webserver, workers, metadata services.
  • Troubleshoot DAG failures, performance issues, and resource bottlenecks.
  • Assist with CI/CD pipelines for DAG and container deployments; enhance monitoring and reliability.

Kenntnisse

Airflow
Python
Kubernetes
DAG development
CI/CD
DBT
Testing
Cloud platforms

Tools

Git
Jenkins
Docker
AKS
Astronomer Astro Private Cloud

Jobbeschreibung

Airflow Platform Engineer1 Experience - 4 - 6 years
Key Responsibilities
DAG Development & Developer Enablement
  • Design, develop, and optimize Airflow DAGs for enterprise use cases
  • Demonstrate strong experience with operators such as PythonOperator, SQL operators, API/HTTP operators, KubernetesPodOperator etc.
  • Apply advanced DAG patterns including dynamic DAGs, branching, sensors, retries, and error handling
  • Troubleshoot DAG failures, performance issues, and integration challenges
  • Enable orchestration of modern data workflows including DBT pipelines
  • Provide reusable templates, best practices, and onboarding support to drive self-service adoption
Platform Operations & Kubernetes Support
  • Support Airflow platform deployed on AKS, covering scheduler, webserver/API, workers, and metadata services
  • Troubleshoot issues related to task execution, scheduling delays, and resource bottlenecks
  • Diagnose Kubernetes-level issues involving pods, networking, storage, and RBAC
  • Work with Azure services such as Key Vault, Storage, and networking
  • Support CI/CD pipelines for DAG and container-based deployments
  • Improve platform stability, monitoring, and alerting
Required Skills
  • Strong experience with Apache Airflow (3.x or later) in production environments
  • Proven experience in DAG development using operators and orchestration patterns
  • Working knowledge of Kubernetes (AKS preferred) to support Airflow runtime
  • Expert-level Python knowledge for building, debugging, and maintaining production-grade workflows and supporting libraries
  • Expertise in writing unit, integration, and DAG validation tests for Airflow workflows, including mocking operators, validating dependencies, and ensuring reliability in CI/CD pipelines
  • Experience with DBT and modern data pipelines
  • Experience with CI/CD tools (Git, Jenkins, Docker)
  • Familiarity with Azure or other cloud platforms
Preferred Skills
  • Experience with Astronomer / Astro Private Cloud
  • Exposure to data platforms such as Snowflake, and APIs
  • Experience with monitoring tools such as Splunk or Prometheus
  • Knowledge of secrets management using Azure Key Vault
Summary

This role is critical to ensuring reliable platform operations and scalable adoption of the Airflow-based orchestration platform. The ideal candidate combines strong Airflow expertise, Kubernetes knowledge, and hands-on development experience to support both platform stability and developer productivity.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Airflow Platform Engineer
Airflow Platform Engineer

Jade Global • Pune District

Vor Ort
INR 1.000.000 - 1.400.000
Airflow Developer
Airflow Developer

Genpact Ltd. • Hyderabad, Bengaluru, Delhi

Vor Ort
INR 900.000 - 1.500.000
Junior Platform Data Engineer
Junior Platform Data Engineer

Gemini Solutions • Gurugram District

Vor Ort
INR 1.200.000 - 1.800.000
Senior Data Engineer (Airflow Specialist)
Senior Data Engineer (Airflow Specialist)

Hvantage Technologies Inc USA • Indien

Hybrid
INR 1.500.000 - 2.000.000
AWS & Airflow Developer
AWS & Airflow Developer

LPL Financials • Hyderabad

Hybrid
INR 1.200.000 - 1.800.000
Airflow Python Developer
Airflow Python Developer

Sutherland • Hyderabad

Vor Ort
INR 1.000.000 - 1.600.000
Senior Developer, Data Engineer
Senior Developer, Data Engineer

Intercontinental Exchange Holdings, Inc. • Hyderabad

Vor Ort
INR 2.500.000 - 4.500.000
Senior Developer, Data Engineer
Senior Developer, Data Engineer

ICE • Hyderabad

Vor Ort
INR 1.800.000 - 3.000.000
Airflow developer
Airflow developer

Intellics Global Services • Mumbai, Pune District, Bengaluru

Hybrid
INR 1.500.000 - 2.500.000
Airflow Python Developer
Airflow Python Developer

Sutherland Company • Hyderabad

Vor Ort
INR 1.200.000 - 1.800.000