Airflow Administration

Tekskills; Hyderabad; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tekskills in Hyderabad seeks a skilled Airflow/Python engineer to manage and optimize Airflow deployments, build and debug complex DAGs, and integrate Airflow with other systems. Experience with containerized Airflow on Kubernetes and Redis streams is highly valued.

The role focuses on installing/upgrading Airflow versions, CI/CD pipelines, and operating at scale for large batch processing, leveraging APIs and event-driven approaches.

Qualifications

• Strong knowledge of Python.
• Airflow administration including installation/upgrade of Airflow versions (CI/CD).
• Experience creating, debugging, and deploying complex DAGs; exposure to Airflow APIs and integration with other systems.
• Experience managing Airflow components as containers on Kubernetes.
• Exposure to Redis caching/streams and event-driven architecture.

Responsibilities

• Airflow administration including installation/upgrade and CI/CD.
• Optimize Airflow components to support large batch volumes.
• Create, debug and deploy complex DAGs; work with Airflow APIs.
• Manage Airflow components as containers on Kubernetes; exposure to Redis caching/streams and event-driven architecture.

Skills

Python

Airflow Administration

Airflow BAU

Redis caching/streams

Event-driven architecture

Tools

Kubernetes

Job description

• Strong knowledge on Python
• Airflow Administration - Includes installation/upgrade of Airflow versions (CI/CD), Exposure to executors & POD operators, Optimization of Airflow components to support complex large volume batches
• Airflow BAU - Create / Debug / Deploy complex DAGs; Exposure to Airflow APIs and how to integrate them with other systems
• Experience on managing Airflow components as containers on Kubernetes
• Exposure to Redis cacheing & streams and event driven architecture