Airflow Administration

Tekskills; Pune District; On-site; INR 1,400,000 - 2,400,000; Full time

Job summary

Tekskills is seeking a skilled engineering professional to manage Airflow-based data workflows in a Kubernetes environment. The role involves implementing Airflow administration tasks, building and debugging complex DAGs, and integrating Airflow APIs with other systems.

The candidate should have strong Python experience and familiarity with Redis caching/streams and event-driven architectures to support large volume batch processing.

Qualifications

• Experience with Python programming in data workflows.
• Hands-on Airflow admin including CI/CD for Airflow versions, executors and PODs.
• Proficient in creating, debugging and deploying complex Airflow DAGs.

Responsibilities

• Manage Airflow components as containers on Kubernetes.
• Work with Redis caching/streams and event-driven patterns.
• Optimize Airflow for large-volume batch processing.

Skills

Python

Airflow Administration

Airflow DAGs

Airflow on Kubernetes

Redis

Event-driven architecture

Tools

Airflow

Kubernetes

Redis

Job description

• Strong knowledge on Python
• Airflow Administration - Includes installation/upgrade of Airflow versions (CI/CD), Exposure to executors & POD operators, Optimization of Airflow components to support complex large volume batches
• Airflow BAU - Create / Debug / Deploy complex DAGs; Exposure to Airflow APIs and how to integrate them with other systems
• Experience on managing Airflow components as containers on Kubernetes
• Exposure to Redis cacheing & streams and event driven architecture