Data Engineer Airflow SQL Snowflake Python

Zensar Technologies; Pune District; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Zensar Technologies is seeking a hands-on Data Engineer in Pune to design, build and run production-grade data pipelines using Apache Airflow. You will own complex DAGs end-to-end, from ingestion through to monitoring, with emphasis on reliability and performance.

The role requires strong production Python, SQL with advanced analytics, and experience in data modeling. Collaboration with data architects, analysts and business stakeholders is essential to elevate pipeline quality and velocity.

Qualifications

• Hands-on with Airflow to design, build and operate pipelines
• Strong production-grade Python coding skills
• Deep SQL skills including windows/analytic functions and query optimization

Responsibilities

• Design, develop and maintain complex Airflow DAGs with multi-stage dependencies
• Ensure idempotent, restartable and backfill-safe pipelines
• Implement robust error handling, retries, SLAs and monitoring
• Develop reusable components: operators, hooks, sensors and DAG factory patterns
• Manage Airflow configurations, connections and scheduling controls
• Write and optimize complex SQL for transformations and data quality checks
• Collaborate with data architects, analysts and stakeholders
• Contribute to CI/CD for DAG deployment and runbooks

Skills

Airflow

Python

SQL

Pandas

PySpark

REST APIs

CI/CD

Git

Data modeling

Education

BE/BTech/MCA/MSc in CS/IT or related

Tools

Airflow

Parquet

Job description

Job Description:

We are looking for a hands-on Data Engineer who can independently design, build and operate production-grade data pipelines using Apache Airflow. This is a delivery-facing engineering role: you will own complex, multi-dependency DAGs end to end from ingestion and transformation logic through to scheduling, monitoring, failure recovery and performance tuning.

The ideal candidate writes production-quality Python, is genuinely strong in SQL (including analytical functions, complex joins and query optimization), and has debugged enough real pipeline failures to have opinions about idempotency, backfills and retry strategy. You will work closely with data architects, analysts and business stakeholders, and will be expected to raise the engineering standard of the pipelines you touch.

Responsibilities

Orchestration & Pipeline Engineering

• Design, develop and maintain complex Airflow DAGs involving multi-stage dependencies, dynamic task generation, branching, sensors, task groups and cross-DAG triggers.
• Build pipelines that are idempotent, restart able and backfill-safe, with correct handling of scheduling intervals, catchup behavior, data intervals and time zones.
• Implement robust error handling and recovery -retries with backoff, SLAs, alerting, callbacks, dead-letter handling and clean partial-failure semantics.
• Develop reusable components: custom operators, hooks, sensors and DAG factory patterns that reduce duplication across the codebase.
• Manage Airflow configuration properly - connections, variables, pools, priority weights, queues and concurrency/parallelism controls.

SQL Development

• Write and optimize complex SQL - multi-table joins, CTEs, window/analytical functions, incremental and merge/upsert logic, slowly changing dimensions.
• Tune query performance: read execution plans, correct inefficient joins and predicates, and apply appropriate partitioning, clustering and indexing strategies.
• Implement data quality and reconciliation checks - row counts, null/duplicate checks, referential integrity and business rule validations as first-class pipeline steps.
• Contribute to dimensional and analytical data model design (star/snowflake schemas, fact and dimension design, grain definition).

Python Development

• Write clean, modular, testable Python following sound engineering practice - packaging, configuration management, logging, exception handling and type hints.
• Build ingestion and transformation utilities: REST/SOAP API integrations, pagination and rate-limit handling, file parsing (CSV, JSON, Parquet, XML) and data frame processing (pandas / PySpark).
• Write unit and integration tests for pipeline logic, including DAG integrity tests, and participate in code reviews.

Operations, Quality & Collaboration

• Monitor and support production pipelines - triage failures, perform root cause analysis, drive permanent fixes, and reduce recurring incidents.
• Work within CI/CD practice: Git-based workflows, branching strategy, automated deployment of DAGs, and environment promotion (Dev → QA → Prod).
• Translate requirements from analysts and business stakeholders into technical designs, and document DAG logic, data lineage and operational runbooks.

Qualifications

• B.E. / B.Tech / MCA / M.Sc. in Computer Science, Information Technology or a related discipline.
• Equivalent practical experience will be considered for candidates with a demonstrable engineering track record.

At Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: One Zensar, Nurturing, Empowering, and Client Focus.

Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. Explore Life at Zensar and join us to Grow. Own. Achieve. Learn. to be the best version of yourself.
We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to race, creed, color, ancestry, religion, sex, national origin, citizenship, age, sexual orientation, gender identity, disability, marital status, family medical leave status, or protected veteran status.