Technical Lead

Adaequare; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Adaequare is seeking Data Engineers / ETL Developers with strong experience in Python, Apache Airflow, and Google Cloud Platform to design and maintain scalable data ingestion and ETL/ELT pipelines.

You will build batch and streaming pipelines, ingest from APIs and databases, transform data for BigQuery, and work with cross-functional teams to ensure data quality, monitoring, and production readiness.

Qualifications

• Strong Python and SQL skills.
• Hands-on Apache Airflow / Cloud Composer experience.
• Experience with AWS & GCP – BigQuery, GCS, Cloud Functions.
• Experience integrating REST APIs and relational databases (SQL Server, Oracle, PostgreSQL, MySQL).
• Experience handling large datasets and multiple file formats.
• Knowledge of data quality, validation, monitoring, and error handling.
• Familiarity with Git, CI/CD, and Agile methodologies.

Responsibilities

• Develop scalable ETL/ELT pipelines using Python and Apache Airflow.
• Ingest data from APIs, relational databases, cloud storage, and external files.
• Process formats such as JSON, CSV, TXT, Parquet, XML, SHP, and GeoJSON.
• Build batch and event-driven pipelines on GCP.
• Load, transform, and optimize data in BigQuery.
• Implement data validation, reconciliation, monitoring, logging, and alerting.
• Develop reusable and configurable ingestion frameworks.
• Collaborate with architects, business teams, and data consumers.
• Participate in code reviews, testing, documentation, and production support.

Skills

Python programming

SQL skills

Airflow / Cloud Composer

AWS & GCP (BigQuery, GCS)

REST APIs & RDBMS

Large datasets & multiple formats

Data quality & monitoring

Git CI/CD Agile

Job description

Data Engineer / ETL Developer Python, Airflow, GCP

Experience: 4–6 Years & 6+ Years
Location: Hyderabad
Open Positions: 7

Job Overview

We are looking for Data Engineers / ETL Developers with strong experience in Python, Apache Airflow, and GCP to design and maintain scalable data ingestion and ETL/ELT pipelines.

Key Responsibilities

• Develop scalable ETL/ELT pipelines using Python and Apache Airflow.
• Ingest data from APIs, relational databases, cloud storage, and external files.
• Process formats such as JSON, CSV, TXT, Parquet, XML, SHP, and GeoJSON.
• Build batch and event-driven pipelines on GCP.
• Load, transform, and optimize data in BigQuery.
• Implement data validation, reconciliation, monitoring, logging, and alerting.
• Develop reusable and configurable ingestion frameworks.
• Collaborate with architects, business teams, and data consumers.
• Participate in code reviews, testing, documentation, and production support.

Required Skills

• Strong Python programming and SQL skills.
• Hands-on experience with Apache Airflow / Cloud Composer.
• Experience with AWS & GCP – BigQuery, GCS, Cloud Functions.
• Experience integrating REST APIs and relational databases such as SQL Server, Oracle, PostgreSQL, or MySQL.
• Experience handling large datasets and multiple file formats.
• Knowledge of data quality, validation, monitoring, and error handling.
• Familiarity with Git, CI/CD, and Agile methodologies.

Ideal Candidate

A hands-on Data Engineer who can independently build reliable data pipelines, troubleshoot production issues, and work effectively with cross-functional teams.