Big Data Engineer(AWS, Python,Airflow)

Zorba AI; Hyderabad; On-site; INR 800,000 - 1,500,000; Full time

Job summary

A technology services company is seeking two AWS/Python/Airflow/Big Data Engineers in Hyderabad with 3–5 years of experience. The role involves designing and maintaining ETL processes, managing workflows on AWS, and optimizing data pipelines. Candidates must have strong skills in Python and knowledge of Big Data technologies like Spark and Hadoop. The company offers a full-time position geared towards engineering and information technology sectors.

Qualifications

• 3–5 years of hands-on experience in data engineering.
• Strong expertise in Python, AWS services, and Big Data technologies.
• Excellent problem-solving and communication skills.

Responsibilities

• Design, develop, and maintain ETL and data pipelines using Apache Airflow.
• Implement scalable solutions on AWS Cloud.
• Collaborate with cross-functional teams to deliver solutions.

Skills

AWS Cloud Services

Python Programming

Apache Airflow

Big Data Technologies

Data Modeling & ETL

Databases & SQL

Problem Solving

Communication

Collaboration

Job description

Job Description – AWS / Python / Airflow / Big Data Engineer

Role: AWS / Python / Airflow / Big Data Engineer

Experience: 3–5 Years

Job Overview: We are seeking two skilled AWS/Python/Airflow/Big Data Engineers with 3–5 years of hands‑on experience to join our data engineering team. The ideal candidates will be responsible for designing, developing, and maintaining scalable data pipelines and workflows in a cloud‑based AWS environment. This role requires strong expertise in Python, Apache Airflow, AWS services, and Big Data technologies.

Key Responsibilities

• Design, develop, and maintain ETL and data pipelines using Apache Airflow.
• Build and manage workflow orchestration (DAGs) for reliable data processing.
• Implement scalable and cost‑efficient solutions on AWS Cloud, leveraging S3, EC2, Lambda, EMR, Glue, Redshift.
• Write efficient, reusable, and maintainable Python code for data processing and automation.
• Work with Big Data frameworks such as Apache Spark and Hadoop for large‑scale data processing.
• Optimize data workflows for performance, scalability, and cost efficiency.
• Collaborate with cross‑functional teams (data analysts, product, DevOps) to understand data requirements and deliver solutions.
• Ensure data quality, reliability, security, and monitoring across all pipelines and workflows.
• Troubleshoot and resolve data pipeline failures and performance bottlenecks.

Required Qualifications

• AWS Cloud Services: Hands‑on experience with core AWS services used in data engineering.
• Python Programming: Strong coding skills with experience using libraries such as Pandas, PySpark, and automation scripts.
• Apache Airflow: Proven experience building and managing Airflow DAGs and workflows.
• Big Data Technologies: Familiarity with Spark, Hadoop, or similar distributed data processing frameworks.
• Data Modeling & ETL: Solid understanding of ETL processes, data transformations, and pipeline design.
• Databases & SQL: Working knowledge of SQL and relational databases.
• Soft Skills: Strong problem‑solving and debugging abilities; excellent communication and collaboration skills; ability to work independently and in a fast‑paced team environment.
• Preferred Qualifications: Experience with CI/CD pipelines and DevOps practices; familiarity with containerization technologies such as Docker and Kubernetes; exposure to cloud monitoring, logging, and performance optimization tools.

Location

Hyderabad, Telangana, India

Seniority Level

Associate

Employment Type

Full‑time

Job Function

Engineering and Information Technology

Industries

IT Services and IT Consulting