Senior Data Engineer- Apache Airflow

Simelabs - An Astek Company

Ernakulam

Presencial

INR 1.500.000 - 3.000.000

Tempo integral

Há 12 dias

Receba uma resposta desta empresa empregadora

Seu currículo e carta de apresentação personalizados exatamente para o que estão contratando.

Passe pelos filtros ATS

Resumo da vaga

Infopark is seeking a Senior Data Engineer with strong Python, PySpark and Apache Airflow skills to design and manage data pipelines in a scalable environment. You will build and maintain Airflow DAGs, orchestrate workflows, perform unit testing, and contribute to data modeling and security compliance.

Experience with Hadoop ecosystem and Snowflake is highly valued. The role involves coordinating with cross-functional teams in India and aligning with data governance standards while leveraging S3

Qualificações

  • Experience in data engineering with Python, PySpark and Apache Airflow.
  • Hands-on experience creating Airflow DAGs, managing workflows and unit testing.
  • Experience with Hadoop, HDFS, Hive on Tez, S3 and AWS CLI operations.

Responsabilidades

  • Develop and maintain Airflow DAGs to orchestrate data pipelines.
  • Design, optimize and secure data workflows with best practices.
  • Perform unit testing and ensure data quality and compliance.

Conhecimentos

Python
PySpark
Apache Airflow
Airflow DAGs
Hadoop
HDFS
Hive on Tez
S3
Snowflake
Data modeling
Facts & Dimensions
Surrogate keys
Secrets management
Data security
Time zone alignment

Ferramentas

Git/GitLab
Livy
Sqoop
SSIS
AWS CLI

Descrição da vaga

Senior Data Engineer- Apache Airflow
  • Experience in data engineering with strong skills in Python, PySpark, and Apache Airflow.
  • Hands-on experience creating Airflow DAGs, managing workflows, and performing unit testing.
  • Experience with Hadoop, HDFS, Hive on Tez, S3, and AWS CLI operations.
  • Strong knowledge of Snowflake, data modeling, facts, dimensions, and surrogate keys.
  • Experience with Git/GitLab, SSIS, Sqoop, Livy, and secrets management tools.
  • Good understanding of data security and compliance standards, with 90% overlap with the US EST time zone.
Obtenha sua revisão de currículo grátis e confidencial.

ou arraste e solte seu arquivo aqui.

Similar jobs

Vagas similares que valem a pena comparar

Data Engineer- Apache Airflow
Data Engineer- Apache Airflow

Astek Group • Ernakulam

Presencial
INR 1.200.000 - 1.800.000
Airflow Python Developer
Airflow Python Developer

Sutherland • Hyderabad

Presencial
INR 1.000.000 - 1.600.000
Senior Data Engineer (Airflow Specialist)
Senior Data Engineer (Airflow Specialist)

Hvantage Technologies Inc USA • Índia

Híbrido
INR 1.500.000 - 2.000.000
Senior Data Engineer
Senior Data Engineer

StartAppss System India Pvt. Ltd. • Indore District

Presencial
INR 1.800.000 - 3.600.000
AWS & Airflow Developer
AWS & Airflow Developer

LPL Financials • Hyderabad

Híbrido
INR 1.200.000 - 1.800.000
Data Engineer
Data Engineer

Amplicomm • Bengaluru

Presencial
INR 1.500.000 - 2.800.000
Data Engineer
Data Engineer

Grid Dynamics • Bengaluru

Híbrido
INR 900.000 - 1.400.000
Airflow Python Developer
Airflow Python Developer

Sutherland Company • Hyderabad

Presencial
INR 1.200.000 - 1.800.000
Senior Data Engineer & Analytics Lead
Senior Data Engineer & Analytics Lead

GenixBit Labs Pvt. Ltd. • Bengaluru

Híbrido
INR 1.200.000 - 2.200.000
Airflow developer
Airflow developer

Intellics Global Services • Mumbai, Pune District, Bengaluru

Híbrido
INR 1.500.000 - 2.500.000