Azure Data Engineer_105438

MyCareernet; Pune District; In loco; INR 1.500.000 - 3.200.000; Tempo pieno

Vantaggi offerti da questo lavoro

Kubernetes experience

Descrizione del lavoro

MyCareernet is seeking a senior data engineer to design, develop, and maintain batch and near-real-time data pipelines using Spark, Python, and Databricks. You will build scalable data solutions across Azure and Google Cloud platforms, developing transformations, ingestion workflows, and curated datasets for analytics.

The role requires strong hands-on experience with distributed processing, cloud design, and collaboration with architects and analytics teams to deliver production-grade data

Competenze

• 6-14 years of relevant data engineering experience.
• Hands-on experience with Apache Spark and Python for distributed data processing.
• Experience designing cloud-based data engineering solutions on Azure or Google Cloud Platform.
• Ability to develop scalable data pipelines, integrations, transformations, and analytical data stores.

Mansioni

• Design, develop, and maintain batch and near-real-time data pipelines using Apache Spark, Python, and Databricks.
• Build scalable data engineering solutions across Microsoft Azure and Google Cloud Platform environments.
• Develop data transformations, ingestion workflows, and curated datasets for reporting and analytical workloads.
• Troubleshoot data pipeline failures, improve reliability, and resolve data quality and processing issues.
• Collaborate with data architects, analytics teams, and application engineers to deliver production-grade data platforms.

Conoscenze

Data Eng Exp

Apache Spark

Python

Azure/GCP

Distributed Processing

Scalable Pipelines

Formazione

B.Tech

M.Tech

BCA

B.Sc.

Descrizione del lavoro

Roles and Responsibilities:

• Design, develop, and maintain batch and near-real-time data pipelines using Apache Spark, Python, and Databricks.
• Build scalable data engineering solutions across Microsoft Azure and Google Cloud Platform environments.
• Develop data transformations, ingestion workflows, and curated datasets for reporting and analytical workloads.
• Troubleshoot data pipeline failures, improve reliability, and resolve data quality and processing issues.
• Collaborate with data architects, analytics teams, and application engineers to deliver production-grade data platforms.

Skills Required:

• 6-14 years of relevant data engineering experience.
• Strong hands-on experience with Apache Spark and Python for distributed data processing.
• Experience designing cloud-based data engineering solutions on Azure or Google Cloud Platform.
• Ability to develop scalable data pipelines, integrations, transformations, and analytical data stores.

Good to Have:

• Experience with Kubernetes and containerized data workloads.

Education:

B.Tech, M.Tech, BCA, or B.Sc. in Computer Science, Information Technology, Engineering, or a related discipline.