Big Data Engineer_105827

MyCareernet

Ernakulam

On-site

INR 1,800,000 - 3,600,000

Full time

19 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

MyCareernet in India seeks a senior data engineer to design and develop data pipelines for large-scale batch and distributed processing.

You will use Apache Spark, Databricks and Python to build scalable data solutions on AWS or Azure, integrating data from multiple sources.

You will work to improve the reliability and efficiency of data workflows, troubleshoot production issues and support cloud-first data platforms for analytics.

Qualifications

  • 6-14 years of relevant data engineering experience.
  • Hands-on experience with AWS or Microsoft Azure cloud services.
  • Strong experience building distributed data-processing solutions with Apache Spark and Databricks.
  • Proficiency in Python for data engineering and pipeline development.
  • Experience developing and supporting scalable data pipelines in a cloud environment.

Responsibilities

  • Design and develop data pipelines for large-scale batch and distributed processing.
  • Build and maintain data engineering solutions using Apache Spark, Python, and Databricks.
  • Integrate data from multiple sources and deliver well-structured datasets for downstream use.
  • Develop cloud-based data solutions on AWS or Microsoft Azure.
  • Improve the reliability and efficiency of data processing workflows and troubleshoot production issues.

Skills

Apache Spark
Databricks
Python
AWS
Azure
Data pipelines
Big data

Education

B.Tech, B.E, or M.Tech

Job description

  • Design and develop data pipelines for large-scale batch and distributed data processing.
  • Build and maintain data engineering solutions using Apache Spark, Python, and Databricks.
  • Integrate data from multiple sources and deliver well-structured datasets for downstream use.
  • Develop cloud-based data solutions on AWS or Microsoft Azure.
  • Improve the reliability and efficiency of data processing workflows and troubleshoot production issues.
Skills Required:
  • 6-14 years of relevant data engineering experience.
  • Hands-on experience with AWS or Microsoft Azure cloud services.
  • Strong experience building distributed data-processing solutions with Apache Spark and Databricks.
  • Proficiency in Python for data engineering and pipeline development.
  • Experience developing and supporting scalable data pipelines in a cloud environment.
Good to Have:
Education:
  • Bachelor's or master's degree; B.Tech, B.E, and M.Tech are suitable qualifications.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer_105833
Senior Data Engineer_105833

MyCareernet • Ernakulam

On-site
INR 1,400,000 - 2,400,000
Big Data Engineer_105824
Big Data Engineer_105824

MyCareernet • Ernakulam

On-site
INR 1,500,000 - 2,500,000
Data Engineer_105899
Data Engineer_105899

MyCareernet • Ernakulam

On-site
INR 2,200,000 - 4,200,000
Data Software Engineer(AWS)_105540
Data Software Engineer(AWS)_105540

MyCareernet • Jaipur

On-site
INR 1,500,000 - 2,700,000
Senior Data Engineer
Senior Data Engineer

Mags Consulting • Chennai

On-site
INR 1,200,000 - 2,200,000
Data Engineer_Spark/Scala
Data Engineer_Spark/Scala

Zorba AI • Mumbai

On-site
INR 1,000,000 - 1,500,000
Data Engineer
Data Engineer

Expleo Group • Chennai District

On-site
INR 1,600,000 - 2,600,000
Data Engineer_Spark/Scala
Data Engineer_Spark/Scala

Zorba AI • Kolkata District

On-site
INR 1,000,000 - 1,500,000
Data Engineer_Spark/Scala
Data Engineer_Spark/Scala

Zorba AI • Maharashtra

On-site
INR 800,000 - 1,200,000
Data Engineer
Data Engineer

Bdreamz Global Solutions • Chennai

Hybrid
INR 1,200,000 - 2,200,000