Azure Databricks Engineer

Turnkeylearning

Hyderabad

In loco

INR 1.000.000 - 2.000.000

Tempo pieno

14 giorni+

Una candidatura completa in un minuto

Curriculum e lettera di presentazione personalizzati, pronti da inviare.

Supera i filtri ATS

Descrizione del lavoro

Turnkeylearning is seeking a Senior Data Engineer based in Hyderabad, India. The ideal candidate will specialize in Azure Databricks and Apache Spark, having proven experience in designing scalable ETL/ELT solutions.

Your responsibilities will include developing data ingestion frameworks and optimizing data pipelines, enabling robust data solutions that drive business insights. Strong collaboration with business stakeholders and data teams is essential.

A solid background in Python, SQL, and data governance is required.

Competenze

  • 3-5 years of experience in Data Engineering.
  • Strong hands-on experience with Databricks.
  • Expertise in Apache Spark (PySpark or Scala Spark).
  • Proficiency in Python, SQL, PostgreSQL.
  • Experience designing and implementing ETL/ELT pipelines.
  • Strong understanding of data warehousing concepts and data modeling.

Mansioni

  • Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
  • Build and optimize ETL/ELT workflows for large volumes of data.
  • Develop data ingestion frameworks from various data sources.
  • Implement data transformation, cleansing, and enrichment processes.
  • Optimize Spark jobs for performance and cost efficiency.

Conoscenze

Databricks
Apache Spark (PySpark or Scala Spark)
Python
SQL
PostgreSQL
Data Engineering
CI/CD
Data Governance

Strumenti

Azure
Terraform
Kafka

Descrizione del lavoro

Senior Data Engineer specializing in Azure Databricks, Apache Spark, and modern data platform architecture. Proven experience in designing enterprise-scale ETL/ELT solutions, implementing Lakehouse architectures using Delta Lake, and optimizing large-scale data pipelines on Azure. Skilled in PySpark, SQL, ADF, ADLS, Synapse Analytics, CI/CD automation, and data governance frameworks. Adept at collaborating with business stakeholders, data scientists, and analytics teams to deliver robust, scalable, and cost-efficient data solutions that drive business insights and operational excellence.

Qualifications

3- 5 years of experience in Data Engineering.

Strong hands-on experience with Databricks.

Expertise in Apache Spark (PySpark or Scala Spark).

Proficiency in Python, SQL, PostgreSQL.

Experience designing and implementing ETL/ELT pipelines.

Strong understanding of data warehousing concepts and data modeling.

Experience working with large-scale distributed data processing systems.

Knowledge of CI/CD processes and version control tools such as Git.

Strong analytical, troubleshooting, and problem-solving skills.

Excellent communication and collaboration skills.

Experience with Microsoft Azure cloud services, including:

Experience with Delta Lake and Lakehouse architecture.

Knowledge of real-time data processing and streaming technologies such as Kafka.

Experience with Infrastructure as Code (Terraform, ARM Templates, or Bicep).

Familiarity with data governance and security best practices.

Responsibilities

Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.

Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.

Develop data ingestion frameworks from various data sources including databases, APIs, files, and streaming platforms.

Implement data transformation, cleansing, and enrichment processes.

Optimize Spark jobs for performance, scalability, and cost efficiency.

Collaborate with business stakeholders, data analysts, and data scientists to understand data requirements.

Ensure data quality, reliability, and governance across data platforms.

Monitor and troubleshoot production data pipelines and resolve performance bottlenecks.

Participate in code reviews, architecture discussions, and best practice implementation.

Create and maintain technical documentation for data solutions and workflows.

Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Azure Databricks Data Engineer
Azure Databricks Data Engineer

Infosys • Bengaluru

In loco
INR 1.800.000 - 3.200.000
Azure Data Engineer
Azure Data Engineer

InfoBeans • Indore District, Pune District

In loco
INR 3.000.000 - 5.400.000
Databricks data engineer
Databricks data engineer

NYX TECH SOLUTION • Pune District

In loco
INR 1.200.000 - 2.100.000
Data Engineering Consultant - Azure Databricks
Data Engineering Consultant - Azure Databricks

Optum • Hyderabad

In loco
INR 4.000.000 - 7.000.000
Sr. Databricks Engineer
Sr. Databricks Engineer

Fa Ewjt Saasfaprod1 • Pune District

Ibrido
INR 3.500.000 - 6.000.000
Sr. Databricks Engineer
Sr. Databricks Engineer

ExlService Holdings, Inc. • Bengaluru

Ibrido
INR 4.500.000 - 7.500.000
Azure Databricks Data Engineer
Azure Databricks Data Engineer

WTW Global Delivery And Solutions • Kolkata District

Ibrido
INR 1.200.000 - 2.600.000
Azure Databricks Data Engineer
Azure Databricks Data Engineer

WTW group • Kolkata District

In loco
INR 1.800.000 - 3.400.000
Senior Data Brick Engineer
Senior Data Brick Engineer

Zenithive Engineering • Ahmedabad District

In loco
INR 1.200.000 - 2.400.000
Data Engineer - Databricks
Data Engineer - Databricks

Onedatasoftware • Coimbatore District

In loco
INR 1.000.000 - 1.500.000