AWS DataBricks Engineer

GeekSoft Consulting; Bengaluru; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

GeekSoft Consulting in Bengaluru, India, seeks a senior Data Engineer to design, build, and continuously improve the client’s online platform. You will own data pipelines, work with Databricks, PySpark, and SQL, and ensure scalability and reliability across cloud environments.

The ideal candidate has 8+ years in data engineering, deep experience with Delta Lake, Databricks workflows, Terraform, and streaming technologies.

Qualifications

• 8+ years of experience in Data Engineering.
• Experience as a Databricks Engineer designing, developing, and maintaining scalable data solutions.

Responsibilities

• Help design, build and continuously improve the clients online platform.
• Research, suggest and implement new technology solutions following best practices/standards.
• Take responsibility for the resiliency and availability of different products.
• Be a productive member of the team.

Skills

Databricks

PySpark

SQL

Data pipelines

ETL/ELT

Delta Lake

Databricks Workflows

Terraform

Azure

AWS

GCP

Git

CI/CD

DevOps

Streaming technologies

Data warehousing

Tools

Unity Catalog

Terraform

Job description

• Help design, build and continuously improve the clients online platform.
  
• Research, suggest and implement new technology solutions following best practices/standards.
  
• Take responsibility for the resiliency and availability of different products.
  
• Be a productive member of the team.
  

Requirements

• 8+ years of experience in Data Engineering.
  
• Experienced Databricks Engineer to design, develop, and maintain scalable data solutions using Databricks and cloud technologies.
  
• Develop and maintain data pipelines using Databricks, PySpark, and SQL.
  
• Work with Delta Lake, Databricks Workflows, and Jobs.
  
• Build and optimize ETL/ELT pipelines for large-scale data processing.
  
• Collaborate with Data Engineers, Architects, and business stakeholders.
  
• Troubleshoot and optimize pipeline and data-processing performance.
  
• Implement data quality, monitoring, and engineering best practices.
  
• Strong hands-on experience with Databricks and PySpark.
  
• Good knowledge of SQL, ETL/ELT, and Data Warehousing.
  
• Experience with Azure, AWS, or GCP.
  
• Familiarity with Git, CI/CD, and DevOps practices.
  
• Strong communication and problem-solving skills.
  
• Experience with Azure Data Factory, Databricks Unity Catalog, Terraform, and streaming technologies.
• A challenging, innovating environment.
  
• Opportunities for learning where needed.