DataBricks Engineer

GeekSoft Consulting; Bengaluru; On-site; INR 3,600,000 - 5,400,000; Full time

Job summary

GeekSoft Consulting is seeking an experienced Data Engineer to design, build and continuously improve the client’s online platform using Databricks, PySpark and SQL on cloud data stacks. You will implement scalable ETL/ELT pipelines, ensure data quality and system resiliency, and collaborate with data architects and stakeholders across Azure, AWS or GCP.

8+ years in data engineering is required; strong communication and problem solving are essential.

Qualifications

• 8+ years of experience in Data Engineering.
• Hands-on Databricks and PySpark experience.
• Experience with cloud platforms (Azure, AWS, or GCP).
• Strong SQL, ETL/ELT, and data warehousing knowledge.
• Familiarity with Git, CI/CD, and DevOps practices.

Responsibilities

• Help design, build and improve the client's online platform.
• Research, suggest and implement new technology solutions following best practices/standards.
• Take responsibility for the resiliency and availability of different products.
• Be a productive member of the team.
• Troubleshoot and optimize pipeline and data-processing performance.
• Implement data quality, monitoring, and engineering best practices.
• Collaborate with Data Engineers, Architects, and business stakeholders.

Skills

Databricks

PySpark

SQL

ETL/ELT

Data pipelines

Data quality

CI/CD

DevOps

Communication

Problem-solving

Tools

Delta Lake

Databricks Workflows

Databricks Unity Catalog

Terraform

Git

Azure Data Factory

Cloud platforms (Azure/AWS/GCP)

CI/CD tools

Streaming technologies

Job description

• Help design, build and continuously improve the clients online platform.
  
• Research, suggest and implement new technology solutions following best practices/standards.
  
• Take responsibility for the resiliency and availability of different products.
  
• Be a productive member of the team.
  

Requirements

• 8+ years of experience in Data Engineering.
  
• Experienced Databricks Engineer to design, develop, and maintain scalable data solutions using Databricks and cloud technologies.
  
• Develop and maintain data pipelines using Databricks, PySpark, and SQL.
  
• Work with Delta Lake, Databricks Workflows, and Jobs.
  
• Build and optimize ETL/ELT pipelines for large-scale data processing.
  
• Collaborate with Data Engineers, Architects, and business stakeholders.
  
• Troubleshoot and optimize pipeline and data-processing performance.
  
• Implement data quality, monitoring, and engineering best practices.
  
• Strong hands-on experience with Databricks and PySpark.
  
• Good knowledge of SQL, ETL/ELT, and Data Warehousing.
  
• Experience with Azure, AWS, or GCP.
  
• Familiarity with Git, CI/CD, and DevOps practices.
  
• Strong communication and problem-solving skills.
  
• Experience with Azure Data Factory, Databricks Unity Catalog, Terraform, and streaming technologies.
• A challenging, innovating environment.
  
• Opportunities for learning where needed.