Data bricks Engineer

Tquanta Technologies; Chennai; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tquanta Technologies is seeking a Databricks Engineer to design and maintain scalable data pipelines using Databricks and Apache Spark in Chennai, India. You will work with Delta Lake for storage and perform data validation and performance tuning.

Responsibilities include building ETL/ELT pipelines with PySpark, Spark SQL, and managing transformations, while collaborating with cloud platforms such as AWS or Azure to deploy reliable data solutions.

Qualifications

• Proficient in building scalable data pipelines using Databricks and Spark.
• Strong knowledge of PySpark, Spark SQL and Delta Lake for data storage and transformation.
• Experience with cloud platforms (AWS or Azure) and data validation.
• Ability to optimize data processing and troubleshoot performance issues.

Responsibilities

• Develop ETL/ELT pipelines using Databricks, PySpark, and Spark SQL.
• Work with Delta Lake for data storage and processing.
• Build and optimize data transformations and workflows.
• Work with cloud platforms such as AWS or Azure.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

ETL/ELT

Data validation

Performance tuning

Tools

AWS

Azure

Job description

Job Description:

We are looking for a Databricks Engineer with hands-on experience in developing and maintaining scalable data pipelines using Databricks and Apache Spark.

Key Responsibilities:

• Develop ETL/ELT pipelines using Databricks, PySpark, and Spark SQL.


• Work with Delta Lake for data storage and processing.


• Build and optimize data transformations and workflows.


• Work with cloud platforms such as AWS or Azure.


• Perform data validation, troubleshooting, and performance tuning