Data Engineer (Databricks Specialist)_Virtual drive

Tata Consultancy Services; Hyderabad; On-site; INR 1,600,000 - 2,600,000; Full time

Job summary

Tata Consultancy Services in Hyderabad is seeking a Data Engineer (Databricks Specialist) with 6–12 years of experience to design and maintain scalable data pipelines using Databricks and PySpark. You will collaborate with cross-functional teams, optimize ETL processes, and ensure data quality across the data lifecycle while leveraging cloud platforms such as Azure, AWS, or GCP.

The role requires strong SQL, data modeling, and warehousing experience, with excellent communication in a fast-paced

Qualifications

• Hands-on experience with Databricks, Spark, and PySpark for big data processing.
• Proficient in Azure, AWS, or GCP for data engineering workflows.
• Strong SQL, schema design and dimensional modeling.
• Experience with ETL/ELT tools such as Azure Data Factory, Informatica, or Talend.
• Familiar with data warehousing tech like Azure Synapse, Snowflake or BigQuery.
• Strong problem-solving and debugging skills.
• Excellent communication and teamwork.
• Ability to work in a fast-paced environment.

Responsibilities

• Design and maintain scalable data pipelines using Databricks and PySpark.
• Implement and optimize ETL processes for large datasets.
• Collaborate with cross-functional teams to define data requirements.
• Ensure data integrity and quality across the lifecycle.
• Integrate solutions with cloud platforms and manage large-scale datasets.

Skills

Databricks & Spark

Cloud Platforms

SQL & Data Modeling

ETL/ELT Tools

Data Warehousing

Problem-Solving

Communication

Teamwork

Fast-Paced Environment

Tools

Azure Data Factory

Informatica

Talend

Azure Synapse

Snowflake

BigQuery

Power BI

Job description

Data Engineer (Databricks Specialist)

Exp: 6 to 12 years

Location: Hyderabad Only

Interview Mode: Virtual interview in Week days

Role & responsibilities

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

(Ideally should not be more than 3-5)

• Databricks & Spark: Hands-on experience with Databricks, Apache Spark, and PySpark for big data processing.
• Cloud Platforms: Proficiency in Azure (preferred), AWS, or GCP for data engineering workflows
• SQL & Data Modeling: Strong knowledge of SQL, schema design, and dimensional data modeling.
• ETL/ELT Tools: Experience with tools like Azure Data Factory, Informatica, or Talend.
• Data Warehousing: Familiarity with technologies such as Azure Synapse, Snowflake, or BigQuery.
• Strong problem-solving and debugging skills.
• Excellent communication and teamwork abilities.
• Ability to work in a fast-paced, dynamic environment.

Good-to-Have

• Knowledge of Event Hub, IoT Hub, and streaming analytics.
• Exposure to Power BI, Azure DevOps, and CI/CD pipelines.
• Understanding of machine learning libraries (MLlib) and predictive analytics.

SN

Responsibility of / Expectations from the Role

1Design and maintain scalable data pipelines using Databricks and PySpark.

2Implement and optimize ETL processes for large datasets.

3Collaborate with cross-functional teams to define data requirements.

4Ensure data integrity and quality across the lifecycle.

5Integrate solutions with cloud platforms and manage large-scale datasets.

Preferred candidate profile