Walk-in | Databricks Data Engineer

Tata Consultancy Services; Visakhapatnam; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tata Consultancy Services in Visakhapatnam is seeking a hands-on data engineer to design, build and optimize data pipelines on the Databricks platform using PySpark and Delta Lake.

You will ingest data from batch and streaming sources, implement ETL/ELT patterns, ensure security and compliance, and collaborate with architects, analytics teams, and DevOps in an Agile environment.

Strong Python/SQL skills and experience with notebooks, Airflow, and cloud storage are essential.

Qualifications

• Hands-on experience with Databricks platform and PySpark/Spark SQL for data processing.
• Delta Lake for ACID-compliant storage and reliability.
• Notebooks for orchestrating workflows and collaboration.
• Strong Python and SQL programming skills with data ingestion from batch and streaming sources.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake).

Skills

Databricks platform

PySpark / Spark SQL

Delta Lake

Python

SQL

Data ingestion

ETL/ELT design patterns

Cloud storage & compute

Airflow

Git / Version control

Agile / DevOps practices

Data security & compliance

Tools

Notebooks

Job description

NOTE: Please carry a copy of your Aadhar or PAN Card Xerox

Relevant Experience Required

• Hands-on experience in developing, maintaining, optimizing data pipelines using Databricks.
• Knowledge of ETL/ELT pipelines, data ingestion, transformation, and orchestration.
• Exposure to data lake, data warehouse, or lakehouse architectures.
• Experience with data migration, integration, and automation.
• Collaboration with senior engineers, architects, and analytics teams for solution delivery.
• Familiarity with security, access control, and compliance best practices.
• Working experience in Agile and DevOps environments

Must Have Skills

• Hands-on experience with Databricks platform, including:
• PySpark / Spark SQL for data processing and transformation
• Delta Lake for ACID-compliant data storage
• Notebooks for workflow orchestration and collaborative development
• Strong programming skills in Python and SQL.
• Experience in data ingestion from batch and streaming sources.
• Knowledge of ETL/ELT design patterns and data pipeline optimization.
• Familiarity with cloud data storage and compute environments (AWS, Azure, or GCP).
• Basic understanding of workflow orchestration tools (Airflow, Databricks Jobs).
• Exposure to DevOps concepts, CI/CD pipelines, and version control (Git).
• Awareness of data security, access control, and compliance considerations.
• Understanding of performance tuning, partitioning, and caching strategies in Spark.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake).