Data Engineer - Databricks

Tata Consultancy Services; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services is seeking a Data Engineer to design and optimize data pipelines on Azure, including Databricks, Data Factory, and related services. The role requires 4–15 years of experience in data engineering and a strong grasp of data architecture, governance, and security.

You will collaborate with data scientists and analysts to deliver scalable solutions, monitor performance, and contribute to CI/CD workflows using Azure DevOps.

Qualifications

• 4–15 years of data engineering experience with a focus on Azure-based data solutions.
• Experience designing and implementing scalable data pipelines and data architectures on Azure.
• Strong data governance, security, and quality practices in data platforms.

Responsibilities

• Design, build, and optimize data pipelines using Azure Data Factory and Databricks.
• Develop scalable data architectures, data warehouses, and storage solutions on Azure.
• Ensure data accuracy, security, and governance across data systems.
• Monitor, troubleshoot, and optimize pipelines for performance and cost.
• Collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
• Contribute to CI/CD workflows using Azure DevOps.

Skills

Data pipelines

Azure Databricks

Azure services

Data architecture

Data quality

Security governance

Performance tuning

Collaboration

CI/CD

Azure DevOps

Tools

Azure Data Factory

SQL

NoSQL

Azure Databricks

Azure DevOps

Job description

TCS is hiring Data Engineer (Databricks with Azure)

Location – Any

Years of Experience – 4-15 years

Job Description

Required Technical Skill Set

• Data Pipeline Development: Design, build, and optimize scalable data pipelines using Azure Data Factory, Azure Databricks, and other Azure services for data ingestion and processing.
• Data Architecture: Develop and implement scalable data architectures, data warehouses, and data storage solutions (SQL/NoSQL) on Azure.
• Data Quality & Security: Ensure data accuracy, consistency, integrity, and security through robust governance and best practice implementation.
• Performance Optimization: Monitor, troubleshoot, and optimize data systems and pipelines for optimal performance, efficiency, and cost-effectiveness.
• Collaboration: Work closely with data scientists, data analysts, business stakeholders, and solution architects to understand requirements and deliver effective data solutions.
• Automation: Implement automated data pipelines and workflows, and contribute to CI/CD (Continuous Integration/Continuous Delivery) using tools like Azure DevOps.