Data Engineer

Tata Elxsi; Pune District, Chennai District, Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

TATA ELXSI is seeking a Senior/Lead Big Data Engineer to design, architect, and implement scalable data platforms and ETL pipelines. You will lead a team, define best practices, and deliver high-quality data solutions with Python, PySpark, and distributed systems. Experience in the Supply Chain domain is a plus.

You will architect end-to-end data pipelines on Databricks, enforce governance, and collaborate with analytics teams to enable real-time decision-making using BI tools like Tableau.

Qualifications

• 4+ years of Big Data/Data Engineering experience.
• Strong proficiency in Python and PySpark with hands-on coding and optimization.
• Advanced SQL skills including complex transformations and performance tuning.
• Experience with Databricks (Delta Lake, notebooks, job orchestration).
• Hands-on experience with Kafka for event-driven architectures.
• Strong experience with MongoDB and Oracle databases.
• Experience with Tableau or similar BI/reporting tools.
• Knowledge of ETL frameworks, data modelling and data warehousing concepts.
• Experience designing scalable and resilient data architectures.

Responsibilities

• Architect scalable ETL/data pipeline frameworks using Python and PySpark.
• Lead end-to-end data engineering solutions on Databricks (batch and streaming).
• Define data engineering standards, governance, and best practices.
• Implement real-time data pipelines using Kafka.
• Integrate heterogeneous data sources (MongoDB, Oracle, APIs, external systems).
• Collaborate with stakeholders to translate requirements into scalable solutions.
• Enable data-driven decisions with reporting and dashboards (Tableau).
• Lead performance tuning, cost optimization, and reliability improvements.

Skills

Python

PySpark

SQL

Tools

Databricks

Delta Lake

Notebooks

Kafka

MongoDB

Oracle

Tableau

Airflow

AWS

Azure

GCP

Job description

TATA ELXSI is hiring for Data Engineer

Location: [Bangalore , Hyderabad, Chennai , Trivandrum]

Experience: 3-6yrs Exp (Relevant Experience)

Employment Type: [Full-time/Contract]

Job Summary

We are looking for a Senior/Lead Big Data Engineer to drive the design, architecture, and implementation of scalable data platforms and ETL pipelines. This role requires deep technical expertise in Python, PySpark, and distributed data systems, along with strong leadership in guiding teams, defining best practices, and delivering high-quality data solutions. Supply Chain domain experience is highly desirable.

Key Responsibilities

• Architect and design scalable, high-performance ETL/data pipeline frameworks using Python and PySpark.
• Lead end-to-end data engineering solutions on Databricks (batch + streaming)
• Define and enforce data engineering standards, best practices, and governance
• Drive design and implementation of real-time data pipelines using Kafka
• Oversee integration of heterogeneous data sources (MongoDB, Oracle, APIs, external systems)
• Collaborate with business stakeholders, product owners, and analytics teams to translate requirements into scalable solutions
• Enable data-driven decision-making by supporting advanced reporting and dashboards (Tableau)
• Lead performance tuning, cost optimization, and reliability improvements of data pipelines
• Mentor and guide junior engineers; conduct code reviews and design reviews
• Ensure production stability with strong monitoring, alerting, and incident management practices
• Contribute to CI/CD setup, automation, and DevOps best practices for data pipelines

Required Skills & Qualifications

• At least 4+ years in Big Data/Data Engineering
• Strong expertise in Python and PySpark (hands-on coding + optimization)
• Advanced SQL skills (complex transformations, performance tuning)
• Proven experience in Databricks (Delta Lake, notebooks, job orchestration)
• Hands-on experience with Kafka for event-driven architecture
• Strong experience with MongoDB and Oracle databases
• Experience working with Tableau or similar BI/reporting tools
• Deep understanding of ETL frameworks, data modelling, and data warehousing concepts
• Experience designing scalable and resilient data architectures

Preferred Qualifications

• Strong Supply Chain domain knowledge (order management, inventory, logistics, demand forecasting)
• Experience with cloud platforms (AWS/Azure/GCP) .
• Exposure to data governance, data quality frameworks, and metadata managementExperience with workflow orchestration tools (Airflow or similar)
• Knowledge of microservices/data APIs integration

Leadership & Ownership Expectations

• Drive solution architecture discussions and decision-making
• Own delivery commitments, timelines, and quality metrics
• Collaborate across teams (QA, DevOps, Product, Business) for E2E solution delivery
• Champion innovation, automation, and continuous improvement initiatives

Soft Skills

• Strong communication and stakeholder management
• Ability to influence technical and business decisions
• Proactive problem-solving and ownership mindset
• Experience working in Agile/Scrum environments.