Tech Lead - Python Pyspark

Tech Mahindra; Chennai District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tech Mahindra in Chennai seeks a Data Engineer with a strong Python background to design, build, and maintain scalable data pipelines and architectures. The ideal candidate will have 5–7 years of experience transforming raw data into actionable insights for data driven decision making.

You will collaborate with data scientists and analysts, implement data models, optimize storage, and ensure data quality. Bachelor's degree required; growth in analytics and cloud data services offered.

Qualifications

• Bachelor's degree in CS/Engineering or equivalent.
• 5–7 years of data engineering experience.
• Strong Python with Pandas/NumPy data manipulation.
• Experience building scalable data pipelines and data warehouses.
• Proficiency in SQL and ETL tools; familiarity with cloud data services.

Responsibilities

• Design, develop, and maintain robust data pipelines and ETL processes.
• Collaborate with data scientists and analysts to understand requirements.
• Implement data models and architectures for BI and analytics.
• Optimize storage and ensure data access performance.
• Monitor data pipelines for quality and reliability.
• Document processes and stay updated on industry trends.

Skills

Python

Pandas

NumPy

SQL

Data warehousing

ETL

Communication

Analytical thinking

Problem solving

Education

Bachelor's degree in Computer Science or Engineering

Tools

Hadoop

Spark

Tableau

Power BI

Docker

Kubernetes

Git

AWS

Azure

GCP

Job description

A Bachelor's or Higher Degree is the minimum entry required for the position

Image

Job Description

• Skill Set : SQL,Python,Hadoop,Hive,pyspark

We are seeking a skilled Data Engineer with a strong background in Python to join our dynamic team at TechM in Chennai. The ideal candidate will have 5 7 years of experience in data engineering, with a proven track record of designing, building, and maintaining scalable data pipelines and architectures. You will play a crucial role in transforming raw data into actionable insights, enabling data driven decision making across the organization.

Responsibilities:

• Design, develop, and maintain robust data pipelines and ETL processes to support data ingestion and transformation.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high quality data solutions.
• Implement data models and architectures that support business intelligence and analytics initiatives.
• Optimize data storage and retrieval processes to ensure efficient data access and performance.
• Monitor and troubleshoot data pipeline performance, ensuring data quality and reliability.
• Document data engineering processes, workflows, and best practices for future reference.
• Stay updated with industry trends and emerging technologies in data engineering and analytics.

Mandatory Skills:

• Strong proficiency in Python, with experience in data manipulation and analysis libraries such as Pandas and NumPy.
• Experience with data warehousing solutions and ETL tools.
• Solid understanding of SQL and experience with relational databases.
• Familiarity with big data technologies such as Hadoop, Spark, or similar frameworks.
• Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and their data services.

Preferred Skills:

• Experience with data visualization tools (e.g., Tableau, Power BI).
• Understanding of machine learning concepts and frameworks.
• Experience with containerization technologies (e.g., Docker, Kubernetes).
• Familiarity with version control systems (e.g., Git).
• Strong problem solving skills and the ability to work independently and collaboratively.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5 7 years of experience in data engineering or a related role.
• Proven experience in building and maintaining data pipelines and architectures.
• Excellent communication skills, with the ability to convey complex technical concepts to non technical stakeholders.