Data Engineer+Python+Pyspark

Alike Thoughts; India; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

LTIMindtree is seeking an experienced Data Engineer with strong Python and PySpark expertise to design, develop and maintain scalable data pipelines and analytics solutions in Bengaluru. The role requires handling large datasets, data transformations, and integration across multiple sources.

The ideal candidate will optimize PySpark jobs, implement data quality processes and collaborate with data scientists, analysts and stakeholders.

Qualifications

• 38 years of relevant experience in Data Engineering.
• Strong hands-on experience with Python.
• Strong experience with PySpark / Apache Spark.
• Excellent knowledge of SQL.
• Good understanding of ETL/ELT concepts and data pipeline development.
• Experience working with large datasets and distributed data processing.
• Git/version control.
• Strong analytical and problem-solving skills.

Responsibilities

• Design, develop and maintain scalable data pipelines.
• Develop data processing and transformation solutions using Python and PySpark.
• Perform data extraction, transformation and loading (ETL/ELT).
• Work with structured and unstructured data from multiple sources.
• Optimize PySpark jobs for performance and scalability.
• Implement data quality, validation and error-handling processes.
• Work with SQL databases, data warehouses and data lakes.
• Collaborate with Data Scientists, Analysts, Architects and business stakeholders.
• Troubleshoot data pipeline failures and production issues.
• Participate in code reviews, unit testing and technical documentation.
• Follow Agile development practices and engineering best practices.

Skills

Python

PySpark

SQL

ETL/ELT

Data pipelines

Git

Analytics

Tools

Databricks

Snowflake

Azure Data Lake

AWS S3

Kafka

Airflow

Job description

Data Engineer Python + PySpark | 38 Years | Bengaluru | Hybrid

Company: Indian MNC
Role: Data Engineer
Experience: 3-8 Years
Location: PAN India LTI Mindtree
Work Mode: Hybrid
Notice Period: Up to 30 Days Preferred
Employment Type: Full Time

Job Description

LTIMindtree is looking for experienced Data Engineers with strong hands-on expertise in Python and PySpark to design, develop and maintain scalable data processing and analytics solutions.

The ideal candidate should have experience working with large datasets, building data pipelines, data transformation and integration using modern data engineering technologies.

Key Responsibilities

• Design, develop and maintain scalable data pipelines.
• Develop data processing and transformation solutions using Python and PySpark.
• Perform data extraction, transformation and loading (ETL/ELT).
• Work with structured and unstructured data from multiple sources.
• Optimize PySpark jobs for performance and scalability.
• Implement data quality, validation and error-handling processes.
• Work with SQL databases, data warehouses and data lakes.
• Collaborate with Data Scientists, Analysts, Architects and business stakeholders.
• Troubleshoot data pipeline failures and production issues.
• Participate in code reviews, unit testing and technical documentation.
• Follow Agile development practices and engineering best practices.

Mandatory Skills

• 38 years of relevant experience in Data Engineering.
• Strong hands-on experience with Python.
• Strong experience with PySpark / Apache Spark.
• Excellent knowledge of SQL.
• Good understanding of ETL/ELT concepts and data pipeline development.
• Experience working with large datasets and distributed data processing.Good understanding of data structures, transformations and optimization.
• Experience with Git/version control.
• Strong analytical and problem-solving skills.

Good to Have

• Experience with AWS / Azure / GCP.
• Knowledge of Databricks.
• Experience with Snowflake / Azure Data Lake / AWS S3.
• Knowledge of Kafka or other streaming technologies.
• Experience with Airflow or other workflow orchestration tools.
• Exposure to CI/CD and DevOps practices.
• Knowledge of data warehousing and dimensional modelling.

Candidate Profile

• Strong programming and data engineering fundamentals.
• Good communication and collaboration skills.
• Ability to work independently and as part of a team.
• Experience handling enterprise-scale data projects.
• Candidates who can join within 30 days are preferred.

Work Location & Mode

Location: Bengaluru
Work Mode: Hybrid
Notice Period: Maximum 30 Days Preferred