Python Data Engineer

Lonvec Technologies Private Limited; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Lonvec Technologies Private Limited is seeking an experienced Python Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions. The role focuses on building ETL workflows, processing large datasets, and optimizing data pipelines in collaboration with data scientists and engineering teams.

You will develop data integrations from multiple sources, perform data transformation and quality checks, and work with cloud-based data platforms and storage

Qualifications

• Experience building scalable data pipelines in Python.
• Knowledge of ETL/ELT workflows.
• Experience with Spark-based processing.
• Ability to optimize data pipelines for large datasets.
• Experience collaborating with data scientists and engineers.
• Familiarity with cloud data platforms and storage.

Responsibilities

• Design and maintain scalable data pipelines using Python.
• Develop ETL/ELT workflows.
• Build data processing solutions with Python and Spark.
• Work with large datasets and optimize performance.
• Monitor and troubleshoot data pipelines.
• Collaborate with analytics and engineering teams.

Skills

Python

Spark

ETL

Data pipelines

Data processing

SQL

Tools

Hadoop

Job description

We are looking for an experienced Python Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions.The ideal candidate will be responsible for building ETL workflows, processing large datasets, optimizing data pipelines, and collaborating with data scientists, analysts, and engineering teams to deliver reliable data solutions. Develop and maintain scalable data pipelines using Python Design and implement ETL/ELT workflows Build data processing solutions using Python and Spark frameworks Work with large datasets and optimize data processing performance Develop data integrations from multiple sources Perform data transformation, validation, and quality checks Work with cloud-based data platforms and storage solutions Optimize SQL queries and database performance Collaborate with analytics and engineering teams Monitor and troubleshoot data pipeline issues

Requirements

• Hadoop
• Data Warehousing