Data Engineer

Lorven Technologies Inc.; Tamil Nadu; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Lorven Technologies Inc. is seeking an experienced Azure Data Engineer in Chennai to design, develop, and optimize scalable data pipelines using Python, Spark, Azure Databricks, and Snowflake. The role focuses on ETL/ELT processes, data transformation, and cloud data platforms across structured and unstructured sources.

The ideal candidate has 6+ years in data engineering with strong PySpark, SQL, and Azure cloud experience, delivering robust data integration and performance-tuned workloads.

Qualifications

• 6+ years of experience in data engineering.
• Proficient in Python for data processing and ETL pipelines.
• Expertise in Spark/PySpark for large-scale processing.
• Hands-on with Azure Databricks and Snowflake.
• Strong SQL and data transformation skills.
• Experience with data security and governance is a plus.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Azure.
• Develop data processing and transformation solutions using Python and Spark.
• Build and optimize data engineering workloads using Azure Databricks.
• Develop PySpark applications for large-scale data processing and transformation.
• Design and implement data ingestion pipelines from multiple sources.
• Work with Snowflake for data loading, transformation, data modeling, and performance optimization.
• Implement data integration between Azure Databricks and Snowflake.
• Perform data cleansing, validation, transformation, and quality checks.
• Optimize Spark jobs, Databricks notebooks, SQL queries, and data pipelines for performance and scalability.
• Develop reusable frameworks and components for data processing and ingestion.
• Troubleshoot data pipeline failures, performance issues, and data quality problems.
• Collaborate with Data Architects, Analysts, Developers, and business stakeholders to understand data requirements.
• Follow best practices for data security, governance, monitoring, and operational support.
• Participate in code reviews, technical documentation, testing, and deployment activities.

Skills

Python

Spark / PySpark

Azure Databricks

Snowflake

ETL/ELT

SQL

Azure Cloud

Big Data

Job description

Job Description – Azure Data Engineer

Job Title: Azure Data Engineer

Experience: 6+ Years

Location: Chennai

Employment Type: Full-Time / FTE

Job Summary

We are looking for an experienced Azure Data Engineer with 6+ years of hands‑on experience in designing, developing, and maintaining scalable data engineering solutions using Python, Apache Spark, Azure Databricks, and Snowflake. The ideal candidate should have strong expertise in data processing, ETL/ELT pipelines, data transformation, and cloud-based data platforms.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Azure.
• Develop data processing and transformation solutions using Python and Apache Spark.
• Build and optimize data engineering workloads using Azure Databricks.
• Develop PySpark applications for large‑scale data processing and transformation.
• Design and implement data ingestion pipelines from multiple structured and unstructured sources.
• Work with Snowflake for data loading, transformation, data modeling, and performance optimization.
• Implement data integration between Azure Databricks and Snowflake.
• Perform data cleansing, validation, transformation, and quality checks.
• Optimize Spark jobs, Databricks notebooks, SQL queries, and data pipelines for performance and scalability.
• Develop reusable frameworks and components for data processing and ingestion.
• Troubleshoot data pipeline failures, performance issues, and data quality problems.
• Collaborate with Data Architects, Analysts, Developers, and business stakeholders to understand data requirements.
• Follow best practices for data security, governance, monitoring, and operational support.
• Participate in code reviews, technical documentation, testing, and deployment activities.

Mandatory Skills

• 6+ years of experience in Data Engineering
• Python – Strong
• Apache Spark / PySpark – Strong
• Azure Databricks – Strong
• Snowflake – Strong
• Strong experience in ETL/ELT and data pipeline development
• Strong SQL and data transformation skills
• Experience working with large-scale datasets and distributed data processing
• Good understanding of Azure cloud data engineering concepts