Azure Data Engineer - Pune

Lumiq; Pune District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

LUMIQ, a leading data and analytics company in the BFSI sector, is seeking an Azure Data Engineer to design, build, and maintain cloud-based data infrastructure in Pune. You will work at the nexus of cloud tech, data engineering, and BFSI domain needs to deliver scalable data solutions.

The role requires strong Azure, Python, SQL and Spark skills with batch ingestion and processing, and collaboration with data scientists to ensure high-quality data access for analytical purposes.

Qualifications

• Minimum 3 years of experience as a Data Engineer with Azure Cloud, Python, SQL, Spark, Batch Ingestion and Processing.
• Strong SQL querying skills and experience with Python and Spark.
• Experience with Batch Ingestion and Processing on Azure.
• Experience with databases such as Postgres, MySQL, or Oracle.

Responsibilities

• Design, develop, and implement data pipelines, ETL processes, and data integration solutions.
• Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures for BFSI needs.
• Optimize data storage, processing, and retrieval for performance and cost-efficiency in Azure.
• Implement data security and compliance for BFSI data.
• Work with data scientists and analysts to enable access to high-quality data for analysis.
• Troubleshoot data-related issues to ensure data availability and reliability for BFSI customers.
• Stay updated on industry best practices and emerging cloud technologies to drive improvement.

Skills

SQL querying

Python

Spark

Azure Cloud

Batch Ingestion

Data retrieval

Tools

Databricks

Synapse

Microsoft Fabric

Kafka

Job description

About the job Azure Data Engineer - Pune

Location: Pune
5 Days WFO
Who we are:
LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance (FSI) industry. We aretrusted by the world's largest FSIs, including insurers, banks, AMCs, and NBFCs, to address their data
challenges. Our clients include 40+ enterprises with over $10B in deposits/AUM, collectively representingabout 1B customers globally.
Our expertise lies in creating next-gen data technology products to help FSI enterprises organize, manage, andeffectively use data. We have consistently challenged the status quo, introducing many industry-firsts like thefirst enterprise data platform in Asia on cloud for a regulated entity

Job Description:

We are seeking a highly skilled Azure Data Engineer to join our dynamic team. As a Data Engineer at LUMIQ, you will play a crucial role in designing, developing, and maintaining our cloud-based data infrastructure to support our BFSI customers. You will work at the intersection of cloud technologies, data engineering, and the BFSI domain to deliver robust and scalable data solutions.

Key Responsibilities:

• Design, develop, and implement data pipelines, ETL processes, and data integration solutions.
• Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures that align with BFSI industry needs.
• Optimize data storage, processing, and retrieval for maximum performance and cost-efficiency in Azure cloud environments.
• Implement data security and compliance measures to ensure the protection and integrity of sensitive BFSI data.
• Work closely with data scientists and analysts to enable seamless access to high-quality data for analytical purposes.
• Troubleshoot and resolve data-related issues, ensuring data availability and reliability for BFSI customers.
• Stay updated on industry best practices, emerging cloud technologies, and trends in the BFSI sector to drive continuous improvement and innovation.

Qualifications:

Minimum 3 years experience as a Data Engineer with strong skills in Azure Cloud, Python, SQL, Spark, Batch Ingestion and Processing.

Must Have Skills:

• Strong SQL querying skills
• Strong experience in Python and Spark.
• Strong experience in Batch Ingestion and Processing.
• Strong experience working on Azure Cloud platform.
• Experience with any Database (Postgres, MySQL, Oracle, etc.)

Good to Have Skills:

• Databricks
• Synapse
• Microsoft Fabric
• Streaming and Kafka (For any future streaming use cases