Data Engineer

Mintskill HR Solutions LLP; Mumbai; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

A leading HR recruitment firm in Mumbai is seeking a Data Engineer to design and maintain scalable data pipelines for the financial services sector. The ideal candidate should have over 5 years of experience in data engineering, strong programming in Python or Java, and proficiency in tools like Kafka and Spark. Responsibilities include building data lakes and integrating data from various sources. This role requires excellent collaboration skills and knowledge of data governance and compliance.

Qualifications

• 5+ years of experience in data engineering or related roles.
• Strong programming skills in Python, Java, or Scala.
• Experience with distributed data systems like Kafka and Spark.

Responsibilities

• Design, develop, and optimise ETL/ELT pipelines for structured and unstructured data.
• Construct and manage data lakes and warehouses.
• Integrate data from multiple sources using tools like Kafka.
• Implement data governance and data quality best practices.

Skills

Data engineering

Python

Java

Scala

SQL

Kafka

Hadoop

Spark

Docker

Jenkins

Tools

MS SQL

PostgreSQL

Oracle

MinIO

Clickhouse

Apache Pinot

Job description

BSE Tech is an established player in e-enabling businesses in the financial services sector. BSE Tech is a leading provider of cutting-edge IT solutions with a focus on Commodities, Banking and Financial Services markets in India.

As a Data Engineer at BTPL, you will design, build, and maintain scalable data pipelines and infrastructure to support analytics, reporting, and business intelligence in the BFSI and capital markets domains. You will work closely with data scientists, analysts, and application teams to ensure reliable, high-quality data delivery.

Key Responsibilities:

• Design, develop, and optimise ETL/ELT pipelines for structured and unstructured data.
  
• Build and manage data lakes, data warehouses, and real-time data streaming solutions.
  
• Integrate data from multiple sources (databases, APIs, files, external feeds) using tools like Kafka, MinIO, and message brokers.
  
• Work with distributed storage and compute platforms (Clickhouse, Apache Pinot, Hadoop, Spark, etc.).
  
• Implement data modelling, data quality, and data governance best practices.
  
• Collaborate with DevOps for CI/CD, monitoring, and automation (Docker, Jenkins, Azure DevOps).
  
• Ensure data security, privacy, and compliance with regulatory standards.
  
• Troubleshoot and resolve data pipeline and infrastructure issues.
  
• Document data flows, architecture, and technical specifications.
  

Requirements

Required Skills & Experience:

• 5+ years of experience in data engineering or related roles.
  
• Strong programming skills in Python, Java, or Scala.
  
• Experience with distributed data systems: Kafka, Clickhouse, Pinot, MinIO, Hadoop, Spark, or similar.
  
• Proficiency in SQL and working with relational (MS SQL, PostgreSQL, Oracle) and NoSQL databases.
  
• Hands-on experience with ETL/ELT tools and data integration frameworks.
  
• Experience with DevOps tools: Docker, Jenkins, Git, Azure DevOps.
  
• Understanding of data modelling, data warehousing, and real-time data processing.
  
• Good communication and teamwork skills.
  

Preferred:

• Experience in BFSI, capital markets, or fintech domains.
  
• Knowledge of data governance, data quality, and regulatory compliance.
  
• Certifications in relevant data engineering, cloud, or big data technologies.