Data Engineer

IBS Software; Ernakulam; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

IBS Software in Cochin (Kochi) seeks a Lead Data Engineer to drive the design, development, and maintenance of scalable ETL/ELT pipelines and data products across multi-cloud environments. You will define data engineering patterns, resolve complex data and production issues, and design efficient, scalable data solutions with best practices.

This role emphasizes technical leadership and collaboration with architects and business teams.

Qualifications

• 5-10 years of data engineering experience.
• Experience designing scalable data pipelines across multi-cloud/multi-region environments.
• Strong understanding of data architectures and modern patterns.

Responsibilities

• Lead design, development, and maintenance of scalable ETL/ELT pipelines and data products.
• Define data engineering patterns and architectures based on requirements.
• Drive technical investigations and resolve complex data and production issues.
• Design scalable, flexible, efficient data solutions with disciplined development practices.
• Provide technical leadership and mentorship to data engineering teams.
• Collaborate with architects, engineering, and business stakeholders to deliver robust data solutions.

Skills

Data Mesh

Data Fabric

Data Lake

Data Warehouse

Amazon EMR

AWS Glue

Python

Java

Scala

Snowflake

ETL/ELT pipelines

Data orchestration tools

Relational databases

NoSQL databases

OLAP

OLTP

Git

Docker

Kubernetes

Encryption

Access control

Data privacy

Compliance

Monitoring

Logging

Observability

Tools

SQL

Job description

Lead Data Engineer

• Location (s): Cochin
• Years of Experience: 5-10yrs

What You’ll Do

• Lead the design, development, and maintenance of scalable ETL/ELT pipelines and data products in multi-cloud, multi-region, distributed environments.
• Define and apply appropriate data engineering patterns and architectures based on business and technical requirements.
• Drive technical investigations and resolve complex data, operational, and production issues.
• Design scalable, flexible, efficient, and supportable solutions using appropriate technologies and disciplined development practices.
• Provide technical leadership, guidance, and mentorship to data engineering teams.
• Collaborate with architects, engineering teams, and business stakeholders to deliver robust data solutions.

Key Skills

• Data Engineering & Architecture: Strong understanding of data engineering patterns and modern architectures such as Data Mesh, Data Fabric, Data Lake, and Data Warehouse.
• Big Data & Cloud: Proficiency in Amazon EMR, AWS Glue, Data Lake, and technologies for processing large datasets.
• Programming: Strong proficiency in Python, Java, or Scala, with solid OOP expertise.
• Data Warehousing: Strong knowledge of data warehousing solutions, preferably Snowflake.
• ETL/ELT & Orchestration: Hands‑on experience with ETL/ELT pipelines and data orchestration tools.
• Databases: Strong expertise in relational and NoSQL databases.
• Data Modelling: Experience designing efficient OLAP and OLTP data models.
• Streaming: Knowledge of streaming data technologies and architectures.
• Version Control: Experience with Git and modern development practices.
• Containers & Orchestration: Understanding of Docker and Kubernetes.
• Data Security: Knowledge of encryption, access control, data privacy, and compliance.
• Monitoring & Logging: Experience implementing monitoring, logging, alerting, and observability for data pipelines.

Role & Responsibilities

• Lead Data Engineer
  • What You’ll Do
  • Lead the design, development, and maintenance of scalable ETL/ELT pipelines and data products in multi-cloud, multi-region, distributed environments.
  • Define and apply appropriate data engineering patterns and architectures based on business and technical requirements.
  • Drive technical investigations and resolve complex data, operational, and production issues.
  • Design scalable, flexible, efficient, and supportable solutions using appropriate technologies and disciplined development practices.
  • Provide technical leadership, guidance, and mentorship to data engineering teams.
  • Collaborate with architects, engineering teams, and business stakeholders to deliver robust data solutions.
  • Key Skills
  • Data Engineering & Architecture: Strong understanding of data engineering patterns and modern architectures such as Data Mesh, Data Fabric, Data Lake, and Data Warehouse.
  • Big Data & Cloud: Proficiency in Amazon EMR, AWS Glue, Data Lake, and technologies for processing large datasets.
  • Programming: Strong proficiency in Python, Java, or Scala, with solid OOP expertise.
  • Data Warehousing: Strong knowledge of data warehousing solutions, preferably Snowflake.
  • ETL/ELT & Orchestration: Hands‑on experience with ETL/ELT pipelines and data orchestration tools.
  • Databases: Strong expertise in relational and NoSQL databases.
  • Data Modelling: Experience designing efficient OLAP and OLTP data models.
  • Streaming: Knowledge of streaming data technologies and architectures.
  • Version Control: Experience with Git and modern development practices.
  • Containers & Orchestration: Understanding of Docker and Kubernetes.
  • Data Security: Knowledge of encryption, access control, data privacy, and compliance.
  • Monitoring & Logging: Experience implementing monitoring, logging, alerting, and observability for data pipelines.

Technical Qualifications

Python + AWS EMR + AWS Glue + Snowflake + ETL/ELT + Data Engineering + Data Architecture + Big Data + SQL