Data Engineers

IBS Software; Ernakulam; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

IBS Software is seeking a data engineer to design, build, and maintain ETL pipelines and data products across multi-cloud environments. You will drive data investigations and resolve complex technical and operational issues, delivering scalable solutions for large datasets.

The role emphasizes data architecture knowledge, streaming data, and building robust data models, with a focus on secure, efficient, and maintainable data processing in a distributed setup.

Qualifications

• Experience designing and maintaining data processing ETL pipelines and data products.
• Proficiency with multi-cloud, multi-region distributed processing environments.

Responsibilities

• Design, build and maintain data processing ETL pipelines & data products in a multi-cloud, multi-region, distributed processing context choosing the right technologies.
• Drive data investigations to deliver a resolution of technical, procedural, and operational issues.
• Solve complex business problems by producing scalable, flexible, efficient and supportable solutions using appropriate technologies.

Skills

Data engineering patterns

Data architecture

Big Data Technologies

Programming Languages

Data Warehousing

ETL/ELT Tools

Database Systems

Data Modelling

Streaming Data

Version Control

Containerization & Orchestration

Data Security

Monitoring & Logging

Tools

Git

Docker

Kubernetes

Airflow

Job description

Job Description

• Design, build and maintain data processing ETL pipelines & data products in a multi-cloud, multi-region, distributed processing context choosing the right technologies.
• Drive data investigations to deliver a resolution of technical, procedural, and operational issues.
• Solve complex business problems by utilizing disciplined development methodology, producing scalable, flexible, efficient and supportable solutions using appropriate technologies.

Skills

• Data engineering patterns: Sound knowledge of the different data engineering patterns to determine what to use and when
• Data architecture: Understanding of data architecture and frameworks like Data mesh, data fabric etc.
• Big Data Technologies: Proficiency in big data technologies such as data lake, EMR, Glue for analysing large datasets.
• Programming Languages: Proficiency in programming languages commonly used in data engineering, such as Python, Java, or Scala. with OOP expertise
• Data Warehousing: Strong knowledge of data warehousing solutions preferably Amazon Snowflake.
• ETL/ELT Tools: Familiarity with orchestration tools
• Database Systems: Expertise in both relational and NoSQL databases.
• Data Modelling: Skill in designing efficient data models, both for OLAP and OLTP systems.
• Streaming Data: Knowledge of streaming data technologies.
• Version Control: Experience with version control systems : Git.
• Containerization and Orchestration: Understanding of containerization technologies (Docker) and container orchestration platforms (Kubernetes).
• Data Security: Knowledge of data encryption, access control, and compliance with data privacy regulations.
• Monitoring and Logging: Proficiency in setting up monitoring and logging solutions for data pipelines using different tools