Data Engineer IND

OSBIndia Private Ltd.; Bengaluru; On-site; INR 900,000 - 1,200,000; Full time

Job summary

OSBIndia Private Ltd. is looking for a Data Engineer responsible for translating business requirements into Azure Cloud-based big data solutions. You will build complex data pipelines and work closely with the data engineering team to ensure alignment with the organization's data strategy.

The ideal candidate should have 4-5 years of IT experience, including at least 3 years in Azure Data Engineering, with proficiency in Python and SQL. This position promises a dynamic environment with opportunities for impactful work in data transformation.

Qualifications

• 4-5 years of IT experience, preferably in Data and Analytics implementation.
• 3 years of Azure Data Engineering experience with development and deployment.
• Experience in MI/BI/Analytics environments (e.g., Kimball).

Responsibilities

• Develop high-quality big-data solutions on Azure Cloud.
• Analyse business practices and identify opportunities for Azure Data Services.
• Develop best practices for data warehousing and data pipelines.

Skills

Python

SQL

SparkSQL

PySpark

Transact-SQL

Data governance

Kafka

NoSQL

Tools

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

Kubernetes

Job description

About Us

OsbIndia (OSBI) is an offshore subsidiary of OSB Group. OSBI was incorporated in 2004 as a key part of the OSB Group’s business strategy to provide operational service support. OSBI works with the Group’s trading brands in the UK, providing exceptional customer support and service delivery to new and existing customers.

OSBI also prides itself on offering operational excellence by devising and utilizing process improvements and functional efficiencies.

Job Purpose

The Data Engineer will be responsible for translating business requirements into Azure Cloud‑based big data solutions for the data modernisation program. The role involves building complex data pipelines, data warehouses, and data lakes using Azure services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

The Azure Data Engineer works closely with other members of the data engineering team and product owners to ensure that data strategy and platforms meet current needs and align with the overall vision of the organisation. The individual will deliver enterprise‑scale data transformation and modernisation programmes.

Ideal candidates are experienced data technologists with a blend of delivery and practical application of contemporary data technologies. Experience within the banking industry, either directly or through vendors or large technology firms, is highly desirable.

Core Responsibilities

• Develop high‑quality big‑data solutions on Azure Cloud leveraging Azure Databricks and other services for advanced data processing and analytics tasks, optimizing performance and scalability.
• Analyse current business practices, processes and procedures, and identify future business opportunities for leveraging Microsoft Azure Data & Analytics Services.
• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for Azure‑cloud‑based data warehousing and data pipelines.
• Work closely with the Azure Data Engineering Manager and technical leads to ensure the execution of plans aligns with commitments.
• Improve data processing efficiency, including migrations from on‑premises to public cloud platforms.

Experience Requirements

• Overall 4‑5 years of IT experience, preferably in Data and Analytics implementation.
• Minimum 3 years of Data Engineering (Azure) delivery experience covering analysis, development, configuration, and deployment.
• 2 years of implementation experience in Azure Data engineering projects.
• Experience in an MI/BI/Analytics environment (Kimball, lake house, data lake) is essential.
• Proficiency in Python, SQL, SparkSQL, and PySpark.
• Strong technical skills including Transact‑SQL language and relational database expertise.
• Understanding of data governance and security best practices.
• Experience with streaming and batch architecture (e.g., Kafka, Kafka Streams, Spark, Flink).
• Experience with at least one distributed data processing framework: Spark (Core, Streaming, SQL), Storm, or Flink.
• Experience with NoSQL solutions such as MongoDB, Cassandra, HBase, Cosmos DB, or cloud‑based NoSQL offerings like DynamoDB, Bigtable.
• Good exposure to development with CI/CD pipelines; knowledge of containerisation, orchestration, and Kubernetes is an added advantage.
• Experience in developing data pipelines using API ingestion and streaming ingestion methods.
• Experience working in a fast‑paced agile environment.