Senior Data Engineer IND

OSBIndia Private Ltd.; India; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

OSBIndia Private Ltd. is seeking a Sr. Data Engineer to translate business requirements into Azure cloud-based big data solutions.

You will build data pipelines, data warehouses, and data lakes using Azure services and will collaborate with product owners to ensure data platforms meet enterprise needs. The role requires substantial experience with Azure data technologies, streaming and batch architectures, and a focus on performance, cost control, and governance within a banking-adjacent

Qualifications

• 6–10 years IT experience preferably in Azure Data and Analytics implementation.
• Minimum 6 years of Data engineering (Azure) delivery experience including analysis / development / configuration/ deployment.
• 5 years of implementation experience in Azure Data engineering projects.
• Experience in an MI / BI / Analytics environment (Kimball, lake house, data lake) is essential
• Proficiency in Python/SQL/SparkSQL/PySpark
• Significant technical skills such as Transact SQL language, relational database skills are essential
• Experience building high-volume, low-latency pipelines and Lakehouse implementations with Delta Lake.
• Demonstrated cost control in Azure (cluster policies, job orchestration, storage tiering, optimization).
• Understanding of data governance and security best practices
• Experience in Streaming and batch architecture (e.g., Kafka, Kafka streams, Spark, Flink)
• Exposure to CI / CD pipelines. Knowledge of containerization, orchestration, and Kubernetes engine would be an added advantage.
• Development of data pipelines using API ingestion and Streaming ingestionmethods.
• Experience in working in a fast-paced agile environment.

Responsibilities

• Implement robust ETL/ELT pipelines using Azure Data Factory/Synapse Pipelines and Azure Databricks (PySpark/Spark SQL) for batch and streaming data.
• Ingest and process streams using Azure Event Hubs / Kafka, Databricks Structured Streaming, or Azure Stream Analytics.
• Develop Delta Lake/Lakehouse layers on ADLS Gen2; build curated datasets and dimensional models on Synapse Dedicated SQL Pools.
• Tune Spark clusters, SQL pools, and pipelines for performance, reliability, and cost.
• Implement DQ frameworks (expectations/unit tests), lineage, and monitoring.
• Implement DevOps workflows with Azure DevOps/GitHub Actions, Bicep/Terraform/ARM, environment promotion, and release governance.
• Triage incidents, conduct root cause analysis, and drive post-incident improvements and runbooks.
• Analyse current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services
• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for Azure cloud-based data warehousing and data pipelines.
• Work closely with the Azure Data engineering Manager and technical leads to ensure the execution of plans corresponds with what is being promised.
• Data processing efficiency improvement, migrations from on-prem to public cloud platforms.

Skills

Azure Data Factory

Azure Databricks

Azure Synapse

Python

SQL

PySpark

Delta Lake

Kafka

Tools

Azure DevOps

Kubernetes

Job description

About us:
OsbIndia (OSBI) is an offshore subsidiary of OSB Group. OSBI was incorporated in 2004 as a key part of the OSB Group’s business strategy to provide operational service support. OSBI works with the Group’s trading brands in the UK, providing exceptional customer support and service delivery to new and existing customers.
OSBI also prides itself on offering operational excellence by devising and utilizing process improvements and functional efficiencies.

• Job Purpose

The Sr. Data Engineer will be responsible for translating business requirements into Azure Cloud-based big data solutions for the data modernisation program. You will be responsible for building complex data pipelines, data warehouses, and data lakes using Azure services like Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

The Sr. Data Engineer will work closely with other members of the data engineering team and product owners ensuring Data strategy and platforms fulfil current needs and are aligned with the overall vision of the organisation. They will be responsible for delivery of enterprise scale data transformation/modernisation programmes.

The ideal candidate will be an experienced data technologist and have a blend of experience across delivery and the practical application of contemporary data technologies. Experience in working within the Banking Industry (in banking, vendors or a large tech firm) is highly desirable.

• Core Responsibilities
• Implement robust ETL/ELT pipelines using Azure Data Factory/Synapse Pipelines and Azure Databricks (PySpark/Spark SQL) for batch and streaming data.
• Ingest and process streams using Azure Event Hubs / Kafka, Databricks Structured Streaming, or Azure Stream Analytics.
• Develop Delta Lake/Lakehouse layers on ADLS Gen2; build curated datasets and dimensional models on Synapse Dedicated SQL Pools.
• Tune Spark clusters, SQL pools, and pipelines for performance, reliability, and cost.
• Implement DQ frameworks (expectations/unit tests), lineage, and monitoring.
• Implement DevOps workflows with Azure DevOps/GitHub Actions, Bicep/Terraform/ARM, environment promotion, and release governance.
• Triage incidents, conduct root cause analysis, and drive post-incident improvements and runbooks.
• Analyse current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services
• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for Azure cloud-based data warehousing and data pipelines.
• Work closely with the Azure Data engineering Manager and technical leads to ensure the execution of plans corresponds with what is being promised.
• Data processing efficiency improvement, migrations from on-prem to public cloud platforms.

• Experience Requirements
• Overall 6-10 years of IT experience preferably in Azure Data and Analytics implementation.
• Minimum 6 years of Data engineering (Azure) delivery experience including analysis / development / configuration/ deployment.
• 5 years of implementation experience in Azure Data engineering projects
• Experience in an MI / BI / Analytics environment (Kimball, lake house, data lake) is essential
• Proficiency in Python/SQL/SparkSQL/PySpark
• Significant technical skills such as Transact SQL language, relational database skills are essential
• Experience building high-volume, low-latency pipelines and Lakehouse implementations with Delta Lake.
• Demonstrated cost control in Azure (cluster policies, job orchestration, storage tiering, optimization).
• Understanding of data governance and security best practices
• Experience in Streaming and batch architecture (e.g., Kafka, Kafka streams, Spark, Flink)
• Good Exposure in development with CI / CD pipelines. Knowledge of containerization, orchestration, and Kubernetes engine would be an added advantage.
• Development of data pipelines using API ingestion and Streaming ingestionmethods.
• Experience in working in a fast-paced agile environment.

We understand your career and how you progress is as unique as your individual personality. We've created a culture and an environment that encourages personal growth and offers our people opportunities to learn and succeed. Whether you're in the early stages of your career or already have an established profession, we're constantly seeking to hire talented individuals who want to make a difference and grow with us. We're a connected company working together to create a business in which we can all take pride and prosper.
If you want to know more about OSBI, please click on the website link: www.osb-india.com

To know more about OSBI culture please find us on Instagram @OSBINDIA

To find out more about the roles & updates please follow our LinkedIn Page www.linkedin.com/company/osbi.