Senior Data Engineer

Kroll; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Kroll is looking for a Senior Data Engineer in Bengaluru, India, to enhance their data practice using AI, machine learning, and advanced analytics. You will design and build scalable data infrastructure while collaborating with various stakeholders, ensuring data integrity and quality.

The ideal candidate will have 5+ years of data engineering experience, proficiency in ETL/ELT development, and a strong understanding of cloud architectures including Azure and Databricks.

Qualifications

• 5+ years of proven experience in data engineering with a track record of success.
• Experience writing ETL/ELT jobs and working with cloud platforms.
• Solid problem-solving and analytical skills.

Responsibilities

• Design and build scalable data infrastructure and Medallion architecture.
• Develop robust ETL/ELT applications for data ingestion and analytics.
• Collaborate with stakeholders for technical solutions and support.

Skills

ETL/ELT development

Azure

Databricks

Python

SQL

REST APIs

Excellent communication

Ability to work with an international team

Education

Bachelor’s/Master’s degree in Computer Science, Engineering, or related field

Tools

FastAPI

Pydantic

Polars

Pandas

Delta Lake

Docker

Kubernetes

CI/CD

Git

Job description

We are hiring a Senior Data Engineer to join our team. At Kroll, we are building a strong, forward-looking data practice that integrates artificial intelligence, machine learning, and advanced analytics. You will be designing, building, and integrating data pipelines from diverse sources and collaborate with teams that serve the world’s largest financial institutions, law enforcement bodies, and government agency.

Day-to-day Responsibilities

• Design and build scalable organizational data infrastructure and Medallion architecture within a Lakehouse environment
• Develop robust, fault-tolerant ETL/ELT applications for seamless data ingestion, transformation, and distribution to enable analytics, reporting, and AI workloads
• Work with different stakeholders and teams to assist with data related technical solutions and support their data infrastructure needs
• Explore and experiment with new use cases, frameworks, and tools to enhance AI capabilities, ensuring data integrity, quality, and reliability
• Identify and implement infrastructure re-designs to improve scalability, optimize data delivery, and automate manual workflows
• Choose the best tools/services/resources to build robust data pipelines
• Collaborate with cross-functional teams to understand data requirements, create robust data models, and deliver actionable insights
• Monitor, troubleshoot, and optimize jobs for performance, addressing data pipeline bottlenecks and ensuring cost efficiency
• Continuously improve engineering processes, balancing speed, quality, and business impact
• Coach, mentor, and provide technical guidance to junior engineers, fostering a culture of continuous learning and development
• Stay updated on emerging technologies and trends in data engineering, recommending and implementing innovative solutions

Requirements

• Bachelor’s/Master’s degree in computer science, engineering, or a related field
• 5+ years of proven experience in data engineering, delivering business-critical software solutions for large enterprises with a consistent track record of success
• Experience writing ETL/ELT jobs
• Experience with Azure and Databricks Platform
• Experience with Python, SQL, and REST APIs
• Excellent communication and the ability to reason about trade-offs
• Ability to work with an international team

Desired Skills

• Cloud architecture principles: compute, storage, networks, security, cost
• Proficiency in using open-source tools, frameworks like FastAPI, Pydantic, Polars, Pandas, Delta Lake, Docker, Kubernetes
• Knowledge of CI/CD, Git, or infrastructure-as-code concepts
• Strong project management skills, with the ability to prioritize tasks and manage multiple projects simultaneously in an Agile environment
• Understanding of how data engineering feeds into Business Intelligence and reporting tools (Power BI/Tableau)
• Strong problem-solving and analytical skills
• Strategic thinker and strong execution orientation
• Ability to work in cross-functional teams
• Attention to detail and data quality

Kroll is committed to equal opportunity and diversity, and recruits people based on merit.