Data Engineer- Mumbai location

BDO India; Mumbai; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

BDO India is seeking a Data Engineer to build and maintain scalable data pipelines in a Databricks/PySpark environment. You will work on lakehouse architectures with Delta Lake and orchestrate jobs using Airflow, ensuring data quality and governance in production.

The role requires 2-5 years of hands-on experience in cloud data engineering (AWS/Azure), strong SQL and proficient Python, plus the ability to diagnose pipeline issues and optimize performance in a fast-paced, regulated environment.

Qualifications

• Strong hands-on experience with Databricks/Spark/PySpark in a production environment able to walk through real debugging scenarios (job failures, data skew, schema mismatches, late-arriving data) with specifics, not textbook answers.
• Delta Lake or equivalent lakehouse experience incremental processing, ACID handling, time travel, optimisation strategies.
• Strong SQL and Python for data engineering (not just scripting able to reason about performance trade-offs).
• Cloud data engineering on AWS (S3, Glue, Lambda, EMR) or Azure equivalents.
• Orchestration tooling (Airflow or similar) — designing DAGs for real dependency chains,not simple linear jobs.
• Experience with data quality/governance tooling (e.g. Collibra, Unity Catalog, or equivalent) and audit/lineage requirements typical in regulated environments.
• Comfortable being tested on scenario-based, whiteboard-style problems — e.ge."pipeline is failing intermittently in prod, walk me through yourdiagnosis,"or "design an ingestion framework for 10 new data sources with different schemas."

Responsibilities

• Build and maintain scalable data pipelines in a Databricks/PySpark environment.
• Design and implement lakehouse architectures with Delta Lake for reliable, fast data processing.
• Develop and orchestrate data workflows using Airflow, ensuring correct dependencies and timely data delivery.
• Ensure data quality and governance, with lineage and audit capabilities in regulated environments.
• Diagnose production issues, optimize performance, and propose robust ingestion frameworks for multiple data sources.

Skills

Databricks

PySpark

Python

SQL

Airflow

Delta Lake

Job description

Role:- Data Engineer

Skills:- Databricks, Pyspark, Python, SQL, Airflow, Delta lake

Location:- Mumbai (5 Days in Office)

Experience:- 2-5 Years

About Company

BDO is a global network of professional services firms with a presence in over 166 countries, revenue of over USD 14 billion, and experience of over 60 years. Its a leading service provider for the mid-markets with client service at its heart.

BDO India Services Private Limited (or BDO India) is the India member firm of BDO International. BDO India offers strategic, operational, accounting and tax, and regulatory advisory & assistance for both domestic and international organizations across a range of industries. BDO India is led by more than 300+ Partners & Directors with a team of over 10,000 professionals operating across 14 cities and 20 offices. We expect to grow sizably in the coming 2-6 years, adding various dimensions to our business and multiplying and increasing the current team size multi-fold.

Required Experience

• Strong hands-on experience with Databricks/Spark/PySpark in a production environment able to walk through real debugging scenarios (job failures, data skew, schema mismatches, late-arriving data) with specifics, not textbook answers.
• Delta Lake or equivalent lakehouse experience incremental processing, ACID handling, time travel, optimisation strategies.
• Strong SQL and Python for data engineering (not just scripting able to reason about performance trade-offs).
• Cloud data engineering on AWS (S3, Glue, Lambda, EMR) or Azure equivalents.
• Orchestration tooling (Airflow or similar) — designing DAGs for real dependency chains,not simple linear jobs.
• Experience with data quality/governance tooling (e.g. Collibra, Unity Catalog, or equivalent) and audit/lineage requirements typical in regulated environments.
• Comfortable being tested on scenario-based, whiteboard-style problems — e.g.,"pipeline is failing intermittently in prod, walk me through yourdiagnosis,"or "design an ingestion framework for 10 new data sources with different schemas."

Good to Have

• Prior experience in BFSI / regulated environments (data lineage, audit trail, compliance-driven data handling).
• Exposure to observability stacks (Grafana, Kibana, Prometheus, OpenSearch).
• CI/CD experience (Jenkins,GitHub Actions) in a banking change-control context. Understanding of document management, workflow orchestration, and case management platforms.