Senior Data Engineer

NB Healthcare Technologies; Hyderabad; On-site; INR 4,000,000 - 7,500,000; Full time

Job summary

NB Healthcare Technologies in Hyderabad, India seeks a Lead Data Engineer to architect and deliver scalable data pipelines and governance across cloud platforms.

You will collaborate with multiple scrum teams, implement Delta Lake with ACID transactions, and ensure data quality and reliability using Spark, PySpark, and Azure Databricks.

The role demands 7–10 years of leadership experience, self-motivation, and strong problem-solving abilities.

Qualifications

• 7-10 years of experience as a Lead Data Engineer.
• Self-driven with minimal supervision.
• Proven experience with T-SQL programming, Azure Databricks, Spark (PySpark/Scala), Delta Lake, Unity Catalog, ADLS Gen2.
• Microsoft TFS, Visual Studio, Devops exposure.
• Experience with cloud platforms such as Azure or any.
• Analytical, problem-solving mindset.

Responsibilities

• Work with different scrum teams to develop all the quality database programming requirements of the sprint.
• Experience in Azure cloud platforms like Advanced Python Programming, Databricks, Azure SQL , Data factory (ADF), Data Lake, Data storage, SSIS.
• Create and deploy scalable ETL/ELT pipelines with Azure Databricks by utilizing PySpark and SQL.
• Create Delta Lake tables with ACID transactions and schema evolution to support real-time and batch processing.
• Experience in Unity Catalog for centralized data governance, access control, and data lineage tracking.
• Independently analyse, solve, and correct issues in real time, providing problem resolution end-to-end.
• Develop unit tests to be able to test them automatically.
• Use SOLID development principles to maintain data integrity and cohesiveness.

Skills

Azure Databricks

PySpark

SQL

Delta Lake

Unity Catalog

T-SQL

Spark

Azure

Python

Devops

Tools

Azure DevOps

Visual Studio

TFS

Job description

Role & responsibilities

• Work with different scrum teams to develop all the quality database programming requirements of the sprint.
• Experience in Azure cloud platforms like Advanced Python Programming, Databricks, Azure SQL , Data factory (ADF), Data Lake, Data storage, SSIS.
• Create and deploy scalable ETL/ELT pipelines with Azure Databricks by utilizing PySpark and SQL.
• Create Delta Lake tables with ACID transactions and schema evolution to support real-time and batch processing.
• Experience in Unity Catalog for centralized data governance, access control, and data lineage tracking.
• Independently analyse, solve, and correct issues in real time, providing problem resolution end-to-end.
• Develop unit tests to be able to test them automatically.
• Use SOLID development principles to maintain data integrity and cohesiveness.

Preferred candidate profile

• 7-10 years of experience as a Lead Data Engineer.
• Self-driven with minimal supervision.
• Proven experience with T-SQL programming, Azure Databricks, Spark (PySpark/Scala), Delta Lake, Unity Catalog, ADLS Gen2
• Microsoft TFS, Visual Studio, Devops exposure.
• Experience with cloud platforms such as Azure or any.
• Analytical, problem-solving mindset.