Principal Software Engineer

Nielseniq India; Pune District; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Nielseniq India is seeking a senior data engineering leader to design and deliver production-grade data solutions on Azure, leveraging Databricks, Spark and Delta Lake. You will guide data pipelines, lead a cross-functional team in an agile environment and drive DevOps practices for reliable, scalable systems.

The role requires extensive experience with Azure data services, distributed computing, and modern software architecture.

Qualifications

• 10+ years of strong software development experience.
• Azure-based data platforms including Databricks, Data Lake, Synapse and Functions.
• Expertise in Apache Spark and Delta Lake for large-scale data pipelines.
• Experience with Airflow for workflow orchestration.
• Azure cloud experience with containerisation (Docker, Kubernetes) and IaC (Terraform, Bicep).
• Knowledge of microservices, event-driven architectures and distributed computing.
• Leading teams in an agile environment with CI/CD, automated tests and DevOps practices.
• Strong problem-solving and stakeholder communication skills.
• Data engineering, data science or analytics-heavy background is desirable.
• Proven line management experience with mentoring and performance management.

Responsibilities

• Design and build production-grade applications using Python/Scala/Java or similar.
• Develop and optimize large-scale data pipelines on Azure-based platforms.
• Lead data engineering teams in an agile setting and mentor engineers.
• Implement CI/CD pipelines and automated testing to improve release velocity.
• Collaborate with data scientists and analytics teams to enable data-driven insights.
• Architect microservices and event-driven systems to support scalable data solutions.
• Ensure robust operational practices including monitoring, logging and incident response.

Skills

Software engineering

Team leadership

Agile

CI/CD

DevOps

Distributed systems

Problem solving

Stakeholder communication

Tools

Azure Databricks

Azure Data Lake

Azure Synapse

Azure Functions

Apache Spark

Delta Lake

Apache Airflow

Docker

Kubernetes

Terraform

Bicep

Job description

Role & responsibilities

• 10+ years of strong software development experience required
• Strong software engineering background with experience in designing and building production-grade applications inPython, Scala, Java, or similar languages.
• Proven experience withAzure-based data platforms, specificallyDatabricks, Azure Data Lake, Azure Synapse, and Azure Functions.
• Expertise inApache SparkandDelta Lake, with experience building large-scale distributed data pipelines.
• Experience withworkflow orchestration toolslikeApache Airflow.
• Cloud experience inAzure, includingcontainerisation (Docker, Kubernetes)andinfrastructure as code (Terraform, Bicep).
• Strong knowledge of modern software architecture, includingmicroservices, event-driven systems, and distributed computing.
• Experience leading teams in anagileenvironment, with a strong understanding ofCI/CD pipelines, automated testing, and DevOps practices.
• Excellent problem-solving and communication skills, with the ability to engage with both technical and non-technical stakeholders.
• Previous experience indata engineering, data science, or analytics-heavy environmentsis highly desirable.
• Provenline management experience, including mentoring, career development, and performance management of engineering teams.