Technical Lead

Cognizant; Kolkata District; Híbrido; INR 2.800.000 - 4.000.000; Jornada completa

Descripción de la vacante

Cognizant is seeking a seasoned data engineering professional for a hybrid day shift role to design, implement, and optimize data pipelines using Azure Data Factory and Databricks. The position targets complex analytics platforms supporting strategic decisions across life and annuities insurance domains.

You will lead end-to-end data ingestion, optimize transformations, enforce data quality, and collaborate with architects and domain experts to deliver scalable, secure data solutions.

Formación

• Hands-on expertise in Azure Data Factory pipeline design and monitoring.
• Advanced Databricks skills for Spark-based processing and performance optimization.
• Experience in relational/analytical data modeling for reliable reporting.
• Knowledge of life and annuities insurance data structures.
• Experience with hybrid work models and structured documentation.
• Version control and release management across environments.
• Azure Data Engineer Associate or Azure Solutions Architect Expert preferred.

Responsabilidades

• Design and implement scalable data pipelines using ADF and Databricks.
• Coordinate end-to-end data ingestion from diverse sources.
• Optimize Databricks transformations and job scheduling.
• Implement data validation, monitoring, and alerting.
• Collaborate with architects and data consumers to translate requirements.
• Promote best practices in data engineering and documentation.
• Ensure data security configurations align with regulatory expectations.
• Support incident resolution and root-cause analysis.
• Mentor junior data engineers.

Conocimientos

Azure Data Factory

Databricks

Data pipeline design

Data quality monitoring

Cloud data architecture

Herramientas

Databricks clusters

Spark

Descripción del empleo

Job Summary

This hybrid day shift role seeks a seasoned technical professional with eight to twelve years of experience to guide complex data solutions using Azure Data Factory and Databricks for a multinational organization. The role focuses on designing implementing and optimizing robust data pipelines and analytics platforms that support strategic decisions and enhance business outcomes across diverse functions including life and annuities insurance domains.

Responsibilities

• Drive the design and implementation of scalable data pipelines using Azure Data Factory and Databricks to ensure timely and reliable data availability for critical business analytics and reporting needs.
• Coordinate end to end data ingestion workflows from diverse source systems into centralized data platforms while ensuring data quality consistency and alignment with enterprise architecture guidelines.
• Optimize Databricks based data transformation processes through efficient coding practices resource tuning and job scheduling to deliver high performing and cost effective data solutions.
• Implement robust data validation monitoring and alerting mechanisms across Azure Data Factory and Databricks workloads to proactively detect issues and minimize disruption to downstream consumers.
• Collaborate closely with architects business analysts and data consumers to translate functional requirements into technical designs that maximize the value of enterprise data assets.
• Guide the adoption of best practices in data engineering including reusable frameworks standardized coding conventions and documentation to improve maintainability and onboarding efficiency across project teams.
• Evaluate and refine data security and access control configurations across Azure services to protect sensitive information and align with regulatory expectations in relevant insurance and financial contexts.
• Coordinate hybrid work collaboration by planning clear task breakdowns online reviews and knowledge sharing sessions that keep distributed team members aligned and productive without requiring travel.
• Support incident resolution and root cause analysis for data pipeline failures or performance degradation by using telemetry and logs to quickly restore stable operations and prevent recurrences.
• Promote the effective use of Databricks features such as notebooks clusters and job orchestration to streamline data experimentation model development and production deployment activities.
• Engage with stakeholders from life and annuities insurance when applicable to understand policy claims and actuarial data characteristics and reflect these domain nuances in data model and pipeline design.
• Document solution architectures data flows and operational procedures in a clear and accessible manner so that support teams and future projects can easily extend or reuse established patterns.
• Mentor junior data engineers by reviewing deliverables sharing practical patterns for Azure Data Factory and Databricks usage and encouraging thoughtful problem solving that contributes to organizational goals.

Qualifications

• Demonstrate strong hands on expertise in Azure Data Factory including pipeline design activity configuration integration runtime management and operational monitoring for complex data environments.
• Apply advanced skills in Databricks covering notebook development Spark based data processing cluster configuration and performance optimization to support large scale data transformation workloads.
• Exhibit solid understanding of relational and analytical data modeling concepts along with experience in building structured data layers that enable reliable reporting and downstream analytics.
• Utilize practical knowledge of life and annuities insurance data structures such as policies premiums claims and reserves when applicable to tailor data solutions that reflect core domain processes.
• Bring experience with hybrid work models including effective communication remote collaboration tools and structured documentation practices that maintain clarity without on site presence.
• Show capability in using version control and release management practices to manage data pipeline changes in a controlled and auditable manner across non production and production environments.

Certifications Required

Azure Data Engineer Associate or Azure Solutions Architect Expert certification preferred for this role.