Data Engineer / Data Architect

Cognizant; Chennai Metropolitan Area; Hybrid; INR 3,000,000 - 4,800,000; Full time

Job summary

Cognizant is seeking an experienced Data Engineer/Architect to design and optimize cloud-based data platforms for an insurance analytics stack. Expect work on AWS, PySpark, Python, and Databricks to support analytics, governance, and scalable architectures in a hybrid environment.

The role emphasizes building end-to-end data pipelines, data models, quality controls, and collaboration with actuaries and underwriters to deliver measurable business value.

Qualifications

• 8–12 years of experience in data engineering or data architecture
• Hands-on expertise with AWS data and compute services for analytical workloads
• Proficiency in PySpark and Python for scalable transformations and testing

Responsibilities

• Design scalable data architectures on AWS for end-to-end data flows in insurance contexts
• Build PySpark/Python data pipelines in Databricks to ingest and curate data from multiple sources
• Optimize Databricks workflows and Spark jobs for performance and cost efficiency
• Develop reusable data models and data marts for insurance entities
• Establish data quality rules and reconciliation for regulatory reporting datasets
• Collaborate with actuaries underwriters and product teams to translate requirements into data solutions
• Implement secure data access patterns on AWS ensuring governed self-service analytics
• Create technical documentation including data flow diagrams and data dictionaries
• Automate testing monitoring and alerting for data pipelines in Python and cloud-native tools
• Mentor peers in PySpark coding standards data modeling and cloud-based patterns

Skills

PySpark

Python

Databricks

AWS

SQL

Data modeling

Data governance

DevOps/DataOps

Hybrid work model

Data pipelines

Tools

Databricks (notebooks, clusters)

AWS services

Spark

Job description

Job Summary

Design and optimize cloud based data platforms that power analytics and decision making for a global property and casualty insurance business using AWS PySpark Python and Databricks within a hybrid work model ensuring resilient data pipelines reliable governance and scalable architectures that enable accurate risk assessment and faster claims processing.

Responsibilities

• Design and implement scalable data architectures on AWS that support end to end data flows for property and casualty insurance underwriting pricing claims and policy administration use cases
• Build robust PySpark and Python based data pipelines in Databricks that ingest transform and curate structured and semi structured insurance data from multiple internal and external sources
• Optimize Databricks workflows and Spark jobs to achieve predictable performance cost efficiency and reliability for large volumes of policy premium exposure and claims data
• Develop reusable data models and canonical data sets that standardize property and casualty insurance entities such as policies coverages claims and reinsurance treaties across business lines
• Establish and maintain data quality rules data validation checks and reconciliation processes that improve accuracy timeliness and completeness of critical regulatory and financial reporting datasets
• Collaborate with actuaries underwriters claims analysts and product teams to translate analytical and reporting requirements into well defined data engineering solutions that create measurable business value
• Implement secure data access patterns on AWS using appropriate services and configurations that protect sensitive customer and financial information while enabling governed self service analytics
• Create and maintain technical documentation including data flow diagrams data dictionaries and architecture blueprints that provide clear traceability from source systems to curated consumption layers
• Automate testing monitoring and alerting for data pipelines using Python and cloud native tools to proactively identify data anomalies and pipeline failures before they affect downstream consumers
• Partner with enterprise and solution architects to align data engineering solutions with target cloud architecture standards helping to modernize legacy insurance platforms and decommission redundant assets
• Drive continuous improvement by profiling datasets identifying performance bottlenecks and implementing tuning strategies that shorten data delivery cycles for reporting dashboards and advanced analytics
• Support business continuity by implementing backup recovery and environment management strategies in AWS and Databricks that reduce operational risk and minimize downtime for critical data services
• Mentor peers and junior engineers by sharing best practices in PySpark coding standards data modeling for insurance and cloud based engineering patterns that raise the overall quality of team deliverables

Qualifications

• Possess eight to twelve years of experience in data engineering or data architecture roles with a consistent focus on delivering production grade solutions in complex enterprise environments
• Demonstrate strong hands on expertise with AWS data and compute services including data storage processing orchestration and security components used to run modern analytical workloads
• Exhibit advanced proficiency in PySpark and Python for building scalable transformation logic unit tests and utility frameworks that can be reused across multiple insurance data pipelines
• Bring proven experience working with Databricks to develop optimize and operate collaborative notebooks jobs and clusters tailored for large scale property and casualty data processing
• Show deep functional understanding of property and casualty insurance concepts including policies coverages rating factors claims lifecycle and regulatory reporting and apply this knowledge to data designs
• Apply solid knowledge of dimensional and relational data modeling practices to design curated data marts and semantic layers that serve analytics actuarial studies and operational reporting
• Utilize strong skills in SQL data profiling and data quality assessment to diagnose source system issues and drive corrective actions with business and technology stakeholders
• Demonstrate effective communication and collaboration abilities to work in a hybrid model with distributed teams conveying complex data topics in clear and concise business language
• Display familiarity with modern DevOps and DataOps practices such as version control continuous integration automated testing and environment promotion for data engineering solutions
• Show commitment to continual learning by staying updated on evolving AWS Databricks and open source data engineering capabilities and by actively incorporating relevant innovations into project delivery