Data Engineering Consultant

Virtusa; Hyderabad; Hybrid; INR 3,500,000 - 6,000,000; Full time

Job summary

Virtusa is seeking a data engineer to design and deploy robust batch and streaming pipelines, ensuring schema evolution, replay-safety, and data quality across GL balances and related data products.

You will build stateful/ stateless transformations, apply dimensional and lakehouse models, and enable analytics through Cube.dev, Power BI, and AI tools, while enforcing RBAC, encryption, masking, and PHI compliance.

Qualifications

• 6+ years of data engineering experience with advanced SQL and Python, Scala, or Java.
• Hands-on experience with Kafka and distributed processing engines (Spark, Flink, Beam).
• Experience building batch/streaming pipelines on cloud platforms, lakehouses, or data warehouses.
• Proficiency in dimensional modeling, data partitioning, incremental processing, and performance tuning.
• Experience with orchestration, CI/CD, data-quality testing, production monitoring, and access controls.

Responsibilities

• Pipeline Engineering: Design and deploy reliable batch, event-streaming, and CDC ingestion pipelines with automated patterns for schema evolution, replay-safety, quarantine, and reprocessing.
• Streaming & Data Modeling: Build stateful/stateless transformations handling late events, watermarks, reattributions, contract amendments, and claim adjustments. Implement dimensional, Data Vault, lakehouse, or effective-dated/bitemporal models.
• Data Quality & Controls: Implement automated control totals, reconciliation, and audit controls to guarantee completeness, validity, and line-of-business integrity across GL balances and data products.
• Semantic & Platform Enablement: Deliver curated datasets optimized for Cube.dev, Power BI, and AI tools. Standardize business definitions (PMPM, IBNR, MCR, capitation). Enforce RBAC, encryption, masking, and PHI compliance.

Skills

Advanced SQL

Python

Scala/Java

Data modeling

Tools

Kafka

Spark

Flink

Beam

dbt

Cube.dev

Power BI

Job description

Key Responsibilities

• Pipeline Engineering: Design and deploy reliable batch, event-streaming, and CDC ingestion pipelines with automated patterns for schema evolution, replay-safety, quarantine, and reprocessing.
• Streaming & Data Modeling: Build stateful/stateless transformations handling late events, watermarks, reattributions, contract amendments, and claim adjustments. Implement dimensional, Data Vault, lakehouse, or effective-dated/bitemporal models.
• Data Quality & Controls: Implement automated control totals, reconciliation, and audit controls to guarantee completeness, validity, and line-of-business integrity across GL balances and data products.
• Semantic & Platform Enablement: Deliver curated datasets optimized for Cube.dev, Power BI, and AI tools. Standardize business definitions (PMPM, IBNR, MCR, capitation). Enforce RBAC, encryption, masking, and PHI compliance.

Mandatory Qualifications

• 6+ years of data engineering experience with advanced SQL and Python, Scala, or Java.
• Hands-on experience with Kafka and distributed processing engines (Spark, Flink, Beam).
• Experience building batch/streaming pipelines on cloud platforms, lakehouses, or data warehouses.
• Proficiency in dimensional modeling, data partitioning, incremental processing, and performance tuning.
• Experience with orchestration, CI/CD, data-quality testing, production monitoring, and access controls.

Preferred Qualifications

• Domain knowledge in healthcare claims, eligibility, capitation, sub-capitation, attribution, or value-based care.
• Experience with financial accounting datasets (journals, subledgers, GL balances, reconciliations).
• Familiarity with healthcare standards (X12 837/835, HL7, FHIR), dbt, Cube.dev, Power BI, and knowledge graphs/ontologies.