Senior Consultant

Virtusa; New York (NY); On-site; USD 140,000 - 180,000; Full time

Job summary

Virtusa is seeking a Senior Data Engineer in New York to lead the design and optimization of complex data pipelines and architectures that support business requirements and analytics initiatives.

The role emphasizes Python PySpark development, AWS Glue and data services, and collaboration with cross-functional teams to deliver scalable data solutions in a healthcare finance context.

Qualifications

• 8+ years IT experience with at least 5 years in data engineering roles.
• Experience leading data engineering teams and delivering large-scale data pipelines.
• Strong Python and PySpark for data processing, with expertise in data lake architectures.

Responsibilities

• Lead development of data pipelines and architectures for large datasets.
• Design, construct, and test data architectures aligned with business requirements.
• Provide technical leadership for data projects, ensuring best practices and high-quality data solutions.
• Collaborate with product, finance, and other units to ensure pipelines meet business needs.
• Oversee development of data solutions enabling predictive and prescriptive analytics.
• Ensure data solutions maintain quality as data moves across environments.
• Align data architecture with Healthfirst solution architecture.

Skills

Python PySpark

SQL

Iceberg data lake

AWS Glue

AWS data services

Bash scripting

Healthcare finance domain

Agile

Problem solving

Communication

Job description

Senior Data Engineer

Position Summary

The Senior Data Engineer leads complex data engineering projects, working on designing data architectures that align with business requirements. This role focuses on optimizing data workflows, managing data pipelines, and ensuring the smooth operation of data systems.

Minimum Qualifications

8+ Years overall IT experience with minimum 5 years of work experience in below tech

Skills

Tech Skills

• Strong experience in Python Scripting and PySpark for data processing
• Proficiency in SQL, dealing with big data processing
• Proven experience in data processing for data lake in Iceberg format, with strong understanding of architecture
• Experience in AWS Glue jobs
• Experience in AWS cloud platform and its data services (S3, Redshift, Lambda, EMR, Airflow, Postgres, SNS, Event bridge)
• Expertise in BASH/Shell scripting
• Strong understanding of healthcare Finance systems and experience leading data engineering teams
• Experience in Agile environments.
• Excellent problem-solving skills and attention to detail.
• Effective communication and collaboration skills.

Responsibilities

• Leads development of data pipelines and architectures that handle large-scale data sets.
• Designs, constructs, and tests data architecture aligned with business requirements.
• Provides technical leadership for data projects, ensuring best practices and high-quality data solutions.
• Collaborates with product, finance, and other business units to ensure data pipelines meet business requirements.
• Oversees development of data solutions that enable predictive and prescriptive analytics.
• Ensures the technical quality of solutions, managing data as it moves across environments.

• Aligns data architecture to Healthfirst solution architecture.