Data Engineer, Specialist

Vanguard India; Hyderabad; Hybrid; INR 1,400,000 - 2,100,000; Full time

Job summary

Vanguard India in Hyderabad is seeking a Data Engineer to build and operate scalable data pipelines on AWS, using PySpark, Spark, and modern data tooling. You will own pipeline reliability, monitor jobs, and implement transformations that support financial data processing.

You should have 5+ years in data engineering, strong Python and SQL, and a reconciliation-first mindset for data quality. The role offers a hybrid work model in Hyderabad with a focus on AI and cloud-based platforms.

Qualifications

• 5+ years in data engineering with strong Python and SQL.
• Hands-on PySpark / Spark for large-scale data transformation.
• Production experience with AWS data services— Glue, Athena, S3, Lambda, Step Functions.
• Solid data modelling skills and a reconciliation-first approach to data quality.
• Ability to read business/technical specs and translate to pipeline logic.
• Familiarity with Git-based workflows, CI/CD, and infrastructure-as-code.

Responsibilities

• Build and operate production data pipelines on AWS (Glue, Spark, S3, Lambda, Step Functions).
• Own operational support: monitoring, incident management, and root cause analysis.
• Refactor transformation logic in PySpark and optimize large Spark jobs.
• Implement data reconciliation checks and verify parity between implementations.
• Translate complex business/financial logic into correct, well-documented code.
• Lead features through production readiness with runbooks and access control.
• Support cloud lakehouse migration and move reporting workloads to Delta/Databricks.

Skills

Python

SQL

PySpark/Spark

AWS data services

Data quality & reconciliation

Requirements to pipeline logic

Git CI/CD IaC

Tools

Databricks

Delta Lake

Job description

About Vanguard

Founded in 1975, Vanguard is one of the world's leading investment management companies. The firm offers investments, advice, and retirement services to tens of millions of individual investors around the globe—directly, through workplace plans, and through financial intermediaries.

Vanguard India

Vanguard’s office in India is a significant milestone in our global expansion. We are committed to establishing an enduring technology center in Hyderabad, Telangana and are excited to be adding talent who will focus on Artificial Intelligence (AI), mobile, and cloud-based technologies that drive our business outcomes and deliver a world-class experience for our clients.

Responsibilities

• Build and operateproduction data pipelineson AWS — Glue (PySpark), Step Functions, Lambda, Athena, SNS, and S3 — from ingestion through transformation to reconciled, reportable output.
• Own operational support for production data pipelines, including monitoring, incident management, PagerDuty response, root cause analysis, and driving timely resolution of production issues while continuously improving platform reliability and observability.
• Develop and refactortransformation logic in PySpark, including migrating legacy Pandas/SQL logic and performance-tuning large-scale Spark jobs.
• Implementdata reconciliation and sign-off— design recon checks (including parity between old and new implementations), investigate discrepancies to root cause, and produce the evidence the business needs to trust a run.
• Translatecomplex business and financial logic(asset classification, valuation and pricing rules, methodology calculations) into correct, testable, well-documented code.
• Take features throughproduction readiness— pre-checks, access/role setup, environment promotion, and clear runbooks for ongoing support.
• Contribute to ourcloud lakehouse migration— modelling refined tables, building repeatable ETL, and lifting reporting workloads onto a modern Delta/Databricks platform.
• Keep running pipelines healthy — monitor scheduled jobs, triage failures (partitioning, schema/data-type issues, backfills), and keep periodic (month-end / quarter-end) processing on time.

What We're Looking For

Must have

• 5+ years in data engineering with strongPythonandSQL.
• Hands-onPySpark / Sparkfor large-scale data transformation, including debugging and performance tuning.
• Production experience withAWS data services— Glue, Athena, S3, Lambda, Step Functions (or close equivalents on another major cloud).
• Soliddata modellingskills and a rigorous, reconciliation-first approach todata quality.
• Ability to read a business or technical specification and turn it into correct, maintainable pipeline logic.
• Comfortable withGit-based workflows, CI/CD, and infrastructure-as-code.

Nice to have

• Databricks/ lakehouse (Delta Lake) experience.
• Background infinancial services— custody, fund accounting, unit pricing, or regulatory/financial reporting.
• Experience withaccess governance and IAM(e.g. role provisioning) and writing operational runbooks.
• Track recordmigrating legacy reporting or ETL onto modern cloud data platforms.

Location

This role is based in Hyderabad, Telangana at Vanguard India. Only qualified external applicants will be considered.

Our mission

Vanguard adheres to a simple purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investment success.

Our commitment to you

Benefits

Vanguard takes the same long-term view of your success—at work and in life with Benefits and Rewards packages that reflect what you care about, throughout all the phases and stages of your life. Our Total Rewards programs provide you and your loved ones with wellness support for key areas in your life:

Financial wellness

We're committed to enabling your financial success and provide competitive offers and programs.

Physical wellness

We're committed to providing benefits that support your physical health and wellness.

Personal wellness

We're committed to providing resources that help support the full scope of your life.

How We Work

Vanguard has implemented a hybrid working model for most of our employees (crew members), designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.