Senior Engineer Batch ETL Big Data And Workflow Orchestration

LPL Financial Global Capability Center; Telangana; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

LPL Financial Global Capability Center in Hyderabad is seeking a skilled software developer to help build enterprise-scale batch ETL pipelines within Account Lifecycle Management. You will work with Informatica, PySpark, and Airflow in a fast-paced team to deliver compliant data solutions.

The role emphasizes performance, reliability, and collaboration with architecture and business stakeholders to translate requirements into robust data processing capabilities.

Qualifications

• Hands-on experience with batch ETL architectures in regulated environments.
• Advanced expertise in Informatica.
• Strong proficiency in PySpark and distributed data processing.
• Hands-on experience with Apache Airflow for DAGs, scheduling and monitoring.
• Experience processing large datasets with focus on performance and reliability.
• Strong SQL and data modeling fundamentals.
• Ability to own complex problem areas end-to-end with minimal supervision.

Responsibilities

• Design, develop, and optimize enterprise-scale batch ETL pipelines for regulatory and analytics platforms.
• Lead ETL development using Informatica and PySpark for high-volume data transformations.
• Build and maintain Airflow DAGs to orchestrate batch workflows, dependencies, retries, SLAs, and monitoring.
• Design scalable solutions for large data processing on distributed and cloud-based data platforms.
• Drive performance, resiliency, data quality, and operational stability of batch processing jobs.
• Collaborate with architecture, upstream/downstream teams, and stakeholders to translate requirements into robust data solutions.
• Enforce data security, governance, auditability, and operational readiness.
• Perform code reviews and mentor junior engineers to raise engineering maturity.

Skills

ETL architectures

Informatica

PySpark

Apache Airflow

SQL

Data modeling

End-to-end ownership

Job description

Job Description

Where Ambition Meets Innovation

At LPL’s Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.

Join us as we create something extraordinary together.

We are seeking a skilled software developer to help with the development of projects within the Account Lifecycle Management domain, that span different technologies and requirements. You will be working with a motivated team in a fast‑paced environment.

A software developer's day involves attending a morning stand‑up meeting, reviewing code, working on complex coding tasks, collaborating with team members, and participating in design and architecture discussions. They also attend meetings like sprint planning and retrospectives, monitor project progress, deploy code, and mentor junior developers. By the end of the day, they review their progress, update documentation, and prepare for the next day's tasks.

Responsibilities

• Design, develop, and optimize enterprise‑scale batch ETL pipelines supporting LPL’s regulatory, supervision, and analytics platforms.
• Lead ETL development using Informatica and PySpark for high‑volume and complex data transformations.
• Build and maintain Airflow DAGs to orchestrate batch workflows, dependencies, retries, SLAs, and monitoring.
• Design scalable solutions for large data processing on distributed and cloud‑based data platforms.
• Drive performance tuning, resiliency, data quality, and operational stability of batch processing jobs.
• Partner with architecture, upstream/downstream system teams, and business stakeholders to translate requirements into robust, compliant data solutions.
• Enforce LPL standards for security, data governance, auditability, and operational readiness.
• Perform code reviews, mentor Engineer I/II resources, and raise overall engineering maturity of the team.

What are we looking for?

We’re looking for strong collaborators who deliver exceptional client experiences and thrive in fast‑paced, team‑oriented environments. Our ideal candidates pursue greatness, act with integrity, and are driven to help our clients succeed. We value those who embrace creativity, continuous improvement, and contribute to a culture where we win together and create and share joy in our work.

Requirements

• Strong hands‑on experience with batch ETL architectures in regulated environments.
• Advanced expertise in Informatica.
• Strong proficiency in PySpark and distributed data processing concepts.
• Hands‑on experience with Apache Airflow (authoring DAGs, scheduling, dependency management, monitoring).
• Experience processing large datasets with a focus on performance and reliability.
• Strong SQL and data modeling fundamentals.
• Ability to own complex problem areas end‑to‑end and operate with minimal supervision.

LPL Global Business Services, LLP - PRIVACY POLICY

Requirements