Data Engineer

LPL Financial Global Capability Center; Hyderabad; On-site; INR 1,800,000 - 3,600,000; Full time

Job summary

LPL Financial Global Capability Center in Hyderabad seeks a Data Engineer to design scalable data pipelines on AWS, enabling data-driven decisions across the enterprise. You will work with Python, Spark, containers, and collaborate with architects, engineers, and business stakeholders.

You should have a BE in Computer Science and 4+ years in data engineering, with strong AWS skills and hands-on Spark, Glue, Lambda, and Kubernetes experience.

Qualifications

• BE in Computer Science or equivalent practical experience.
• 4+ years of data engineering or distributed systems experience.
• Strong hands-on AWS cloud services experience.
• Advanced proficiency in Python.
• Hands-on experience with Apache Spark and AWS services such as Glue, Lambda, Step Functions, EventBridge, S3, Athena, RDS Aurora PostgreSQL.
• Experience building event-driven data pipelines.

Responsibilities

• Design, develop, and maintain distributed data pipelines on AWS.
• Ingest, process, and deliver data at scale using batch and event-driven patterns.
• Develop event-driven data solutions for near real-time processing.
• Ensure end-to-end data quality, data lineage, and governance with monitoring.
• Build data services using Docker containers and manage workloads on Kubernetes.
• Collaborate with data consumers, architects, platform teams, and business stakeholders.
• Contribute to monitoring, logging, alerting, and incident response for data platforms.

Skills

Python programming

Distributed systems

Education

BE in Computer Science

Tools

AWS

Docker

Kubernetes

Apache Spark

AWS Glue

AWS Lambda

Step Functions

EventBridge

S3

Athena

RDS Aurora PostgreSQL

Job description

Job Description:

Where Ambition Meets Innovation

At LPL’s Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.

Join us as we create something extraordinary together.

Job Overview

We are looking for Data Engineer who will be part of the Data Ingest Batch Integration organization responsible for building scalable, secure, and high-performance data platforms that support data ingestion for up/downstream applications. This role is critical to enabling data-driven decision-making across the enterprise.

The Data Engineer requires strong hands-on cloud data engineering expertise, deep knowledge of distributed and event-driven architectures, and the ability to collaborate closely with application engineers, architects, and business stakeholders. This role plays a key part in designing, developing, and operating data pipelines that meet high standards for reliability, data quality, lineage, and governance.

The Data Engineer will contribute to building modern, cloud-native data solutions leveraging AWS, Python, Spark, APIs, and containers. They will also partner with platform, architecture, and DevOps teams to ensure consistency, scalability, and operational excellence across data pipelines.

Responsibilities

• Design, develop, and maintain distributed data pipelines on AWS that ingest, process, and deliver data at scale. Implement both batch and event-driven data processing patterns using AWS-native services.
• Build and support event-driven data solutions using asynchronous, decoupled architectures to enable near real-time processing and system scalability.
• Provide hands-on engineering using Python and AWS Glue with pyspark to process large datasets efficiently. Apply best practices in distributed systems, performance optimization, and fault tolerance.
• Ensure end-to-end data quality, implement validation and monitoring checks, establish data lineage, and manage orchestration workflows to ensure reliable and auditable data movement.
• Build and deploy data services using Docker containers and manage workloads on Kubernetes following cloud-native and DevOps best practices.
• Collaborate with data consumers, architects, platform teams, and business stakeholders to gather requirements, design scalable solutions, and deliver high-quality outcomes.
• Contribute to monitoring, logging, alerting, and incident response for data platforms. Ensure systems meet reliability, performance, and security standards.

What are we looking for?

We are looking for strong data engineers who can deliver reliable, scalable, and high-quality data solutions. The ideal candidate thrives in a fast-paced environment, is passionate about data engineering, and is comfortable working across teams to solve complex data problems.

Requirements

• B.E in Computer Science, Engineering, or equivalent practical experience
• 4+ years of experience in data engineering or distributed systems development
• Strong hands-on experience with AWS cloud services
• Advanced proficiency in Python
• Hands-on experience with Apache Spark and AWS AWS services such as Glue, Lambda, Step functions, Event Bridge, S3, Athena, RDS Aurora Postgresql
• Experience building event-driven data pipelines

Preferences

• Experience designing distributed data processing architectures
• Strong knowledge of data quality, data lineage, and orchestration
• Experience in API development using AWS Lambda, API Gateway and Python
• Hands-on experience with Docker and Kubernetes

Core Competencies

• Excellent verbal and written communication skills
• Strong analytical and problem-solving skills
• Ability to collaborate with cross-functional teams
• Strong ownership mindset and attention to detail
• Experience working in Agile environments
• Commitment to continuous learning

LPL Global Business Services, LLP - PRIVACY POLICY