Sr Engineer

LPL Financial LLC; Hyderabad; On-site; INR 4,200,000 - 6,600,000; Full time

Job summary

LPL Financial LLC in Hyderabad is seeking an accomplished Senior Data Engineer with 8+ years of experience to join the Data & Platform Engineering organization. You will build scalable, secure data pipelines on AWS, using Python and Spark, and collaborate with architects, engineers, and stakeholders to deliver high-quality data platforms.

Responsibilities include designing event-driven architectures, ensuring data quality and lineage, and delivering APIs and containerized services with Docker

Qualifications

• BS in Computer Science, Engineering, or equivalent practical experience.
• 8+ years of experience in data engineering or distributed systems development.
• Strong hands-on experience with AWS cloud services.
• Advanced proficiency in Python.
• Hands-on experience with Apache Spark.
• Experience building event-driven data pipelines.
• Experience designing distributed data processing architectures.
• Strong knowledge of data quality, data lineage, and orchestration.
• Experience in API development.
• Hands-on experience with Docker and Kubernetes.

Responsibilities

• Design, develop, and maintain distributed data pipelines on AWS that ingest, process, and deliver data at scale.
• Implement both batch and event-driven data processing patterns using AWS-native services.
• Build and support event-driven data solutions using asynchronous, decoupled architectures to enable near real-time processing and system scalability.
• Provide hands-on engineering using Python and Apache Spark to process large datasets efficiently.
• Ensure end-to-end data quality, implement validation and monitoring checks, establish data lineage, and manage orchestration workflows.
• Design and develop RESTful and event-driven APIs to expose data and data services to internal and external consumers.
• Build and deploy data services using Docker containers and manage workloads on Kubernetes following cloud-native and DevOps best practices.
• Collaborate with data consumers, architects, platform teams, and business stakeholders to gather requirements, design scalable solutions, and deliver high-quality outcomes.
• Contribute to monitoring, logging, alerting, and incident response for data platforms.

Skills

Python

Apache Spark

AWS

Data pipelines

REST APIs

Event-driven

Docker

Kubernetes

CI/CD

Agile

Education

Bachelor's degree in Computer Science or related field

Tools

Docker

Kubernetes

AWS

Job description

Where Ambition Meets InnovationAt LPL’s Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.Join us as we create something extraordinary together.Job Overview:We are seeking an accomplished Senior Data Engineer with 8+ years of experience who will be part of the Data & Platform Engineering organization responsible for building scalable, secure, and high-performance data platforms that support analytics, reporting, and downstream applications. This role is critical to enabling data-driven decision-making across the enterprise.The Senior AWS Data Engineer requires strong hands-on cloud data engineering expertise, deep knowledge of distributed and event-driven architectures, and the ability to collaborate closely with application engineers, architects, and business stakeholders. This role plays a key part in designing, developing, and operating data pipelines that meet high standards for reliability, data quality, lineage, and governance.The Senior AWS Data Engineer will contribute to building modern, cloud-native data solutions leveraging AWS, Python, Spark, APIs, and containers. They will also partner with platform, architecture, and DevOps teams to ensure consistency, scalability, and operational excellence across data pipelines.Responsibilities:Data Pipeline Development:Design, develop, and maintain distributed data pipelines on AWS that ingest, process, and deliver data at scale. Implement both batch and event-driven data processing patterns using AWS-native services.Event-Driven Architecture:Build and support event-driven data solutions using asynchronous, decoupled architectures to enable near real-time processing and system scalability.Technical Expertise:Provide hands-on engineering using Python and Apache Spark to process large datasets efficiently. Apply best practices in distributed systems, performance optimization, and fault tolerance.Data Quality, Lineage & Orchestration:Ensure end-to-end data quality, implement validation and monitoring checks, establish data lineage, and manage orchestration workflows to ensure reliable and auditable data movement.API Development:Design and develop RESTful and event-driven APIs to expose data and data services to internal and external consumers.Containers & Cloud-Native Development:Build and deploy data services using Docker containers and manage workloads on Kubernetes following cloud-native and DevOps best practices.Stakeholder Collaboration:Collaborate with data consumers, architects, platform teams, and business stakeholders to gather requirements, design scalable solutions, and deliver high-quality outcomes.Operational Excellence:Contribute to monitoring, logging, alerting, and incident response for data platforms. Ensure systems meet reliability, performance, and security standards.What are we looking for?We are looking for strong data engineers who can deliver reliable, scalable, and high-quality data solutions. The ideal candidate thrives in a fast-paced environment, is passionate about data engineering, and is comfortable working across teams to solve complex data problems.Requirements:BS in Computer Science, Engineering, or equivalent practical experience8+ years of experience in data engineering or distributed systems developmentStrong hands-on experience with AWS cloud servicesAdvanced proficiency in PythonHands-on experience with Apache SparkExperience building event-driven data pipelinesExperience designing distributed data processing architecturesStrong knowledge of data quality, data lineage, and orchestrationExperience in API developmentHands-on experience with Docker and KubernetesPreferences:Experience with CI/CD pipelines and DevOps practices.Familiarity with backend services, microservices, or serverless architectures on AWS.Experience working in Agile delivery teams.Knowledge of observability, monitoring, and production support practices.Core Competencies:Excellent verbal and written communication skillsStrong analytical and problem-solving skillsAbility to collaborate with cross-functional teamsStrong ownership mindset and attention to detailExperience working in Agile environmentsCommitment to continuous learningLPL Global Business Services, LLP - PRIVACY POLICY