AWS And Airflow Developer

LPL Financial Global Capability Center; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

LPL Financial Global Capability Center in Hyderabad is seeking an AWS & Airflow Developer with 4-6 years of data engineering experience. You will design, build and support cloud-based data pipelines using Python, Apache Airflow and AWS services to enable analytics and reporting.

You'll work with a cross-functional Data Engineering team to deliver scalable, secure data workflows, participate in CI/CD and cloud modernization initiatives, and help drive best practices across the stack.

Qualifications

• Bachelor's degree in CS/IT/Eng or related field.
• 4-6 years in Data Engineering, ETL Development or Data Integration.
• Strong Python programming skills.
• Hands-on Apache Airflow for orchestration and scheduling.
• Experience with AWS services including S3, Glue, Lambda, Step Functions, Athena, EventBridge, RDS/Aurora PostgreSQL, CloudWatch.
• Strong SQL skills with PostgreSQL, Oracle or SQL Server.
• Experience with batch and event-driven data pipelines.
• Understanding of data modeling, warehousing and ETL/ELT concepts.
• Experience with Git for source control.
• Knowledge of monitoring, logging and debugging distributed workflows.
• Strong analytical, problem-solving and communication skills.

Responsibilities

• Develop data transformation logic using Python, SQL and PySpark where applicable.
• Design, develop and maintain scalable data pipelines using Airflow and AWS services.
• Build and optimize ETL/ELT workflows to ingest, transform and process data from various sources.
• Develop and manage Airflow DAGs, dependencies, scheduling, monitoring and error handling.
• Implement data ingestion using AWS such as S3, Glue, Lambda, Step Functions, Athena, RDS/Aurora.
• Monitor pipeline performance and troubleshoot workflow failures and data quality issues.
• Collaborate with architects and analysts to understand data requirements.
• Support production deployments, incident management and RCA.
• Participate in cloud modernization and migrations to AWS.
• Contribute to CI/CD, documentation and engineering best practices.
• Develop and maintain dashboards, monitoring and alerting for observability.
• Participate in Agile ceremonies and technical design discussions.

Skills

Python

Airflow

AWS

SQL

ETL/ELT

Education

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field

Tools

PySpark

Docker

Kubernetes

Job description

Job Description:

Where Ambition Meets Innovation

At LPL's Global Capability Center, you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We're proud to be expanding and reaching new heights in Hyderabad.

Join us as we create something extraordinary together.

Job Overview

We are seeking a highly motivated AWS & Airflow Developer with 4-6 years of experience in designing, developing and supporting cloud-based data pipelines, workflow orchestration and data integration solutions. The ideal candidate will have hands-on expertise in Python, Apache Airflow, AWS cloud services, SQL and ETL/ELT development, with a strong understanding of data engineering best practices.

This role will be part of the Data Engineering team responsible for building scalable, secure and reliable data pipelines that support enterprise analytics, reporting and downstream applications.

Responsibilities

• Develop data transformation logic using Python, SQL and PySpark where applicable.
• Design, develop and maintain scalable and reliable data pipelines using Apache Airflow and AWS services.
• Build and optimize ETL/ELT workflows to ingest, transform and process data from various source systems.
• Develop and manage Airflow DAGs, task dependencies, scheduling, monitoring and error handling.
• Implement data ingestion solutions using AWS services such as S3, Glue, Lambda, Step Functions, EventBridge, Athena and RDS/Aurora.
• Monitor data pipeline performance and troubleshoot workflow failures, data quality issues and operational incidents.
• Collaborate with architects, business analysts and application teams to understand data requirements and deliver scalable solutions.
• Support production deployments, incident management, root cause analysis and performance tuning activities.
• Implement data validation, reconciliation and quality checks to ensure accurate and reliable data delivery.
• Participate in cloud modernization and migration initiatives to AWS-based platforms.
• Contribute to CI/CD implementation, automation, documentation and engineering best practices.
• Develop and maintain operational dashboards, monitoring and alerting solutions for pipeline observability.
• Participate in Agile ceremonies, sprint planning and technical design discussions.

What are we looking for?

We are looking for strong AWS & Airflow Developer who can deliver reliable, scalable, and high-quality data solutions. The ideal candidate thrives in a fast-paced environment, is passionate about data engineering, and is comfortable working across teams to solve complex data problems.

Requirements

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4-6 years of experience in Data Engineering, ETL Development, or Data Integration.
• Strong proficiency in Python programming.
• Strong hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Experience with AWS cloud services, including:
• Amazon S3, Glue, Lambda, Step Functions, Athena, EventBridge, RDS/Aurora PostgreSQL, CloudWatch
• Strong SQL skills with experience in relational databases such as PostgreSQL, Oracle, or SQL Server.
• Experience developing and supporting batch and event-driven data pipelines.
• Understanding of data modeling, data warehousing and ETL/ELT concepts.
• Experience with source control systems such as Git.
• Knowledge of monitoring, logging and troubleshooting distributed data workflows.
• Strong analytical, problem-solving and debugging skills.
• Excellent verbal and written communication skills.

Preferences

• Experience with PySpark and distributed data processing frameworks.
• Knowledge of AWS EMR, Redshift, or Snowflake.
• Experience working with Docker and containerized applications.
• Exposure to Kubernetes or Amazon EKS.
• Familiarity with REST API integration and event-driven architectures.
• Experience implementing CI/CD pipelines using GitHub Actions, GitLab CI, or AWS CodePipeline.
• Understanding of data governance, metadata management and data lineage concepts.
• Experience working in Agile/Scrum environments.
• Exposure to infrastructure-as-code tools such as Terraform or AWS CloudFormation.

Core Competencies

• Strong ownership and accountability.
• Excellent analytical and troubleshooting skills.
• Ability to work independently and within cross-functional teams.
• Strong attention to detail and commitment to quality.
• Effective communication and stakeholder management skills.
• Continuous learning mindset and adaptability to emerging technologies.
• Ability to manage multiple priorities in a fast-paced environment.

LPL Global Business Services, LLP - PRIVACY POLICY