Cloud & AI Data Engineer II: Build Scalable Pipelines

LPL Financial; Fort Mill (SC); On-site; USD 114,903,000 - 191,495,000; Full time

Job summary

LPL Financial is seeking an Engineer II, Data (Cloud & AI) to design, build, and operate cloud-native data solutions at enterprise scale. You will work across data ingestion, transformation, and validation pipelines on AWS, using Terraform IaC, and apply AI-assisted development tools to boost velocity while maintaining production reliability.

The role emphasizes scalable data pipelines, platform observability, and collaboration with product managers and engineers to improve data governance and

Qualifications

• Bachelor’s degree in Computer Science, Engineering or related field; 3+ years of software/cloud/data engineering OR Master’s with 1+ year relevant experience.
• Experience building, supporting, or maintaining cloud-based applications, data platforms, or production systems.
• Hands-on experience with AWS services including S3, Lambda, Glue, CloudWatch, IAM, EventBridge, and Athena
• Proficiency in Python and/or PySpark for data transformation, SQL, API integrations, and Git/GitHub
• Experience in ETL/ELT pipelines, data validation, data quality practices, and batch/event-driven processing

Responsibilities

• Design, develop, and maintain cloud-based data ingestion and transformation pipelines.
• Onboard new vendor and enterprise data sources.
• Optimize processing performance for large data volumes.
• Build reusable ingestion, validation, and transformation frameworks.
• Develop automated data quality validation processes.
• Develop AWS-based solutions and Terraform infrastructure.
• Support CI/CD deployment pipelines and observability.
• Explore AI-assisted development tools to improve velocity and efficiency.
• Participate in incident response and production support.
• Collaborate with product managers, architects, analysts, and business stakeholders.

Skills

Python

PySpark

SQL

Git/GitHub

Data engineering

Education

Bachelor's degree in CS/Engineering

Master's degree in CS/Data AI/IS

Tools

AWS services

Terraform

CI/CD pipelines

OpenAI APIs

Bedrock

Vector databases

Job description

LPL Financial is seeking an Engineer II, Data (Cloud & AI) to design, build, and operate cloud-native data solutions at enterprise scale. You will work across data ingestion, transformation, and validation pipelines on AWS, using Terraform IaC, and apply AI-assisted development tools to boost velocity while maintaining production reliability.

The role emphasizes scalable data pipelines, platform observability, and collaboration with product managers and engineers to improve data governance and