API & Data Engineer II

lplfinancial; Fort Mill (SC); On-site; USD 55,000 - 92,000; Full time

Benefits offered by this job

401K matching

Health benefits

Paid time off

Job summary

LPL Financial is seeking a Data Engineer II to design and maintain scalable data pipelines and services. You will build Python-based applications, deploy on AWS EKS, and implement API/microservices patterns while ensuring security and performance.

Responsibilities include developing with PySpark, integrating DynamoDB and PostgreSQL, and collaborating in an Agile environment with clear documentation and CI/CD workflows.

Qualifications

• Proficiency in Python and related frameworks.
• Strong experience with AWS services and data tools.
• Hands-on with PySpark, relational and NoSQL databases.

Responsibilities

• Design, develop, and maintain Python-based applications and services.
• Build data pipelines using AWS Glue and PySpark.
• Deploy containerized apps on AWS EKS and implement microservices patterns.
• Integrate AWS data stores (DynamoDB, PostgreSQL) and Kafka messaging.
• Write clean, testable code with TDD in an Agile Scrum setting.
• Document designs in Confluence and manage tasks in JIRA.

Skills

Python

AWS services

PySpark

API Gateway

AWS Lambda

EKS

DynamoDB

PostgreSQL

Kafka

GitHub CI/CD

Terraform

OOP / design patterns

Agile / Scrum

RESTful services

Testing (TDD)

Tools

GitHub

CI/CD

Jira

Confluence

Docker

Kubernetes / EKS

Terraform

Job description

Where Ambition Meets Innovation

Build a career that matches all your initiative with an impressive dose of innovation. From cutting-edge resources and a collaborative environment to the freedom to make an impact and more, you'll find the ingredients you need at LPL Financial to shape your success while helping clients pursue their financial goals.

Job Overview:

At LPL Financial we consider it our mission to take care of our advisors so they can take care of their client. Joining as a Data Engineer II, you will be responsible for building and maintaining the sustainable, scalable capabilities that support delivery of the highest standards of software products, processes for our Advisors and End Clients.

Responsibilities:

• Design, develop, and maintain Python-based applications and services.
• Build and optimize data pipelines using AWS Glue and PySpark.
• Develop and deploy containerized applications on AWS EKS.
• Implement API and Microservices patterns using API Gateway and AWS Lambda.
• Integrate with AWS DynamoDB, PostgreSQL, and other data stores.
• Implement messaging and streaming solutions using Kafka.
• Write clean, maintainable, and testable code following Test Driven Development (TDD) principles.
• Collaborate in an Agile Scrum environment, participating in sprint planning, daily stand-ups, and retrospectives.
• Document technical designs and processes in Confluence and manage tasks in JIRA.
• Work with GitHub for version control and CI/CD pipelines.
• Apply basic Terraform skills for infrastructure as code.
• Ensure application security, scalability, and performance.
• Communicate effectively with stakeholders and team members, providing clear documentation and status updates.

What are we looking for?

We want strong collaborators who can deliver a world-class client experience . We are looking for people who thrive in a fast-paced environment, are client-focused, team oriented , and are able to execute in a way that encourages creativity and continuous improvement .

Requirements:

• Minimum of 3 years of Proficiency in Python and related frameworks (e.g., Flask, FastAPI, Django).
• Minimum of 2 Strong experience with AWS services: Lambda, API Gateway, Glue, EKS, DynamoDB, S3.
• Experience with PySpark for data processing.
• Solid understanding of relational and NoSQL databases (PostgreSQL, DynamoDB).
• Hands-on experience with GitHub and CI/CD workflows.

Core Competencies:

• Excellent knowledge of OOP concepts, data modeling design patterns/frameworks using Search, Index, and Distribute Caching tools such as Coherence, Apache Solr, etc.
• Experience with Agile development process
• Excellent knowledge in designing and developing micro services
• Ability to lead design discussions with Architecture Team
• Good analytical, problem solving skills, communication, and documentation skills
• Ability to handle multiple projects and with strong interpersonal skills and ability to work in a collaborative team environment
• Great track record of implementing the distributed systems with legacy and modern technical tools
• Problem-solving mindset with attention to detail.
• Ability to adapt to changing priorities and technologies.
• Strong collaboration and proactive communication.
• Eagerness to learn new technologies and continuously improve skills.

Preferences:

• AWS Associate-level certifications (e.g., AWS Certified Developer - Associate, AWS Certified Solutions Architect - Associate).
• Additional AWS or cloud-related certifications are a plus.
• Experience with Kong API Gateway.
• Familiarity with GitHub Copilot and other AI-assisted development tools.
• Basic knowledge of Terraform for infrastructure automation.
• Experience with Agile Scrum methodology and tools like JIRA.
• Familiarity with Kafka for event-driven architectures.
• Strong understanding of TDD and unit testing frameworks (pytest, unittest).

Pay Range:

$40.10-$66.83/hour

Actual base salary varies based on factors, including but not limited to, relevant skill, prior experience, education, base salary of internal peers, demonstrated performance, and geographic location. Additionally, LPL Total Rewards package is highly competitive, designed to support your success at work, at home, and at play - such as 401K matching, health benefits, employee stock options, paid time off, volunteer time off, and more.

Company Overview:

LPL Financial Holdings Inc. (Nasdaq: LPLA) is among the fastest growing wealth management firms in the U.S. As a leader in the financial advisor-mediated marketplace(6) , LPL supports over 32,000 financial advisors and the wealth management practices of approximately 1,100 financial institutions, servicing and custodying approximately $2.3 trillion in brokerage and advisory assets on behalf of approximately 8 million Americans. The firm provides a wide range of advisor affiliation models, investment solutions, fintech tools and practice management services, ensuring that advisors and institutions have the