Data Engineer II — Python, AWS & Microservices

LPL Financial LLC; Charlotte, Fort Mill (NC, SC); On-site; USD 55,000 - 92,000; Full time

Benefits offered by this job

401K matching

Health benefits

Employee stock options

Paid time off

Volunteer time off

Job summary

LPL Financial is seeking a Data Engineer II to build scalable data capabilities supporting advisors and clients. You will design Python-based services, optimize pipelines with AWS Glue and PySpark, and deploy containerized apps on AWS EKS. Collaboration in Agile environments and strong coding practices are expected.

The role emphasizes API/microservices, database integration (DynamoDB, PostgreSQL), Kafka messaging, and Terraform for IaC. Strong communication and documentation skills are required.

Qualifications

• Minimum of 3 years of proficiency in Python and related frameworks (e.g., Flask, FastAPI, Django).
• Minimum of 2 strong experience with AWS services: Lambda, API Gateway, Glue, EKS, DynamoDB, S3.
• Experience with PySpark for data processing.
• Solid understanding of relational and NoSQL databases (PostgreSQL, DynamoDB).
• Hands-on experience with GitHub and CI/CD workflows.

Responsibilities

• Design, develop, and maintain Python-based applications and services.
• Build and optimize data pipelines using AWS Glue and PySpark.
• Develop and deploy containerized applications on AWS EKS.
• Implement API and Microservices patterns using API Gateway and AWS Lambda.
• Integrate with AWS DynamoDB, PostgreSQL, and other data stores.
• Implement messaging and streaming solutions using Kafka.
• Write clean, maintainable, and testable code following Test Driven Development (TDD) principles.
• Collaborate in an Agile Scrum environment, participating in sprint planning, daily stand-ups, and retrospectives.
• Document technical designs and processes in Confluence and manage tasks in JIRA.
• Work with GitHub for version control and CI/CD pipelines.
• Apply basic Terraform skills for infrastructure as code.
• Ensure application security, scalability, and performance.
• Communicate effectively with stakeholders and team members, providing clear documentation and status updates.

Skills

Python

PySpark

OOP / Data Modeling

Agile

Microservices

Communication

Tools

AWS

GitHub

Terraform

Kafka

DynamoDB / PostgreSQL

CI/CD

Job description

LPL Financial is seeking a Data Engineer II to build scalable data capabilities supporting advisors and clients. You will design Python-based services, optimize pipelines with AWS Glue and PySpark, and deploy containerized apps on AWS EKS. Collaboration in Agile environments and strong coding practices are expected.

The role emphasizes API/microservices, database integration (DynamoDB, PostgreSQL), Kafka messaging, and Terraform for IaC. Strong communication and documentation skills are required.