Senior Data Engineer: Real-Time Pipelines on AWS (Python)

EPAM Systems Inc; Polska; On-site; PLN 180,000 - 240,000; Full time

Job summary

EPAM Systems Inc is seeking a Senior Data Engineer in Poland to join an established ETF analytics platform. You will design, build, and optimize Python-based data pipelines handling real-time and historical market data, leveraging AWS S3, ECS, EC2, Lambda, and workflow services.

You will work in a small agile team, participate in client coordination, and produce comprehensive documentation while providing L3 production support and exploring new technologies to improve performance and reliability.

Qualifications

• 3+ years of experience in Data Software Engineering.
• Proficient in Python development for data engineering environments.
• Experience designing data lakehouse architectures and real-time streaming pipelines.
• Experience integrating AWS services including S3, ECS, EC2.

Responsibilities

• Design, develop, and optimize Python-based data pipelines to process streaming market data with minimal latency.
• Integrate AWS cloud services to support ingestion, processing, storage, and delivery of real-time and historical data.
• Collaborate within a small agile team of 3 to 5 engineers, enabling rapid decision-making.
• Participate in coordination meetings with client managers and internal teams to align deliverables.
• Produce and maintain technical documentation for system understanding and knowledge transfer.
• Provide L3 production support, root cause analysis, and corrective actions.
• Evaluate and adopt emerging technologies to enhance pipeline performance and reliability.

Skills

Python

Data pipelines

AWS

Lakehouse

Streaming data

S3

ECS

EC2

Lambda

Step Functions

EventBridge

CloudWatch

DynamoDB

NoSQL

Production support

Finance domain

Job description

EPAM Systems Inc is seeking a Senior Data Engineer in Poland to join an established ETF analytics platform. You will design, build, and optimize Python-based data pipelines handling real-time and historical market data, leveraging AWS S3, ECS, EC2, Lambda, and workflow services.

You will work in a small agile team, participate in client coordination, and produce comprehensive documentation while providing L3 production support and exploring new technologies to improve performance and reliability.