Senior Python Data Engineer

EPAM Systems, Inc.; Wrocław; Hybrid; PLN 180,000 - 320,000; Full time

Job summary

EPAM Systems, Inc. in Poland is seeking a hands-on Senior Python Data Engineer to join the Regulatory Reporting team in Wroclaw. The role focuses on building data-driven applications and scalable data pipelines to support regulatory and data clients.

You will design data models, collaborate across the SDLC, and work with product owners and technologists. 5+ years in data engineering with Databricks, Python, Spark and cloud experience is expected.

Qualifications

• Bachelor's degree in Computer Science or equivalent in IT field.
• 5+ years of experience in a similar data engineering role.
• Hands-on in building scalable data pipelines for batch and streaming data.

Responsibilities

• Design, implement and maintain data-driven applications.
• Develop and deliver end-to-end data solutions.
• Engineer reliable data pipelines for sourcing, processing, distributing and storing data.
• Design data models to support growing data volumes.
• Contribute to all stages of the software development lifecycle.
• Provide technical expertise and innovative solutions.
• Collaborate with product owner and other technologists.
• Share expertise and best practices with the team.

Skills

Databricks

Python

Spark

Big Data

Data pipelines

SQL

PostgreSQL

Azure

Unix/Linux

Shell scripting

CI/CD

GitLab

Delta Lake

DLT

AI in SDLC

English (B2)

Education

Bachelor's degree in Computer Science or equivalent in IT

Tools

Azure SQL

PostgreSQL

SQL databases

Job description

We are seeking a hands-on Senior Python Data Engineer to join the Regulatory Reporting team in Poland within the Investment Bank's Digital Operations Stream, working to transform how the client does business by simplifying and improving the products and platforms relied on by Regulatory and Data Clients. Please note that working from the client's office is required 3 days per week.

Responsibilities

• Design, implement and maintain new and existing data-driven applications
• Develop and deliver solutions end to end
• Engineer reliable data pipelines for sourcing, processing, distributing and storing data
• Design data models that can support ever-growing volumes
• Contribute to all stages of the software development lifecycle
• Provide technical expertise, recommendations and innovative solutions
• Collaborate with the product owner and other technologists
• Share expertise and best practice with colleagues and contribute regularly to the engineering culture

Requirements

• Hold a Bachelor's degree in Computer Science or equivalent in the IT field with 5+ years of experience in a similar role
• Background in Big Data, Data warehouse and Data lake
• Showcase of hands-on experience building scalable data pipelines, processing high volume data in batch and stream for near real-time analytics
• Expertise in Databricks, Python and Spark
• Familiarity with Delta Lake, DLT and GitLab
• Knowledge of Cloud technologies such as Azure, Unix/Linux and shell scripting
• Expertise designing applications considering NFRs around high throughput and low latency for data processing
• Expertise building automated functional testing and CI/CD pipelines using industry standard tools
• Capability to quickly implement prototypes and iterate to harden the product with good engineering principles
• Hands-on usage of AI in engineering SDLC
• English proficiency at B2 level or higher
• Nice to have Proficiency in database development including SQL, PL/SQL, PostgreSQL and Azure SQL