Senior Data Engineer: AI-Powered Data Pipelines

Luxoft; Województwo małopolskie; On-site; PLN 180,000 - 240,000; Full time

Job summary

DXC Luxoft, Poland, is seeking an experienced data engineer to design, implement, and maintain data-driven applications in aRegulatory Reporting team. You will build scalable pipelines, craft data models, and contribute to all phases of the software lifecycle.

The role requires 10+ years in IT, expert SQL/Python/Databricks skills, and cloud, Big Data, and CI/CD experience. Collaboration with cross-country teams and a focus on high throughput/low latency define this position.

Qualifications

• Bachelor's degree in Computer Science or equivalent in the IT field.
• 10+ years of experience in a similar role.
• Hands-on experience with SQL, Databricks, Python (advanced).
• Hands-on experience building scalable data pipelines, processing high-volume data in batch as well as stream for near real-time analytics.
• Hands-on usage of AI in engineering SDLC.
• Experience with Big Data, Data Warehouse, and Data Lake.
• Experience with Cloud technologies (e.g., Azure), Unix/Linux, and shell scripting.
• Experience with Spark, Delta Lake, DLT, GitLab.
• Extensive experience designing applications, specifically considering the NFRs around high throughput and low latency for data processing.
• Expertise in building automated functional testing and CI/CD pipelines using industry-standard tools.
• Ability to quickly implement prototypes and iterate to harden the product with good engineering principles

Responsibilities

• design, implement, and maintain new and existing data-driven applications
• ability to develop and see through solutions end-to-end.
• engineer reliable data pipelines for sourcing, processing, distributing, and storing data
• design data models that can support ever-growing volumes
• contribute to all stages of the software development lifecycle
• provide technical expertise, recommendations, and innovative solutions
• work collaboratively alongside the product owner and other technologists
• share expertise and best practices with colleagues and contribute regularly to our engineering culture

Skills

SQL

Databricks

Python

Data pipelines

Big Data

Data Warehouse

Data Lake

Azure Cloud

Unix/Linux

Shell scripting

Spark

Delta Lake

DLT

GitLab

CI/CD

Automated testing

Education

Bachelor's degree in Computer Science

Tools

Databricks

GitLab

Spark

Delta Lake

DLT

Job description

DXC Luxoft, Poland, is seeking an experienced data engineer to design, implement, and maintain data-driven applications in aRegulatory Reporting team. You will build scalable pipelines, craft data models, and contribute to all phases of the software lifecycle.

The role requires 10+ years in IT, expert SQL/Python/Databricks skills, and cloud, Big Data, and CI/CD experience. Collaboration with cross-country teams and a focus on high throughput/low latency define this position.