Lead Data Platform Engineer: AI & Streaming

GFT Technologies SE; Kraków; Hybrid; PLN 214,000 - 282,000; Full time

Job summary

GFT Technologies SE in Kraków is seeking an accomplished Senior Data Engineer for a B2B contract to design and build scalable data platforms and lead engineering initiatives in a highly regulated environment.

You will drive data pipelines, work with Databricks, Delta Lake, and Delta Live Tables, and implement CI/CD practices while collaborating with cross-functional teams to deliver impactful data products.

Qualifications

• Master's degree in Computer Science or a related IT field.
• 12+ years of experience in a similar role.
• Excellent leadership and team collaboration skills.
• Hands-on experience using AI in the software development lifecycle.
• 5+ years of experience with Databricks
• Strong expertise in Python
• Strong expertise in Apache Spark
• Experience with Delta Lake and Delta Live Tables (DLT)
• Experience with GitLab
• Experience with Big Data solutions
• Experience with Data Warehouse and Data Lake architectures
• Hands-on experience building scalable batch and streaming data pipelines
• Experience processing high-volume data for near real-time analytics
• Strong cloud experience, preferably Azure
• Experience with Unix/Linux environments
• Shell scripting skills
• Experience designing high-performance applications with strict non-functional requirements
• Experience building automated testing frameworks and CI/CD pipelines
• Ability to rapidly prototype and harden solutions using sound engineering principles
• Openness to work hybrid from our client's office (Kraków) 3 days a week

Responsibilities

• Design and build scalable data platforms and data processing solutions
• Build and optimize batch and streaming data pipelines for high-volume data processing
• Lead technical design activities with a focus on performance, scalability, and reliability
• Develop and maintain Databricks-based data solutions
• Implement CI/CD pipelines and engineering automation practices
• Drive adoption of AI-powered engineering practices across the software development lifecycle
• Collaborate with cross-functional teams to deliver business-critical data products
• Create prototypes and evolve them into production-ready solutions
• Ensure compliance with non-functional requirements including high throughput and low latency
• Support application modernization and data integration initiatives
• Mentor engineers and promote engineering best practices
• Participate in knowledge transfer and handover activities

Skills

Python

Apache Spark

Databricks

Delta Lake

Delta Live Tables

GitLab

Big Data

Data Warehouse

Data Lake

Azure

Unix/Linux

Shell scripting

CI/CD pipelines

AI in SDLC

Education

Master's degree in Computer Science or related IT field

Tools

GitLab

Azure SQL

PostgreSQL

SQL

PL/SQL

Job description

GFT Technologies SE in Kraków is seeking an accomplished Senior Data Engineer for a B2B contract to design and build scalable data platforms and lead engineering initiatives in a highly regulated environment.

You will drive data pipelines, work with Databricks, Delta Lake, and Delta Live Tables, and implement CI/CD practices while collaborating with cross-functional teams to deliver impactful data products.