Data Platform Engineer - Databricks on AWS

Grid Dynamics; Poznań; On-site; PLN 180,000 - 280,000; Full time

Benefits offered by this job

Bleeding-edge projects

Motivated team

Competitive salary

Flexible schedule

Medical insurance

Sports

Development opportunities

Well-equipped office

Job summary

Grid Dynamics is seeking a Data Engineer (Databricks / Spark) to join a lean, engineering-focused team building a new data platform for a global payments technology company. You will collaborate with the Tech Lead and client engineers to deploy Databricks on AWS pipelines and own automated tests and data quality checks.

You will also develop real-time analytics with ClickHouse, contribute to platform patterns, and support a legacy Hadoop setup, with opportunities to grow into AI/ML and MLOps.

Qualifications

• Commercial experience in data engineering, building production data pipelines and transformations.
• Databricks: Hands-on experience building pipelines and workloads on Databricks.
• Spark / PySpark: Strong practical skills in developing and optimizing Spark workloads.
• Python & SQL: Strong programming skills in Python and SQL.
• Quality Mindset: Ability to own automated testing and data quality validation as part of everyday delivery.

Responsibilities

• New Platform Development: Build pipelines and data transformations on Databricks on AWS, developing Bronze, Silver and Gold layers as part of a platform that unifies the engineering approach across many consumers and use cases.
• Spark Performance: Develop and optimize Spark / PySpark workloads, including those behind monthly reports for around 40,000 merchants.
• Real-Time Analytics: Build ClickHouse ingestion and analytical structures for low-latency use cases.
• Engineering Quality Ownership: Implement automated tests, integration tests and data quality checks yourself, with no dedicated QA on the team, and take part in code reviews.
• Legacy Platform Support: Troubleshoot production issues and deliver targeted fixes in a legacy Hadoop platform with file-based sources and Kafka streams (around 10% of your time).
• Reusable Patterns: Contribute to shared engineering patterns that become the standard for the new platform, with potential growth into AI/ML and MLOps.

Skills

Data engineering experience

Databricks

Spark PySpark

Python & SQL

Quality mindset

Job description

Grid Dynamics is seeking a Data Engineer (Databricks / Spark) to join a lean, engineering-focused team building a new data platform for a global payments technology company. You will collaborate with the Tech Lead and client engineers to deploy Databricks on AWS pipelines and own automated tests and data quality checks.

You will also develop real-time analytics with ClickHouse, contribute to platform patterns, and support a legacy Hadoop setup, with opportunities to grow into AI/ML and MLOps.