Big Data Engineer (Spark/Scala/Java)

Luxoft Poland; Kraków; On-site; PLN 65,000 - 85,000; Full time

Job summary

Luxoft Poland is seeking an experienced developer to work on the ACQA platform, utilizing Microsoft Azure technologies. The position involves developing big data technologies, programming complex production systems, and working with CI/CD pipelines.

The ideal candidate should have strong skills in Apache Spark, Scala, Java, and be familiar with Kubernetes. This role offers a unique opportunity to work within a global team on innovative solutions.

Qualifications

• Experience in programming complex production systems in Scala or Java or Python.
• Development of big data technologies like Apache Spark and Azure Databricks.
• Experience with containerisation in development, build, and runtime environments.

Responsibilities

• Development of big data technologies like Apache Spark and Azure Databricks.
• Programming complex production systems in Scala or Java or Python.
• Development and/or operations of CI/CD pipelines using cloud-friendly build systems.

Skills

Apache Spark

Azure Databricks

Scala

Java

Python

CI/CD pipelines

GitLab

Kubernetes

Tools

Terraform

Job description

ACQA is built on Microsoft Azure cloud computing technology. It aims to deliver:

• Scalable cost-efficient infrastructure, using cloud PaaS components.
• Single core platform, open architecture, designed for change, itemised $cost metrics, automated data lineage.
• Shared across Front Office, Finance and Risk, improving regulatory compliance.
• One-Platform / One-Experience -fast to train, easy to operate, retaining talent.

The ACQA platform is made up of a series of components providing the next generation valuation and risk management services.

Responsibilities

• Development of big data technologies like Apache Spark and Azure Databricks
• Programming complex production systems in Scala or Java or Python
• Experience in a platform engineering role on a major cloud provider (ideally Microsoft Azure)
• Development and/or operations of CI/CD pipelines using modern cloud-friendly build systems like GitLab
• Experience with containerisation in development, build and runtime environments including experience of Kubernetes

Mandatory Skills Description

• Development of big data technologies like Apache Spark and Azure Databricks
• Programming complex production systems in Scala and Java
• Experience in a platform engineering role on a major cloud provider (ideally Microsoft Azure)
• Development and/or operations of CI/CD pipelines using modern cloud-friendly build systems like GitLab
• Experience with containerisation in development, build, and runtime environments, including experience with Kubernetes

Nice‑to‑Have Skills Description

• Production experience with Terraform on a major public cloud provider (ideally Microsoft Azure)
• Testing and instrumentation practices, observability and alerting tooling
• Understanding of information modelling, data structures and algorithms
• Technical architecture and low-level design
• Agile development and planning practices
• Ability to collaborate effectively in a large global team and influence key architectural decisions