Scala/Spark Optimization Engineer (HPC)

TN United Kingdom; United Kingdom; Remote; GBP 50,000 - 90,000; Full time

Job summary

An exciting opportunity awaits a skilled Scala Engineer focused on accelerating Apache Spark workloads and developing high-performance solutions. This role involves optimizing applications for complex data processing tasks and enhancing JVM-based solutions for performance and scalability. You'll collaborate on integrating solutions with Apache engine components and troubleshoot performance bottlenecks in big data and HPC environments. Join a forward-thinking company at the forefront of big data and high-performance computing, where your contributions will drive innovation and excellence in cutting-edge projects.

Qualifications

• Strong proficiency in Scala and deep understanding of JVM ecosystem.
• Experience with Apache Spark performance optimization.

Responsibilities

• Optimize and accelerate Apache Spark applications for large-scale data processing.
• Design and implement domain-specific languages and contribute to compiler optimization.

Skills

Scala Programming

Apache Spark

JVM Ecosystem

Compilers

Domain-Specific Languages (DSLs)

High-Performance Computing (HPC)

Distributed Systems

Analytical Skills

Tools

Kubernetes

Docker

Job description

Social network you want to login/join with:

Client:

IC Resources

Location:

remote, uk, United Kingdom

Job Category:

-

EU work permit required:

Yes

Job Reference:

8e63fb95280d

Job Views:

6

Posted:

24.04.2025

Expiry Date:

08.06.2025

Job Description:

Job Title: Scala Engineer – Apache Spark & HPC

Job Description:
We are seeking an experienced Scala Engineer to focus on accelerating Apache Spark workloads and developing high-performance solutions. This role requires expertise in Scala, a deep understanding of the JVM & domain-specific languages (DSLs), and experience with compilers or HPC.

Key Responsibilities:

• Optimise and accelerate Apache Spark applications for complex, large-scale data processing tasks.
• Develop and enhance JVM-based solutions to ensure performance, scalability, and reliability.
• Design and implement domain-specific languages (DSLs) and contribute to compiler optimisation pipelines.
• Collaborate on integrating solutions with Apache engine components and distributed systems.
• Troubleshoot, profile, and resolve performance bottlenecks in big data and HPC environments.

Required Skills and Experience:

• Strong proficiency in Scala programming and a deep understanding of the JVM ecosystem.
• Hands-on experience with Apache Spark, particularly in performance optimization.
• Solid knowledge of compilers, domain-specific languages (DSLs), or high-performance computing (HPC).
• Proven experience in designing, developing, and optimising distributed systems.
• Strong analytical skills and a focus on delivering efficient, high-quality solutions.

Nice-to-Have:

• Experience with other distributed computing frameworks or big data technologies.
• Knowledge of cloud-based platforms and containerisation (e.g., Kubernetes, Docker).

This is an exciting opportunity for a skilled engineer to work on cutting-edge projects at the intersection of big data and high-performance computing.