Lead Spark Engineer, Big Data Risk Analytics

Rose International; Jersey City (NJ); On-site; USD 180,000 - 230,000; Full time

Job summary

Rose International seeks a Lead Engineer for Big Data Risk Aggregation to own the core platform that transforms raw calculation output into final, reportable risk numbers. You will build and run industrial-scale data pipelines that power risk reporting and analytics for global regulators.

You will lead a team of data engineers, developing a metadata-driven aggregation platform using Spark, and delivering fast, auditable results for multi-billion-dollar capital decisions.

Qualifications

• 10+ years in hands-on, large-scale data engineering in capital markets or risk tech.
• Expert-level Spark with performance tuning and partitioning.
• Deep knowledge of VaR, CCAR, Stress Testing, FRTB.
• Strong Java and/or Python, expert SQL.
• Experience building semantic data models bridging raw data to BI/AI.

Responsibilities

• Architect and build high-performance, metadata-driven aggregation platform with Spark.
• Design algorithms to reduce high-dimensional risk calculation complexity.
• Build a semantic data layer for BI and AI queries.
• Optimize Spark partitioning and minimize network shuffles for performance.
• Create a self-service platform for quants to define new aggregations.
• Develop APIs and datasets for notebooks to enable analysis.
• Load metrics into Pinot/Druid for analytics.

Skills

Apache Spark expert

Java/Python programming

SQL expert

Risk aggregation

Data modeling / semantic layer

High‑performance data pipelines

Problem solving at scale

Education

Quant/Technical degree

Tools

Apache Pinot

Apache Druid

Job description

Rose International seeks a Lead Engineer for Big Data Risk Aggregation to own the core platform that transforms raw calculation output into final, reportable risk numbers. You will build and run industrial-scale data pipelines that power risk reporting and analytics for global regulators.

You will lead a team of data engineers, developing a metadata-driven aggregation platform using Spark, and delivering fast, auditable results for multi-billion-dollar capital decisions.