Lead Python Engineer - Big Data & Risk Analytics

TSR Consulting

Jersey City (NJ)

On-site

USD 160,000 - 200,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

TSR Consulting seeks a Lead Python Engineer in Jersey City (NJ) for a 6-month contract to architect and operate large-scale data aggregation platforms focused on risk analytics. You will design and optimize Spark-based pipelines and semantic data models that feed BI dashboards and AI-driven queries.

The role requires deep expertise in VaR, CCAR and FRTB risk contexts, plus experience delivering auditable, high-performance OLAP data layers with Pinot or Druid.

Qualifications

  • 10+ years of hands-on large-scale data engineering experience, preferably in Capital Markets or Risk technology.
  • Expert-level Apache Spark knowledge including architecture, performance tuning, shuffle optimization, and partitioning.
  • Deep understanding of risk aggregation for VaR, CCAR, Stress Testing, and FRTB.
  • Strong programming skills in Java and/or Python with expert-level SQL.
  • Experience building semantic layers and data models enabling BI and AI applications.
  • Proven delivery with high-performance OLAP databases such as Apache Pinot or Apache Druid.

Responsibilities

  • Architect and build a high-performance, metadata-driven aggregation platform using Apache Spark.
  • Design and implement algorithms to reduce computational complexity for high-dimensional risk aggregations.
  • Create and maintain a semantic layer and business-friendly data model to power BI and AI-driven queries.
  • Optimize Spark internals including partitioning, shuffle strategies, and memory for extreme performance.
  • Build a self-service framework that enables quants and risk managers to define new aggregations without new code.
  • Deliver high-performance APIs and curated datasets for UIs and Python Notebooks to support analysis and replication.
  • Load aggregated metrics into OLAP stores such as Apache Pinot or Apache Druid to support analytics and UI layers.

Skills

Apache Spark
Python
Java
SQL
Risk aggregation
VaR
BI/AI data models

Education

Degree in Computer Science or quantitative field

Tools

Apache Pinot
Apache Druid

Job description

Job Title: Lead Python Engineer - Big Data & Risk Analytics

Location: Jersey City, NJ

Duration: Contract - 6 months (C2H potential)

Job Description

We are seeking a Lead Engineer, Big Data Risk Aggregation to join our dynamic team. The ideal candidate will have strong experience in large-scale data engineering, Apache Spark, and risk aggregation within Capital Markets and a proven ability to design and operate high-performance, auditable aggregation platforms that deliver reportable risk metrics.

Responsibilities
  • Architect and build a high-performance, metadata-driven aggregation platform using Apache Spark.
  • Design and implement algorithms to reduce computational complexity for high-dimensional risk aggregations.
  • Create and maintain a semantic layer and business-friendly data model to power BI and AI-driven queries.
  • Optimize Spark internals including partitioning, shuffle strategies, and memory for extreme performance.
  • Build a self-service framework that enables quants and risk managers to define new aggregations without new code.
  • Deliver high-performance APIs and curated datasets for UIs and Python Notebooks to support analysis and replication.
  • Load aggregated metrics into OLAP stores such as Apache Pinot or Apache Druid to support analytics and UI layers.
Required Skills & Qualifications
  • 10+ years of hands-on large-scale data engineering experience, preferably in Capital Markets or Risk technology.
  • Expert-level Apache Spark knowledge including architecture, performance tuning, shuffle optimization, and partitioning.
  • Deep understanding of risk aggregation for VaR, CCAR, Stress Testing, and FRTB.
  • Strong programming skills in Java and/or Python with expert-level SQL.
  • Experience building semantic layers and data models enabling BI and AI applications.
  • Proven delivery with high-performance OLAP databases such as Apache Pinot or Apache Druid..
  • Degree in Computer Science, Financial Mathematics, Financial Engineering, or a related quantitative field.
Preferred Skills
  • Experience leading data engineering teams and establishing best practices for big data processing.
  • Demonstrated strength in performance diagnostics across CPU, memory, and network I/O.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Python Engineer: Spark-Powered Big Data Risk Analytics
Lead Python Engineer: Spark-Powered Big Data Risk Analytics

TSR Consulting • Jersey City (NJ)

On-site
USD 160,000 - 200,000
Java Spark Backend Developer
Java Spark Backend Developer

Motion Recruitment • Jersey City (NJ)

Hybrid
USD 150,000 - 210,000
Lead Software Engineer – Market Risk Reporting & Visualization
Lead Software Engineer – Market Risk Reporting & Visualization

JPMorgan Chase & Co. • Jersey City (NJ)

On-site
USD 120,000 - 180,000
Software Engineer III - Market Risk Reporting & Visualization
Software Engineer III - Market Risk Reporting & Visualization

JP Morgan Chase • Jersey City (NJ)

On-site
USD 120,000 - 170,000
Lead Data Engineer | New Jersey , Virginia – USA
Lead Data Engineer | New Jersey , Virginia – USA

DigitalXNode • New Jersey

On-site
USD 150,000 - 200,000
Lead Software Engineer - Market Risk
Lead Software Engineer - Market Risk

Testing Site - JPMC • Jersey City (NJ)

On-site
USD 140,000 - 210,000
Analytics Engineer
Analytics Engineer

Cleartech Recruiting • New York (NY)

On-site
USD 120,000 - 190,000
Senior Lead Software Engineer - Python, PySpark, Big Data, Data pipeline, ML/AI
Senior Lead Software Engineer - Python, PySpark, Big Data, Data pipeline, ML/AI

JPMorgan Chase & Co. • Plano (TX)

On-site
USD 140,000 - 180,000
Software Engineer III – Market Risk Reporting & Visualization
Software Engineer III – Market Risk Reporting & Visualization

JPMorgan Chase & Co. • Jersey City (NJ)

On-site
USD 140,000 - 190,000
Senior Lead Software Engineer - Python, PySpark, Big Data, Data pipeline, ML/AI
Senior Lead Software Engineer - Python, PySpark, Big Data, Data pipeline, ML/AI

JP Morgan Chase • Plano (TX)

On-site
USD 140,000 - 210,000