Data Engineer

Novel Financial Solutions; Pune District; Hybrid; INR 1,800,000 - 2,400,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Novel Financial Solutions invites a Senior Hands-on Technologist to join our Massive Scale Risk Analytics team in Pune on a 6-month contract-to-hire basis. You will architect and implement robust data pipelines, process billions of PV calculations, and optimize Spark-based jobs for performance at scale.

You will design intelligent data APIs for on-demand risk data, integrate NLP-driven interfaces, and load data into Pinot, Druid, and Trino OLAP engines.

Qualifications

• Degree in Computer Science, Financial Mathematics, or Financial Engineering.
• Expert-level experience with big data technologies, especially Apache Spark.
• Strong SQL skills and experience with Pinot, Druid, or Trino.
• Programming skills in Python, Java, or Scala.
• Interest in AI-first development and NLP for analytics.

Responsibilities

• Architect and build scalable data pipelines for billions of PV calculations.
• Develop large-scale aggregation jobs with Apache Spark.
• Create intelligent data APIs for on-demand risk data.
• Integrate NLP capabilities for query-based data exploration.
• Load and model data into OLAP engines like Pinot, Druid, or Trino.
• Build interactive analytics tools and dashboards on TOP of OLAP layer.
• Collaborate with Front Office, Quant, and Risk teams.

Skills

Apache Spark

Python

Java

Scala

SQL

Education

Degree in Computer Science or quantitative field

Tools

Apache Pinot

Apache Druid

Trino

Job description

Hi ,

Greetings from Novel Financial Solutions Pvt Ltd.

Novel is one of the Leading companies and we are a 25+ Years old Service Organization with presence in USA, India, Singapore and Hong Kong.

• Role : Senior Hands-on Technologist, Massive Scale Risk Analytics
• Location: Pune
• Duration- 6 Months Contract to hire)
• No of days of working: 5 Days (Hybrid)

Job Description

For Spark with Java:

Key Responsibilities

• Architect and build robust, scalable data pipelines to ingest and process billions of trade-level PV calculations from various stress engines.
• Develop and optimize large-scale aggregation jobs using Apache Spark, ensuring high performance and efficiency.
• Design and deliver a suite of "intelligent data APIs" that provide flexible, on-demand access to both aggregated and non-aggregated risk data for teams across the firm.
• Integrate Natural Language Processing (NLP) capabilities to create intuitive, query-based interfaces for data exploration, lowering the barrier to entry for complex analytics.
• Load and model massive aggregated datasets into high-performance OLAP engines like Apache Pinot, Apache Druid, and Trino.
• Build powerful, interactive analytical tools and dashboards on top of the OLAP layer, providing summary views and lightning-fast drill-down capabilities.
• Partner directly with senior stakeholders in the Front Office, Quantitative teams, and Risk Management to understand their analytical needs and deliver innovative solutions.

What We're Looking For

• A true passion for data, analytics, and solving complex problems at massive scale.
• A degree in a quantitative or technical field such as Computer Science, Financial Mathematics, or Financial Engineering.
• Expert-level, hands-on experience with big data technologies, particularly Apache Spark.
• Proven experience with high-performance OLAP databases such as Apache Pinot, Apache Druid, or Trino.
• Strong programming skills in Python, Java and/or Scala, and expert-level SQL.
• Strong background in fundamental computer science concepts, including data structures and algorithms.
• A mindset for 'AI-first' development, constantly looking for ways to embed intelligence into systems.
• Experience or a strong interest in applying Natural Language Processing (NLP) to data access and analytics.
• A knack for solving 'needle in a haystack' problems, with a talent for debugging complex data and access issues.
• Experience in the financial industry with an understanding of market risk, derivatives, and risk calculations (VaR, Stress Testing, PV) is highly desirable.
• Exceptional problem-solving skills and the ability to work independently and lead technical projects.
• Excellent communication skills, with the confidence to collaborate with senior business and quantitative stakeholders.

Regards,

Snehal Makasare

[e-mail]