Senior Java Developer, Credit Risk & Data Engineering

BIP US; New York (NY); Hybrid; USD 150,000 - 200,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Voluntary benefits

Short-term disability

Long-term disability

HSA

FSA

401(k) matching

Performance bonuses

Referral bonuses

Employee assistance program

Public holidays

Job summary

BIP US is seeking a senior software/data engineer to design and deliver scalable data-processing solutions for Capital Markets and Risk data platforms. You will architect batch and streaming pipelines, build backend services, and collaborate with risk and technology teams.

Ideal candidates have 8+ years in enterprise settings, strong Java and Spark experience, and deep knowledge of Parquet, SQL, and cloud data platforms.

Qualifications

• 8+ years of professional experience in software/data engineering or application development.
• Strong Java programming with Python/Scala for data-intensive apps.
• Production Spark experience and large-scale data processing expertise.
• Experience with Parquet and columnar data formats; partitioning and compression.
• Strong SQL skills for large datasets; data transformation and reconciliation.
• Designing scalable data pipelines and batch/streaming processing.
• Knowledge of distributed computing, concurrency, and architecture.
• Experience building backend services/APIs and enterprise components.
• Experience with cloud data platforms (AWS/EMR) and CI/CD practices.
• Ability to translate complex requirements into technical solutions.

Responsibilities

• Design, develop, and support scalable software and data-processing solutions for financial services.
• Manage full software lifecycle from requirements through deployment and support.
• Build and optimize batch and streaming pipelines using Spark.
• Develop scalable backend services, processing engines, and APIs.
• Ingest, transform, and analyze trade, position, and risk data.

Skills

Java programming

Python/Scala

Spark

Parquet/columnar data

SQL

Distributed computing

Data pipelines

Backend services

Cloud/AWS

CI/CD

Communication

Education

Bachelor's degree in CS/Engineering/Math

Tools

Hive

HDFS

Iceberg

HBase

Kafka

Dremio

EMR

Job description

Salary: $150,000 - 200,000 per year

Requirements

• 8+ years of professional experience in software engineering, data engineering, or application development, preferably in Financial Services, Capital Markets, Risk, or another complex enterprise setting.
• Strong hands-on Java programming skills, with working knowledge of Python and/or Scala for data-intensive application development.
• Production experience with Apache Spark, including building and optimizing large-scale distributed processing workloads.
• Experience with Parquet and large columnar datasets, including partitioning, schema management, compression, encoding, and performance optimization.
• Strong SQL skills and experience querying, transforming, reconciling, and analyzing large datasets.
• Experience designing scalable data pipelines, transformation frameworks, aggregation processes, and batch or distributed processing solutions.
• Knowledge of distributed computing, concurrency, software design, data structures, algorithms, and application architecture.
• Experience tuning data workloads, including Spark joins, partitioning, caching, data skew, resource utilization, and throughput.
• Experience building and integrating backend services, APIs, and enterprise application components.
• Experience across the software development lifecycle, from requirements and design through development, testing, deployment, and production support.
• Strong debugging, performance optimization, data-quality analysis, and problem-solving abilities.
• Experience working with business, risk, quantitative, and technology stakeholders and translating complex requirements into technical solutions.
• Strong written and verbal communication skills suited to a client-facing consulting environment.
• Bachelors degree in Computer Science, Engineering, Mathematics, or a related technical field, or equivalent professional experience.
• Preferred: Experience supporting Credit Risk, Market Risk, regulatory risk, or enterprise trade and risk data platforms.
• Preferred: Familiarity with financial data such as trades, positions, sensitivities, reference data, counterparty or exposure data, RWA, and VaR.
• Preferred: Experience with Big Data technologies such as Hive, HDFS, Iceberg, HBase, Impala, Kafka, Dremio, or EMR.
• Preferred: Experience with batch and streaming data processing, including real-time and historical financial data.
• Preferred: Experience with Java, Python, Scala, Spark/PySpark, SQL, Spring Boot, and modern backend frameworks.
• Preferred: Experience with cloud data platforms, particularly AWS and EMR, as well as containerization and modern CI/CD practices.
• Preferred: Performance engineering across Spark, Java, SQL, and large-scale storage platforms.
• Preferred: Experience modernizing, refactoring, or reverse engineering legacy technology and data platforms.
• Preferred: Experience with data quality, reconciliation, lineage, metadata management, and auditability in regulated environments.
• Preferred: Experience at Tier 1 investment banks or other large financial institutions, and prior management consulting experience.
• Preferred: Exposure to AI/ML techniques or agentic engineering tools used in data or software engineering.
• Preferred: Ability to work independently and own complex technical challenges from analysis through implementation.

Responsibilities

• Design, develop, enhance, and support scalable software and data-processing solutions for Capital Markets, Credit Risk, Market Risk, and related Financial Services functions.
• Manage development across the full software lifecycle, from requirements analysis and technical design through implementation, testing, deployment, and production support.
• Build and optimize large-scale batch and, where needed, streaming pipelines using Apache Spark and related distributed technologies.
• Develop scalable backend services, processing engines, APIs, and integrations using Java, Python, Scala, or comparable technologies.
• Ingest, transform, enrich, aggregate, reconcile, and analyze trade, position, sensitivity, reference, counterparty, and risk data.
• Apply effective partitioning, compression, encoding, compaction, and data-access strategies to Parquet and other columnar or table storage formats.
• Tune Spark workloads using methods such as repartitioning, salting, broadcast joins, and caching.
• Partner with Credit Risk, Market Risk, front-office, quantitative, data, architecture, and technology stakeholders to understand needs and develop solutions.
• Convert complex risk, business, and data requirements into robust technical designs and scalable processing frameworks.
• Troubleshoot, refactor, and optimize existing applications, pipelines, and large or complex codebases.
• Improve performance, scalability, maintainability, reliability, throughput, and code quality across applications and data platforms.
• Build in data-quality controls, reconciliation, lineage, transparency, and auditability for regulated financial environments.
• Contribute to architecture and technical design discussions, recommending data models, storage patterns, processing frameworks, and implementation approaches.
• Review code and advance software and data engineering standards and best practices.
• Diagnose and resolve complex application, integration, data, Spark, database, and production issues.
• Collaborate with globally distributed development, risk, data, and business teams.
• Explain technical concepts clearly to technical and non-technical client stakeholders.

Technologies

• AI
• AWS
• Big Data
• Backend
• CI/CD
• Cloud
• Hive
• Support
• Java
• Kafka
• Python
• PySpark
• SQL
• Scala
• Spark
• Spring
• Spring Boot

More

We are Business Integration Partners (BIP), a digital consulting company with 6,000 employees worldwide, working at the intersection of business and technology to deliver sustainable solutions. Our Financial Services practice serves Capital Markets, Insurance, and Payments, supported by Data & AI, Cybersecurity, Risk & Compliance, Change Management, and Digital Transformation capabilities. We are growing our U.S. presence and Capital Markets and Financial Services teams, with offices in New York, Charlotte, Chicago, and Houston. This client-facing consulting role focuses on financial data and risk platforms. The base salary range is $150,000-$200,000. Benefits include medical, dental, and vision coverage; voluntary benefits; short- and long-term disability; HSA and FSA options; matching 401(k); discretionary performance and referral bonuses; an employee assistance program; 11 public holidays; 20 days of PTO; 7 sick days; a PTO buy-and-sell program; volunteer days; paid parental leave; and remote or hybrid work support. We provide equal employment opportunities and maintain a workplace free from discrimination, harassment, and bias. Actual compensation may vary based on factors including skills, education, experience, and knowledge.

last updated 41 week of 2026