Pricing & Derived Data - Senior Software Engineer

drweng; Chicago (IL); On-site; USD 175,000 - 250,000; Full time

Benefits offered by this job

Annual discretionary bonus

Job summary

DRW in Chicago is hiring a Senior Software Engineer to help build a new data platform delivering intra-day and end-of-day prices across asset classes. You'll work with Elixir, gRPC, PostgreSQL, Kafka, and ClickHouse to support low-latency access for trading systems and researchers.

You will shape architecture decisions, drive scalable ingestion/storage, and collaborate across teams to ensure consistent, gap-free data delivery. Prior experience with data streaming and open formats is valued.

Qualifications

• Senior generalist with significant experience building data products.
• Must be able to lead architectural decisions and tradeoffs.
• Strong understanding of event streaming, change data capture, open table formats and stream/batch processing.

Responsibilities

• Designing and building high-volume, low-latency data processing systems (ingestion, storage, distribution).
• Scale ingestion pipelines handling thousands of events per second across nodes.
• Operate and support data products with time- and correctness-sensitive delivery requirements.

Skills

Senior generalist

Architectural decisions

Data products

Tools

Kafka

Delta Lake

Iceberg

ClickHouse

Spark

Flink

Elixir

Job description

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.

Headquartered in Chicago with offices throughout the U.S., Canada, Europe, and Asia, we trade a variety of asset classes including Fixed Income, ETFs, Equities, FX, Commodities and Energy across all major global markets. We have also leveraged our expertise and technology to expand into three non-traditional strategies: real estate, venture capital and cryptoassets.

We operate with respect, curiosity and open minds. The people who thrive here share our belief that it's not just what we do that matters-it's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus.

UP (Unified Platform) consolidates data and technologies into a comprehensive solution, accelerating time to market. Optimizing decision-making and research, our platform simplifies implementing new ideas, enhancing our competitive edge. We work from high-level requirements where programmers are expected to gain an intimate understanding of the business. We strive for an environment where developers have the autonomy to pursue business solutions without arbitrary management constraints. We are a group of polyglots and technologists always seeking to find the right tool for each problem. Broadly, the development teams at DRW believe in rapid deployment and feedback, automated testing, and cohesive, transparent, lightly coupled service architectures.

The Pricing and Derived Data team within UP is seeking an outstanding Senior Software Engineer to join us in building a new data platform that provides intra-day and end-of-day prices across the firm.

The team is responsible for pricing millions of instruments across many asset classes and exchanges in near-real-time. Clients range from trading systems needing low-latency updates to researchers working with large historical datasets.

Our current architecture is written mostly in Elixir, which we enjoy and derive great value from. Inter-service streaming communication is over gRPC and the data layer comprises PostgreSQL, Kafka, and ClickHouse. Demand for our services is growing rapidly across a variety of access patterns, and we are now building a comprehensive data platform as an extension of our existing system to meet this demand.

This position presents an opportunity to shape the future direction of our work. The successful candidate will play a key role in answering questions such as:

• How do we guarantee that a client sees the same prices whether they query the warehouse, subscribe to a stream, or call our gRPC API?
• How do we let clients load a snapshot and then switch to live updates with no gaps or duplicates?
• What parts of our systems should stay in Elixir, and where should we consider other technologies?

Experience

We are looking for strong senior generalists who have a considerable experience building data products. Candidates should be able to provide examples of significant projects that they have led from design through to operating in production, including responsibility for architectural decisions. We are interested in hearing from candidates who have had to choose between technologies and can talk to us about why they would or would not make that choice again.

Relevant areas include event streaming, change data capture, open table formats and stream and batch processing. Associated technologies include Kafka, Delta Lake, Iceberg, ClickHouse, Spark, and Flink. We are interested in a deep understanding of the architectural requirements that these technologies are suitable for rather than years of experience in a specific one of them.

Prior experience in Elixir or in the financial domain is not required, but candidates should be interested in working in these areas.

Examples of projects you may have led include:

• Designing and building a high-volume and/or low-latency data processing system, including ingestion, storage, and distribution.
• Scaling an ingestion pipeline handling thousands or tens or thousands of events per second on a single node (optimisation) and/or across multiple nodes (distribution).
• The operation and support of a data product which has time- and correctness-sensitive delivery requirements.

Personal traits

• Humility and respect for others, and trust in our teammates.
• The ability to learn, adapt and grow.
• The capacity to independently drive projects to completion, but a preference to collaborate with teammates.
• Excellent listening and communication skills

The annual base salary range for this position is $175,000 to $250,000 depending on the candidate's experience, qualifications, and relevant skill set. The position is also eligible for an annual discretionary bonus.

In ad