Core Data Engineer

Flow Traders; Amsterdam; On-site; EUR 65,000 - 90,000; Full time

Job summary

A leading trading firm in Amsterdam is seeking a Core Data Engineer to manage the lifecycle of critical data products. You will develop automated data pipelines and collaborate across teams to deliver high-quality data. The ideal candidate has Python programming skills, experience with big data, and strong project management abilities. This full-time role demands an organized individual who can balance stakeholder needs while ensuring the quality and applicability of data products.

Qualifications

• Excellent programming skills in any language, preferably Python.
• Proficiency in Linux, SQL, and both relational and non-relational databases.
• Ability to work with large datasets and experience with big data solutions.

Responsibilities

• Deliver timely and high-quality core data products.
• Develop and manage the end-to-end lifecycle of data products.
• Create a common data model across data products.
• Evaluate and onboard new data sources.
• Automate workflows and provide guidance on data architecture.

Skills

Programming skills (Python preferred)

Working proficiency with Linux

SQL and relational/non-relational databases

Experience with large datasets

Airflow

Kafka

GCP

Organized and detail-oriented

Interpersonal skills

Project management

Tools

Pandas

Spark

Apache Beam

Apache NiFi

Job description

Flow Traders is looking for a Core Data Engineer to own the end-to‑end lifecycle of our most fundamental data products, including our Security Master; our reconciled ETF Compositions; high‑resolution historical market data; and records of our trades, orders, quotes, and inquiries. High‑quality data is the cornerstone of Flow Traders' strategic direction - beyond reducing complexity and BAU for stakeholders across the organization, quality data is a key requirement for our growth of quantitative trading styles.

In this role you will craft and execute the blueprint for these products, leveraging our Data Platform to build and monitor automated pipelines which ensure that users across the organization have reliable data at their fingertips when they need it. Further, you will become an expert in these data sets, leveraging that knowledge to optimize our use of the data we have and explore new sources for additional value. Finally, you will constantly collaborate with teams across the organization to ensure that our in‑house systems are designed with data in mind, streamline additional data‑related workflows, and work with stakeholders to maximize the value they extract from our data.

As the architect and guardian of our most valuable data, you will have a unique opportunity to make your mark on Flow Traders. You will be accountable for delivering these data products at an exceptional level of quality, ensuring that they are not only timely and pristine, but powerful enough to drive Flow Traders' future growth. Ultimately, your work will deliver one of the most important building blocks to the firm's strategic growth for the years to come.

What You Will Do

• Be accountable for timely and high-quality delivery of core data products to the rest of the organization: Security Master, ETF Compositions, Market Data, Trades / Orders / Quotes / Inquiries
• Develop and own the end-to‑end lifecycle of these data products: build and maintain pipelines, develop quality checks, correct day‑to‑day issues, and systematize/automate manual processing steps
• Define and manage project plans to assume ownership of these data products
• Create and refine a common data model (esp. identifiers, normalized schemas, use of master data) across these data products
• Investigate currently available sources of data and ensure that Flow extracts maximum information and value from them
• Explore, evaluate, and onboard additional sources of available data (both independently and at the request of stakeholders)
• Work closely with other teams to: automate existing workflows, ensure pipelines leverage standard Flow technologies, provide guidance on data-related architecture/design topics, ensure licensing and access control are appropriate

What You Need To Succeed

• Excellent programming skills (any language, Python preferred)
• Working proficiency with Linux, SQL and relational/non‑relational databases
• Demonstrated ability to work with large datasets and familiarity with big data solutions
• Experience with Airflow, Kafka, and GCP (esp. GKE, Dataproc, and Big Query) highly desired
• Experience with Pandas, Spark, Apache Beam, Apache NiFi a plus
• Organized with strong attention to detail and quality
• Innately driven to generalize, systematize, and automate
• Strong interpersonal skills, works very well with both technical and non‑technical stakeholders
• Able to balance individual stakeholder demands to preserve quality and wide applicability of data
• Organized and able to define, manage, and execute projects (often several in parallel) while effectively triaging operational concerns
• Primarily technical, but domain‑literate (exposure to symbology, security reference/pricing data, ETF compositions, exchange market data)

Flow Traders does not accept unsolicited resumes from any professional staffing or search firms. All resumes, and any other information identifying potential candidates, submitted to any employee at Flow Traders via‑email, the Internet or directly without a valid and signed search agreement will be deemed free to contact by Flow Traders without any restrictions and no placement fee of any kind will be paid in the event the candidate is hired by Flow Traders.

Seniority level

Mid‑Senior level

Employment type

Full‑time

Job function

Information Technology and Engineering

Software Development