Senior Data Engineer

Quantum Technology Solutions Inc; London; On-site; GBP 60,000 - 80,000; Full time

Job summary

Quantum Technology Solutions Inc is seeking a Senior Data Engineer to design and maintain scalable data systems supporting AI and trading technologies. In this key role, you will develop robust data pipelines, improve data quality, and work collaboratively on cutting-edge projects, ensuring the optimal performance and accessibility of our data infrastructure.

Benefits

Opportunity for career growth

Collaborative work environment with experts

Participation in innovative AI projects

Qualifications

• 5+ years of experience in Data Engineering with proven ability to build data pipelines.
• Proficiency with modern data technologies and experience in cloud ecosystems.
• Strong understanding of data security compliance and governance.

Responsibilities

• Design, build, and optimize scalable data pipelines for AI research and production.
• Architect data ingestion, transformation, and storage systems for high performance.
• Automate ETL processes and enhance data workflows for efficiency.

Skills

Data Pipeline Development

SQL Skills

Python

Cloud Technologies

Data Quality Assurance

Collaboration

Education

Bachelor's Degree in Computer Science or related field

Tools

Apache Airflow

Spark

Kafka

Snowflake

Job description

Get AI-powered advice on this job and more exclusive features.

Quantum is building next-generation AI and trading technologies that harness cutting-edge research and data science. As part of our rapid growth, we are seeking a highly skilled Senior Data Engineer to support our Data & AI team by designing and maintaining robust, scalable, and production-grade data systems. This is greenfield project, allowing for full product ownership and key decision making.

Role Overview:

As a Senior Data Engineer at Quantum, you will be instrumental in building the infrastructure that powers our Data Science, AI and Trading tools. You will work closely with the Data and Technology teams to ensure data accessibility, quality, and scalability - particularly focusing on trading (time-series databases), analytical, and AI pipelines.

This is a highly collaborative role, ideal for someone who thrives on taking full ownership from systems design to implementation in a fast-paced, research-driven environment and wants to be part of building world-class trading system capabilities from the ground up.

Key Responsibilities:

• Develop and Maintain Data Pipelines:

Design, build, and optimise scalable data pipelines to support AI research and production systems, particularly for unstructured, text-heavy and time-series based datasets.

• Data Infrastructure Design:

Architect and implement data ingestion, transformation, storage, and retrieval systems, ensuring they are resilient, high-performing, and fit for future growth.

• Data Quality and Exploration:

Support data exploration efforts by ensuring high data quality, developing validation frameworks, and contributing to continuous data improvement initiatives.

• Collaboration with AI Teams:

Work closely with the Principal Data Scientist to operationalise RAG systems, fine-tune data retrieval processes, and optimise training datasets for AI model development.

• Automation and Optimisation:

Automate ETL (Extract, Transform, Load) processes, reduce manual intervention, and continuously identify opportunities to enhance the efficiency and reliability of data workflows.

• Support Research and Prototyping:

Build and maintain flexible data systems to support rapid experimentation, research validation, and the transition of prototypes into production environments.

• Monitoring and Troubleshooting:

Implement robust monitoring, logging, and alerting for data pipelines to proactively detect issues and maintain high availability and performance.

• Documentation and Best Practices:

Establish and maintain high standards for data engineering documentation, coding practices, and data governance.

Required Skills and Qualifications

• 5+ years of experience in Data Engineering, with a strong background in building data pipelines at scale.
• Proficiency with modern data technologies (e.g Apache Airflow, Spark, Kafka, Snowflake, or similar).
• Strong SQL skills and experience with cloud databases and data warehouses (AWS, GCP, or Azure ecosystems).
• Expertise in working with unstructured data and NLP-related datasets.
• Proficiency in one programming language, preferably Python with experience in data processing libraries such as Pandas, PySpark, or Dask.
• Familiarity with MLOps and deploying AI/ML models into production environments.
• Knowledge of Retrieval-Augmented Generation (RAG) frameworks or interest in learning and supporting RAG systems.
• Experience implementing scalable APIs and integrating data services with AI and analytics platforms.
• Strong understanding of data security, compliance, and governance best practices.
• Excellent collaboration and communication skills, able to work closely with technical and non-technical stakeholders.

Preferred Qualifications

• Experience supporting AI/ML research teams.
• Familiarity with LLM (Large Language Model) pipelines and vector databases (e.g. Pinecone, FAISS).
• Background in data versioning and experiment tracking (e.g DVC, MLflow).
• Familiarity with time-series datasets and databases.

Why Join Quantum?

• Work at the forefront of AI innovation with a team passionate about changing the future of trading and technology.
• Take ownership and make a direct impact from day one.
• Collaborate closely with world-class AI researchers, data scientists, and engineers.
• Opportunity for career growth as part of a rapidly expanding AI and data science team.

Quantum is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Software Development

Referrals increase your chances of interviewing at Quantum Technology Solutions Inc by 2x

Sign in to set job alerts for “Senior Data Engineer” roles.

London, England, United Kingdom 1 week ago

London Area, United Kingdom £60,000.00-£65,000.00 19 hours ago

London Area, United Kingdom $100,000.00-$120,000.00 2 weeks ago

Team Lead - Senior Analytics Engineer - Utilities

London, England, United Kingdom 2 weeks ago

Chigwell, England, United Kingdom 3 weeks ago

Senior Software Engineer - Card Integrations - Visa / Mastercard

London, England, United Kingdom 1 week ago

London, England, United Kingdom 1 month ago

Grafana specialist - Senior Data Engineer. Trading Firm. 6 month rolling contract. £700-£800/ Day Inside IR35. Hybrid 2/3 Days a week in Central London office.

Greater London, England, United Kingdom 12 hours ago

Senior Software Engineer, Robotics & Microscope Control

London, England, United Kingdom 1 month ago

London Area, United Kingdom £90,000.00-£105,000.00 14 hours ago

London, England, United Kingdom 1 week ago

Senior Data Analyst | Greenfield Trading House | Up to £100k + Bonus | London Hybrid 3x/week

London, England, United Kingdom 2 weeks ago

London, England, United Kingdom 2 weeks ago

Data Architect - Data Modelling - Dimensional - Relational - Data Vault

London, England, United Kingdom 3 days ago

Data Architect - AWS & Snowflake Expertise

City Of London, England, United Kingdom £500.00-£750.00 1 week ago

London, England, United Kingdom 1 month ago

City Of London, England, United Kingdom £60,000.00-£60,000.00 1 day ago

London, England, United Kingdom 4 weeks ago

Senior AI/ML Engineer (Data Science & Software Focus)

London, England, United Kingdom 3 weeks ago

Business Data Architect – Keying and Linking

London, England, United Kingdom 2 weeks ago

London, England, United Kingdom 2 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.