Data Developer: AI Pipelines & Vector Databases

P2P; Montreal (administrative region); On-site; CAD 80,000 - 120,000; Full time

Job summary

A leading trading firm is seeking a Data Developer to design and implement data pipelines for AI and ML systems. The role involves building ingestion pipelines for both structured and unstructured data, collaborating with ML engineers, and optimizing workflows for efficiency. Candidates should possess a degree in Computer Science or related fields and have 2-5 years of practical experience. The position offers a dynamic environment focused on innovation and data-driven solutions.

Qualifications

• 2-5 years of experience building data systems and pipelines in production environments.
• Experience with vector databases and semantic search systems.
• Familiarity with optimization techniques for LLM inference.

Responsibilities

• Design and build data pipelines for RAG systems.
• Build ingestion pipelines for structured and unstructured data.
• Collaborate with ML engineers to optimize data formats.

Skills

Data pipelines construction

RAG architectures

Python programming

Distributed data processing

Data quality best practices

Education

Bachelor's or Master's degree in Computer Science, Data Engineering, or related field

Tools

Apache Spark

DAG-based orchestration platforms

Docker

Milvus

Job description

A leading trading firm is seeking a Data Developer to design and implement data pipelines for AI and ML systems. The role involves building ingestion pipelines for both structured and unstructured data, collaborating with ML engineers, and optimizing workflows for efficiency. Candidates should possess a degree in Computer Science or related fields and have 2-5 years of practical experience. The position offers a dynamic environment focused on innovation and data-driven solutions.