Real-Time Data Engineer for AI-Powered Pipelines

Ampstek; Brampton; On-site; CAD 90,000 - 150,000; Full time

Job summary

Ampstek is seeking a data architect to design and operate ETL pipelines for structured and unstructured data, while managing vector databases and hybrid search layers. You will enforce governance and build real-time data flows that keep embeddings up to date.

You will work with distributed systems, tuning high-throughput infrastructure and implementing privacy controls across AI pipelines to ensure fresh and trustworthy information for AI agents.

Qualifications

• Experience with distributed data systems and SQL.
• Experience tuning high-throughput database infrastructure.
• Familiarity with embedding models and chunking strategies.
• Knowledge of data governance, privacy controls, and auditability.

Responsibilities

• Architect ETL pipelines for structured and unstructured data.
• Manage vector infrastructure including pgvector, Redis, and Azure AI Search.
• Implement zero-trust access, privacy controls, and compliance in AI pipelines.
• Build real-time, event-driven architectures to refresh embeddings and indexes.

Skills

ETL & Data Modeling

Vector Databases

Data Governance

Real-time Embedding Updates

Chunking & Embeddings

Search Infrastructure

Performance Tuning

Data Quality & Lineage

Tools

pgvector

Redis

Azure AI Search

Kafka

Spark

Flink

Great Expectations

OpenLineage

Job description

Ampstek is seeking a data architect to design and operate ETL pipelines for structured and unstructured data, while managing vector databases and hybrid search layers. You will enforce governance and build real-time data flows that keep embeddings up to date.

You will work with distributed systems, tuning high-throughput infrastructure and implementing privacy controls across AI pipelines to ensure fresh and trustworthy information for AI agents.