Data Engineer

Ampstek; Brampton; On-site; CAD 90,000 - 150,000; Full time

Job summary

Ampstek is seeking a data architect to design and operate ETL pipelines for structured and unstructured data, while managing vector databases and hybrid search layers. You will enforce governance and build real-time data flows that keep embeddings up to date.

You will work with distributed systems, tuning high-throughput infrastructure and implementing privacy controls across AI pipelines to ensure fresh and trustworthy information for AI agents.

Qualifications

• Experience with distributed data systems and SQL.
• Experience tuning high-throughput database infrastructure.
• Familiarity with embedding models and chunking strategies.
• Knowledge of data governance, privacy controls, and auditability.

Responsibilities

• Architect ETL pipelines for structured and unstructured data.
• Manage vector infrastructure including pgvector, Redis, and Azure AI Search.
• Implement zero-trust access, privacy controls, and compliance in AI pipelines.
• Build real-time, event-driven architectures to refresh embeddings and indexes.

Skills

ETL & Data Modeling

Vector Databases

Data Governance

Real-time Embedding Updates

Chunking & Embeddings

Search Infrastructure

Performance Tuning

Data Quality & Lineage

Tools

pgvector

Redis

Azure AI Search

Kafka

Spark

Flink

Great Expectations

OpenLineage

Job description

You will architect ETL pipelines, manage vector databases, enforce governance, and build real-time data flows that continuously update embeddings and indexes. Your work ensures AI agents operate with fresh, trustworthy information.

What You Will Do

• Data Pipelines: Build ETL flows for structured/unstructured data, ensuring normalization, deduplication, and semantic consistency.
• Vector Infrastructure: Manage pgvector, Azure AI Search, Redis vector indexing, and hybrid search layers.
• Data Governance: Implement zero-trust access, privacy controls, and compliance within AI context pipelines.
• Real-time Processing: Build event-driven architectures that continuously refresh embeddings and indexes.

Required Qualifications

• Deep experience with distributed data systems, SQL, and orchestration tools.
• Experience tuning high-throughput database infrastructure.
• Knowledge of Google’s GECX is a plus.
• Familiarity with chunking strategies and embedding models.

Skillset Requirements

• ETL & Data Modeling: Designing pipelines for structured/unstructured data, normalization, deduplication, and semantic consistency.
• Vector Databases: pgvector, Redis, Azure AI Search, hybrid search, and index optimization.
• Distributed Data Systems: Kafka, Spark, Flink, or similar event-driven architectures.
• Data Governance: Zero-trust access, privacy controls, compliance, and auditability.
• Real-time Embedding Updates: Event-driven refresh pipelines for RAG and agent memory systems.
• Chunking & Embeddings: Semantic chunking, metadata tagging, and embedding model selection.
• Search Infrastructure: BM25, hybrid search, inverted indexes, and ranking algorithms.
• Performance Tuning: High-throughput read/write optimization.
• Data Quality & Lineage: Validation, schema enforcement, and lineage tracking (e.g., Great Expectations, OpenLineage).