Data Platform Developer

Computationaldesign; Quebec; On-site; CAD 110,000 - 170,000; Full time

Job summary

Newforma is seeking a talented Data Platform Developer to join our Platform Engineering team and architect the data foundation powering our AI-driven capabilities and analytics. You will design modern data architectures on AWS, establish medallion/lakehouse patterns, and build event-driven pipelines that process billions of documents in real time.

Collaborate with AI and software architects to align data strategy with platform goals, implement governance, and deliver scalable analytics

Qualifications

• Minimum 5+ years in data engineering or analytics engineering.
• Hands-on AWS data services experience.
• Experience with medallion architecture or lakehouse platforms.
• Proficiency in SQL and data modeling.

Responsibilities

• Design data architectures and governance across the organization.
• Build real-time data pipelines and CDC patterns.
• Collaborate with AI engineering to optimize data for ML.
• Develop BI-ready data warehouses and data marts.
• Ensure data quality and security.

Skills

Data architecture

Python

SQL

AWS data services

Event-driven design

Data modeling

Communication

Team collaboration

Problem solving

Tools

AWS Glue

AWS Step Functions

Lambda

EMR

Redshift

Athena

Job description

We're seeking a talented Data Platform Developer to join our Platform Engineering team and architect the data foundation that will power Newforma's next generation of AI-driven capabilities and analytics. You'll design and implement modern data architectures including medallion/lakehouse patterns, build event-driven data pipelines that process billions of project documents and communications in real-time, and create the analytics infrastructure that enables both business intelligence and AI/ML initiatives. This is a foundational role at an exciting time—as we migrate to AWS and invest heavily in AI, you'll establish the data practices and infrastructure that will serve the company for years to come.

In this role, your responsibilities will include:

Data Architecture & Strategy Design and implement medallion architecture (bronze, silver, gold layers) or lakehouse patterns on AWS to organize and transform data at scale Establish data modeling standards, governance practices, and quality frameworks across the organization Define data retention, archival, and lifecycle management policies for massive volumes of project data Create reference architectures and best practices for data engineering across teams Partner with the Director of AI Engineering to design data pipelines optimized for AI/ML workloads including vector embeddings and model training Work with the Lead Software Architect to ensure data architecture aligns with overall platform strategy Design data schemas and structures that support both analytical queries and AI applications Event-Driven Architecture Design and implement event-driven data architectures using AWS EventBridge, Kinesis, MSK (Kafka), SNS, and SQS Build real-time data streaming pipelines that capture, process, and route project events across the platform Architect event schemas and patterns for domain events (document uploads, email filing, RFI submissions, etc.) Implement change data capture (CDC) patterns to stream database changes to data lakes and analytics systems Design event-driven workflows that trigger AI processing, notifications, and downstream system updates Establish event governance including versioning, documentation, and monitoring Optimize event processing for low latency and high throughput at scale Data Pipeline Development Build robust, scalable ETL/ELT pipelines using AWS Glue, Step Functions, Lambda, and EMR Develop data transformation jobs that cleanse, enrich, and structure unstructured project data Implement data quality checks, validation rules, and monitoring throughout pipelines Create reusable pipeline components and frameworks that teams can leverage Optimize pipeline performance and cost efficiency for processing billions of documents Handle diverse data formats including emails, PDFs, CAD drawings, images, and structured databases Implement data lineage tracking and metadata management Analytics & Business Intelligence Design and build data warehouses and data marts using Amazon Redshift, Athena, or similar technologies Create dimensional models and star schemas optimized for analytical queries Build datasets and aggregations that power executive dashboards and operational reports Implement BI solutions using tools like QuickSight, Tableau, PowerBI, or similar platforms Partner with product and business teams to understand analytics requirements and deliver insights Create self-service analytics capabilities that empower teams to explore data independently Establish KPIs, metrics, and reporting frameworks for product and business analytics AI/ML Data Infrastructure Prepare and structure data to support AI initiatives including document classification, semantic search, and intelligent agents Build pipelines for generating and storing vector embeddings for RAG (Retrieval-Augmented Generation) systems Create training datasets and feature stores for machine learning models Implement data versioning and experiment tracking for AI/ML workflows Design scalable inference pipelines that serve AI models with fresh, contextualized data Collaborate with the AI Engineering team to optimize data formats and access patterns for LLM applications Data Operations & Monitoring Implement comprehensive monitoring, alerting, and observability for data pipelines and systems Build data quality dashboards and anomaly detection systems Create operational runbooks and documentation for data platform components Optimize costs across data storage, processing, and querying Ensure data security, encryption, and compliance with privacy regulations Participate in on-call rotation to support production data systems

Requirements for the position include:

• 5+ years of experience in data engineering, analytics engineering, or related roles
• Strong hands‑on experience with AWS data services including S3, Glue, Athena, Redshift, Kinesis, EventBridge, Lambda, and EMR
• Proven expertise designing and implementing event‑driven architectures using streaming technologies (Kafka/MSK, Kinesis, EventBridge)
• Experience building medallion architectures, lakehouse platforms, or similar modern data architectures (bronze/silver/gold patterns, Delta Lake, Iceberg)
• Proficiency with SQL and database design including both relational (PostgreSQL, MySQL) and analytical databases (Redshift, Snowflake)
• Strong programming skills in Python for data processing, transformation, and automation
• Experience with data orchestration tools such as Apache Airflow, AWS Step Functions, or Prefect
• Knowledge of data modeling techniques including dimensional modeling, star schemas, and data vault
• Experience with analytics and BI tools (QuickSight, Tableau, PowerBI, Looker) and building reports/dashboards
• Understanding of data quality, data governance, and master data management principles
• Familiarity with infrastructure‑as‑code (Pulumi, Terraform, CloudFormation) for managing data infrastructure
• Strong problem‑solving skills and ability to optimize complex data workflows
• Excellent communication skills with ability to explain technical concepts to diverse audiences
• Team player who collaborates effectively across engineering, product, and business teams

Nice to have qualifications for this position include:

• AWS certifications (AWS Data Analytics - Specialty, AWS Solutions Architect, or similar)
• Experience with Azure data services (Data Factory, Synapse, Event Hubs) and Azure-to-AWS data migrations
• Knowledge of real‑time stream processing frameworks (Apache Spark Streaming, Flink, Kafka Streams)
• Experience preparing data for AI/ML applications including vector databases (Pinecone, Weaviate, pgvector)
• Familiarity with document processing, OCR, and unstructured data extraction techniques
• Experience with data catalog and metadata management tools (AWS Glue Data Catalog, Alation, Collibra)
• Knowledge of .NET/C# and integrating data pipelines with .NET applications
• Understanding of SaaS multi‑tenancy patterns in data architecture
• Experience with data privacy and compliance frameworks (GDPR, SOC 2, CCPA)
• Background in the AECO industry or project management domain
• Familiarity with graph databases (Neptune, Neo4j) for relationship modeling
• Experience with serverless data architectures and cost optimization strategies
• Knowledge of dbt (data build tool) or similar transformation frameworks
• Bilingual in English and French.

Collaboration

Collaborate with other platform engineering team members to accomplish tasks Participate in agile ceremonies including daily stand‑ups, sprint planning, and retrospectives Work closely with development teams and with the software architect to establish good data engineering practices for newly developed features