Senior Data Engineer

Iris Software Inc.; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

Iris Software Inc. in Toronto seeks a Senior Data Engineer to design and optimize production-grade data pipelines and data products on lakehouse platforms using Apache Iceberg.

You will build logical and physical data models, implement batch and real-time ingestion with Spark/PySpark and Kafka, manage Iceberg schemas, and ensure data quality, governance, and regulatory reporting. This role requires strong collaboration with architects and stakeholders.

Qualifications

• 10+ years in data engineering, data modeling and warehousing.
• Experience building large-scale data transformation initiatives.
• Proficiency with modern lakehouse platforms and governance.

Responsibilities

• Design, develop, and optimize production-grade data pipelines and data products.
• Create enterprise data models to support analytical and regulatory reporting.
• Develop batch and real-time ingestion using Spark/PySpark and Kafka.
• Define Iceberg table structures, partitioning, and schema evolution.
• Implement CDC, upserts, and time-travel data management patterns.
• Drive data quality controls and regulatory compliance requirements.

Skills

Data Engineering

Data Modeling

Data Warehousing

Batch & Real-time Pipelines

Iceberg

Spark/PySpark

Kafka

Regulatory/Compliance Awareness

Tools

Apache Iceberg

Spark

Kafka

Job description

Iris's Fortune 100 direct client is looking for Senior Data Engineer.

Location: Toronto Ontario/Berkeley Heights, NJ

Key Responsibilities:

• 10+ years of experience in Data Engineering, Data Modeling, Data Warehousing, and large-scale Data Transformation initiatives.
• Design, develop, and optimize production-grade data pipelines and data products using Apache Iceberg on modern lakehouse platforms.
• Create logical, physical, canonical, dimensional, and enterprise data models to support analytical, operational, and regulatory reporting requirements.
• Develop and implement batch and real-time ingestion frameworks using Spark/PySpark, Kafka, or similar distributed processing technologies.
• Define and manage Iceberg table structures, partitioning strategies, schema evolution, metadata management, and data lifecycle processes.
• Implement CDC, merge/upsert, incremental processing, time-travel, historical data retention, and ACID-compliant data management patterns.
• Optimize Iceberg performance through partition evolution, file compaction, small-file management, query tuning, and storage optimization techniques.
• Collaborate with architects, application teams, and business stakeholders to establish reusable engineering standards, data models, and governance frameworks.
• Design and implement data quality controls, reconciliation processes, lineage tracking, security policies, and regulatory compliance requirements.
• Troubleshoot and resolve complex data platform issues involving performance, scalability, concurrency, consistency, and operational reliability.
• Support CI/CD automation, testing frameworks, deployment processes, monitoring, alerting, and operational support for data pipelines and platforms.
• Drive modernization initiatives involving migration of legacy Data Warehouse, Hadoop, Netezza, or traditional ETL platforms to cloud-based lakehouse architectures using technologies such as Databricks, Snowflake, AWS, Trino/Presto, Athena, EMR, Glue Catalog, and Apache Iceberg. Preferred experience in Banking, Financial Services, Risk, Finance, Regulatory Reporting, or Capital Markets datasets.

About Iris Software Inc.

With 4,000+ associates and offices in India, U.S.A. and Canada, Iris Software delivers technology services and solutions that help clients complete fast, far-reaching digital transformations and achieve their business goals. A strategic partner to Fortune 500 and other top companies in financial services and many other industries, Iris provides a value-driven approach - a unique blend of highly-skilled specialists, software engineering expertise, cutting-edge technology, and flexible engagement models. High customer satisfaction has translated into long-standing relationships and preferred-partner status with many of our clients, who rely on our 30+ years of technical and domain expertise to future-proof their enterprises. Associates of Iris work on mission-critical applications supported by a workplace culture that has won numerous awards in the last few years, including Certified Great Place to Work in India; Top 25 GPW in IT & IT-BPM; Ambition Box Best Place to Work, #3 in IT/ITES; and Top Workplace NJ-USA.