Databricks Architect – Informatica, Spark & PySpark

Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!; Brampton; Hybrid; CAD 110,000 - 150,000; Full time

Job summary

Astra-North Infoteck Inc. in Brampton, Ontario, invites a Databricks Architect to design scalable data platforms using Databricks Lakehouse, Delta Lake, Spark, and PySpark, and to architect batch and real-time data pipelines across cloud environments.

You will lead ETL modernization and migrations from Informatica PowerCenter/IICS/IDMC to Databricks, while defining governance, security, performance optimization, and best practices for enterprise data solutions.

Qualifications

• Strong expertise in Databricks, Spark, PySpark, SQL, Delta Lake, and Unity Catalog.
• Hands-on experience with Informatica PowerCenter and/or IICS/IDMC.
• Experience in data warehousing, data modeling, and cloud platforms (Azure preferred).
• Knowledge of CI/CD, Git, and Agile methodologies.

Responsibilities

• Design and implement scalable data platforms using Databricks Lakehouse, Delta Lake, Spark, and PySpark.
• Architect batch and real-time data pipelines on Azure/AWS/GCP.
• Lead ETL modernization and migration from Informatica PowerCenter/IICS/IDMC to Databricks.
• Define data governance, security, performance optimization, and best practices.
• Collaborate with business and technical teams to deliver enterprise data solutions.

Skills

Databricks

Spark

PySpark

SQL

Delta Lake

Unity Catalog

Informatica PowerCenter

IICS/IDMC

data warehousing

data modeling

Azure

Git

CI/CD

Agile methodologies

Cloud certifications

Tools

Informatica PowerCenter

IICS/IDMC

Git

CI/CD pipelines

Job description

Job Description:

Databricks Architect (with Informatica)-Databricks, Informatica PowerCenter and/or IICS/IDMC, Spark, PySpark, SQL, Delta Lake, Unity Catalog.

Brampton, ON - Hybrid (4 Days WFO)

Role Descriptions:

Design and implement scalable data platforms using Databricks Lakehouse| Delta Lake| Spark| and PySpark.Architect batch and real-time data pipelines on Azure/AWS/GCP.Lead ETL modernization and migration from Informatica PowerCenter/IICS/IDMC to Databricks.Define data governance| security| performance optimization| and best practices.Collaborate with business and technical teams to deliver enterprise data solutions.

Essential Skills:

• Required Skills:-Strong expertise in Databricks| Spark| PySpark| SQL| Delta Lake| Unity Catalog.
• Hands-on experience with Informatica PowerCenter and/or IICS/IDMC.
• Experience in data warehousing| data modeling| and cloud platforms (Azure preferred).
• Knowledge of CI/CD| Git| and Agile methodologies.
• Preferred Skills:-Databricks and Cloud certifications.
• Experience with Snowflake| Azure Synapse| Kafka| or Data Governance tools.