Remote Databricks Platform Architect (Data/AI)

Intersources Inc; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Intersources Inc. seeks a Databricks Engineer to design, build, and operate a Databricks-based Data & AI platform using Medallion Architecture for scalable analytics.

You will engineer end-to-end data pipelines, ingest data from PeopleSoft, Salesforce, and D2L; enable governance, security, and ML readiness; work in a cloud environment with Unity Catalog. This is a remote, 1-year contract with laptop usage permitted.

Qualifications

• Hands-on Databricks, Delta Lake and Spark for large-scale data engineering.
• ELT pipeline design, orchestration, and monitoring in cloud environments.
• Implementing Medallion Architecture with data versioning and schema enforcement.
• Proficiency in SQL, Python or Scala for data transformations.
• Integrating enterprise systems like PeopleSoft, Salesforce, D2L into a central platform.
• Familiarity with governance, lineage and metadata management.

Responsibilities

• Design, implement, and optimize end-to-end Databricks data pipelines following Medallion Architecture.
• Ingest data from enterprise systems (PeopleSoft, D2L, Salesforce) via APIs/JDBC.
• Develop data quality checks, governance and monitoring; track ETL performance; use Unity Catalog for governance.
• Enforce data security, privacy controls, masking and anonymization; collaborate with security teams.
• Provide data for model training; support MLflow lifecycle and MLOps workflows.
• Architect data lakes on ADLS or S3; build data marts and warehousing; optimize storage.
• Maintain technical docs, runbooks, and train stakeholders on Databricks and governance.
• Report weekly hours and progress; track deliverables and risks.

Skills

Databricks

Delta Lake

Apache Spark

ELT pipelines

Medallion Architecture

SQL

Python/Scala

Data governance

Data lineage

Metadata management

Tools

Unity Catalog

MLflow

MLOps

Job description

Intersources Inc. seeks a Databricks Engineer to design, build, and operate a Databricks-based Data & AI platform using Medallion Architecture for scalable analytics.

You will engineer end-to-end data pipelines, ingest data from PeopleSoft, Salesforce, and D2L; enable governance, security, and ML readiness; work in a cloud environment with Unity Catalog. This is a remote, 1-year contract with laptop usage permitted.