Senior Data Developer

Jobtailor; Toronto; On-site; CAD 120,000 - 180,000; Full time

Job summary

Jobtailor in Toronto, Canada seeks an experienced Data Platform Engineer to build and evolve a Databricks-based data platform powering product features, machine learning, and internal reporting.

You will design and scale data ingestion and transformation pipelines, model raw data into clean datasets, and improve observability, governance, security, and performance while collaborating with software, ML, product, and analytics teams in an agile environment.

Qualifications

• 6+ years of experience as a data developer in a modern, production-grade cloud data platform (Databricks or similar).
• Experience operating under ambiguity and making technical tradeoffs that balance velocity and value delivery with maintainability and technical debt.
• Strong Python and SQL skills.
• Experience with CDC pipelines.
• Experience managing cloud infrastructure with Terraform (or similar).
• Experience building CI/CD workflows using DABs and GitHub Actions (or similar).
• Experience in query optimization, resource allocation and management, cost management, and data lake performance.
• Experience working in a fast-paced, agile environment.
• Expert level of English, both spoken and written, required.
• Knowledge of compliance and regulatory frameworks (e.g., GDPR, CCPA, SOC2, FedRAMP) considered for extra consideration.
• Experience with event-streaming with Kafka and Flink (or similar data-streaming architectures) considered for extra consideration.
• Backend software development experience, including application data storage and event-driven architecture considered for extra consideration.
• Experience building and operating vector databases and RAG considered for extra consideration.

Responsibilities

• Build and evolve the Databricks-based data platform used for product features, machine learning, and internal reporting
• Design and develop reliable and scalable data ingestion and transformation pipelines
• Model raw data into clean, reliable datasets powering product features and internal analytics
• Build platform abstractions and tooling improving developer experience for data producers and consumers
• Improve observability, testing, and monitoring for reliability and performance
• Manage and contribute to data governance, security, compliance, and access controls
• Contribute to platform standards and architecture as the data platform scales
• Collaborate with software development, machine learning, product, and analytics teams to support data use cases

Skills

Databricks

Python

SQL

Cloud Infra

Terraform

GitHub Actions

CDC pipelines

Kafka

Flink

Agile

Tools

Databricks

Terraform

GitHub Actions

Kafka

Flink

Job description

• Build and evolve the Databricks-based data platform used for product features, machine learning, and internal reporting
• Design and develop reliable and scalable data ingestion and transformation pipelines
• Model raw data into clean, reliable datasets powering product features and internal analytics
• Build platform abstractions and tooling improving developer experience for data producers and consumers
• Improve observability, testing, and monitoring for reliability and performance
• Manage and contribute to data governance, security, compliance, and access controls
• Contribute to platform standards and architecture as the data platform scales
• Collaborate with software development, machine learning, product, and analytics teams to support data use cases

Requirements

• 6+ years of experience as a data developer in a modern, production-grade cloud data platform (Databricks or similar equivalent)
• Experience operating under ambiguity and making technical tradeoffs that balance velocity and value delivery with maintainability and technical debt
• Strong Python and SQL skills
• Experience with CDC pipelines
• Experience managing cloud infrastructure with Terraform (or similar infrastructure-management equivalent)
• Experience building CI/CD workflows using DABs and GitHub Actions (or similar)
• Experience in query optimization, resource allocation and management, cost management, and data lake performance
• Experience working in a fast-paced, agile environment
• Expert level of English, both spoken and written, required
• Knowledge of compliance and regulatory frameworks (e.g., GDPR, CCPA, SOC2, FedRAMP) considered for extra consideration
• Experience with event-streaming with Kafka and Flink (or similar data-streaming architectures) considered for extra consideration
• Backend software development experience, including application data storage and event-driven architecture considered for extra consideration
• Experience building and operating vector databases and RAG considered for extra consideration

Core Competencies

Demonstrates expertise in building and evolving Databricks-based data platforms, focusing on data ingestion, transformation, and governance. Proficient in Python, SQL, and managing cloud infrastructure, with a strong emphasis on compliance and performance optimization.

Highest-signal resume keywords

• Databricks Data Platform Development
• Python Programming
• SQL Proficiency
• Cloud Infrastructure Management
• CI/CD Workflow Development

ATS Optimization Keywords

Hard Skills

• Data Ingestion Pipelines
• Data Transformation
• Query Optimization
• Event-Streaming with Kafka
• CDC Pipelines
• Data Lake Performance
• Vector Databases
• Event-Driven Architecture
• Terraform
• GitHub Actions

Soft Skills

• Collaboration
• Problem-Solving
• Adaptability

Industry Keywords

• Data Governance
• Compliance
• GDPR
• CCPA
• SOC2
• FedRAMP
• Agile Environment

Tools & Technologies

• Databricks
• Terraform
• GitHub Actions
• Kafka
• Flink