Senior Data Engineer - IoT & Real-Time Analytics (Remote)

Pearlfisher; Vancouver; Hybrid; CAD 120,000 - 180,000; Full time

Benefits offered by this job

RRSP matching

Unlimited vacation

Work-from-home flexibility

Spacious downtown Vancouver office

Job summary

Pearlfisher in Vancouver is seeking a senior Data Engineer to join a high-growth Clean Energy software team. You’ll design and build data pipelines, manage large time-series datasets, and deploy ML-enabled data intelligence in the cloud.

You will collaborate with data scientists and engineers to deliver scalable, real-time analytics using Kafka, Kinesis, Spark, and AWS services. A passion for data quality and shipping robust systems is essential.

Qualifications

• Bachelor’s Degree in Computer Science or related field.
• Experience building scalable infrastructure software or distributed systems for commercial online services.
• Production experience with orchestration and batch processing technologies like Apache Airflow.
• 3+ years of production experience with Python; Java is a plus.
• 3+ years of experience with large-scale data warehousing, mining or analytic systems.
• 3+ years of building big data pipelines using Kafka or Kinesis.
• 3+ years of Experience with AWS services (EC2, S3, Lambda, Kinesis, EMR, Redshift).
• Production experience with real-time streaming technologies (Flink, Spark, Samza).

Responsibilities

• Establish scalable, automated processes for large-scale data analyses, model development, validation and implementation.
• Productize ML applications.
• Propose and validate batch and real-time streaming solutions.
• Analyze and extract information from large streaming data to create an event-driven data pipeline.
• Collaborate with scientists and engineers to deploy new features.

Skills

Python

SQL

NoSQL

Data pipelines

Machine Learning

Leadership

AWS

Kubernetes

Education

Bachelor's degree in Computer Science or related field

Tools

Airflow

Kafka

Kinesis

Spark

Flink

Kubernetes

Job description

Pearlfisher in Vancouver is seeking a senior Data Engineer to join a high-growth Clean Energy software team. You’ll design and build data pipelines, manage large time-series datasets, and deploy ML-enabled data intelligence in the cloud.

You will collaborate with data scientists and engineers to deliver scalable, real-time analytics using Kafka, Kinesis, Spark, and AWS services. A passion for data quality and shipping robust systems is essential.