Remote Data Engineer: Scalable Big Data Pipelines

Crossing Hurdles; United States; Remote; CAD 41,000 - 110,000; Part time

Job summary

Crossing Hurdles is seeking a Big Data Engineer for an hourly remote contract. You will design, build, and maintain scalable data pipelines and architectures to support business objectives across teams.

You will implement data integration and processing using Python and big data technologies, develop distributed storage solutions, monitor performance, enforce data quality and governance, and document solutions for both technical and non-technical stakeholders.

Qualifications

• Experience building large-scale data pipelines.
• Advanced proficiency in Python for data processing.
• Strong knowledge of relational and NoSQL databases.
• Experience with distributed data processing frameworks such as Hadoop, Spark, or Flink.
• Understanding of data modeling, ETL processes, and data warehousing principles.
• Excellent written and verbal communication skills for technical and non-technical audiences.
• Ability to work independently with strong attention to detail.

Responsibilities

• Design, build, and maintain scalable big data pipelines and architectures.
• Collaborate with cross-functional teams to understand data requirements and business objectives.
• Implement data integration, transformation, and processing solutions using Python and big data technologies.
• Develop, manage, and optimize distributed databases and storage systems.
• Monitor, troubleshoot, and improve data systems for high availability and performance.
• Enforce data quality, security, and governance standards across data solutions.
• Document technical solutions and communicate complex data concepts clearly.

Skills

Python

Big data pipelines

NoSQL databases

Hadoop

Spark

Flink

Data modeling

ETL processes

Data warehousing

Communication

Independent work

Tools

Hadoop

Spark

Flink

Job description

Crossing Hurdles is seeking a Big Data Engineer for an hourly remote contract. You will design, build, and maintain scalable data pipelines and architectures to support business objectives across teams.

You will implement data integration and processing using Python and big data technologies, develop distributed storage solutions, monitor performance, enforce data quality and governance, and document solutions for both technical and non-technical stakeholders.