Senior Data Engineer (Big Data Pipelines)

Placements24; Kimberley; Remote; CAD 120,000 - 180,000; Full time

Benefits offered by this job

Health benefits

Performance bonuses

Remote work

Professional development

Job summary

Placements24 is seeking a Senior Data Engineer for a fully remote role. You will design and optimize scalable data pipelines, build data lakes and warehouses on cloud platforms, and implement ETL/ELT processes to support advanced analytics and ML initiatives.

You will work with vast datasets and collaborate with data scientists to ensure data quality and accessibility. Strong remote communication and self-discipline are essential for success in this distributed environment.

Qualifications

• 6+ years of data engineering experience.

Responsibilities

• Design, develop, and optimize scalable data pipelines for batch and real-time data processing.
• Build and maintain data warehousing solutions and data lakes on cloud platforms.
• Implement ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and reliability across all data systems.
• Collaborate with data scientists and analysts to understand their data requirements and provide efficient data access.
• Develop and maintain data infrastructure, monitoring performance and proactively addressing issues.
• Implement data governance best practices and ensure compliance with data privacy regulations.
• Automate data processes and workflows using scripting and orchestration tools.
• Evaluate and integrate new data technologies and tools to improve data processing capabilities.
• Participate in code reviews and contribute to architectural decisions for data platforms.
• Effectively communicate technical solutions and data insights within a remote team environment.

Skills

Big data frameworks

SQL

NoSQL databases

Python/Scala/Java

Cloud data platforms

Airflow/Prefect

Data modeling

Remote collaboration

Problem solving

Education

Bachelor's degree in CS/Engineering

Tools

Spark

Hadoop

Hive

AWS EMR

Redshift

BigQuery

Airflow

Prefect

Job description

About the Role

Our client, a cutting-edge data analytics firm, is seeking a highly skilled Senior Data Engineer to join their distributed, fully remote team. This role is critical for designing, building, and maintaining robust data pipelines and infrastructure that support the company's advanced analytics and machine learning initiatives. You will work with vast datasets, implement efficient data processing solutions, and ensure data quality and accessibility for data scientists and analysts. As a remote-first organization, strong communication, self-discipline, and experience with collaborative remote tools are paramount. This is an exceptional opportunity to work with state-of-the-art big data technologies from anywhere.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines for batch and real-time data processing.
• Build and maintain data warehousing solutions and data lakes on cloud platforms.
• Implement ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and reliability across all data systems.
• Collaborate with data scientists and analysts to understand their data requirements and provide efficient data access.
• Develop and maintain data infrastructure, monitoring performance and proactively addressing issues.
• Implement data governance best practices and ensure compliance with data privacy regulations.
• Automate data processes and workflows using scripting and orchestration tools.
• Evaluate and integrate new data technologies and tools to improve data processing capabilities.
• Participate in code reviews and contribute to architectural decisions for data platforms.
• Effectively communicate technical solutions and data insights within a remote team environment.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• Minimum of 6 years of experience in data engineering or a related role.
• Proven experience in designing and building large-scale data pipelines using big data technologies (e.g., Spark, Hadoop, Hive).
• Proficiency in SQL and experience with various relational and NoSQL databases.
• Experience with cloud data platforms (e.g., AWS EMR, Redshift, Glue; Azure Data Factory, Synapse; GCP Dataflow, BigQuery).
• Strong programming skills in Python, Scala, or Java.
• Experience with data warehousing concepts and ETL/ELT tools.
• Familiarity with data orchestration tools like Airflow or Prefect.
• Understanding of data modeling and database design principles.
• Excellent problem-solving, analytical, and debugging skills.
• Strong communication and collaboration skills, essential for remote work.
• Demonstrated ability to work independently and manage priorities effectively in a remote setting.

Benefits

• Our client offers a highly competitive salary, comprehensive health benefits, performance bonuses, and ample opportunities for professional development, including advanced training and certifications.
• This is a fully remote position, allowing for excellent work-life balance and flexibility.
• The company cultivates a strong remote work culture with regular virtual team meetings, collaborative online platforms, and opportunities for significant career growth.
• Join a leading data-driven organization and build the future of data infrastructure from anywhere.