testsdfsd

Nessie; Manchester; Hybrid; GBP 40,000 - 60,000; Full time

Job summary

A leading technology firm in Manchester is seeking a Data Developer to implement data ingestion and integration solutions for their simulation platform. The role involves working with data systems and architectures, ensuring data quality, and creating custom data models for analytics. Applicants should have a degree in a STEM discipline and experience in data-centric roles. This position offers a hybrid work environment and comprehensive benefits, including health insurance for employees and dependents.

Benefits

Extended health care

Dental insurance

Life insurance

Long-term disability insurance

Qualifications

• Experience in a data centric development role with an understanding of data system architectures.
• Experience designing and implementing data pipeline solutions accessing data from big data frameworks.
• Data stack proficiency in SQL, Python, and Java.

Responsibilities

• Implement data ingestion and integration solutions to support the simulation platform.
• Manage and build ETL processes and ensure data quality monitoring.
• Create custom data models for automated analysis and reporting.

Skills

SQL

Python

Java

API development utilizing GraphQL

Data analytics engineering

ETL solutions architecture

Kafka or RabbitMQ

Education

Degree in STEM related discipline (Engineering or Computer Science)

Tools

AWS

GCP BigQuery

SQL Server

PostgreSQL

Databricks

MLflow

Airbyte

Job description

Company Overview

Stream Systems (www.streamsystems.ca) is a leading-edge technology company that enables enterprises to optimize and make better decisions. Our SimOpti intelligence platform brings AI, machine learning and simulation to power business intelligence across complex operations, in any industry.

Position Summary

Reporting to the Software Development Manager, we are looking for a Data Developer experienced in continuous data process improvement across a range of data systems and subsystems. For this role, you will work closely with our data producers and consumers to enable Stream to automate data provisioning across data domains for multiple data use cases. Your data engineering expertise will implement data architectures, most importantly data mesh. Your ability to implement data technologies quickly and efficiently for any data layer is critical to our agile data flow.

What you’ll be doing

The role’s direct responsibilities include:

• Implement data ingestion and data integration solutions to support Stream’s simulation platform on AWS as part of a data mesh strategy.
• Manage and build ETL processes, extraction & integration of data from various data sources enabling a continuous data value stream.
• Oversee data quality monitoring, including processing, management and cleaning of data flows leveraged for analytics purposes while ensuring federated governance.
• Create custom data models for automated analysis, training machine learning, and enabling embedded BI reporting with row level security.
• Leverage best practices and best-in-class products and services to build cost-effective, scalable, and reliable data pipelines, data domains and data consumer API’s.
• Define and manage relational and distributed semi-structured NoSQL data warehouses and schemas that form the core of data mesh or fabric architectures.
• Translate data requirements into detailed designs and pipeline architecture, ensuring proper documentation of methodologies including deployable data services with a data mesh.
• Define best practices around data lineage, versioning, services deployment, extraction, modeling, consumption & governance at Stream.

Requirements

To ensure success in the role, you will need the following:

• Experience and formal qualifications in a STEM related discipline, such as Engineering or Computer Science, or certification and extensive work experience in a data centric development role with an understanding of how to implement data system architectures (distributed data mesh preferred)
• Experience designing and implementing data pipeline solutions accessing data from big data frameworks (AWS, GCP BigQuery) as well as hybrid architectures with on-premise data connectors (SQL Server, PostgreSQL, Historian, Timeseries)
• Data stack working proficiency in SQL, Python, Java, and API development utilizing GraphQL. Experience with messaging platforms like Kafka or RabbitMQ, data ingestion tools such as Airbyte, DBT, Databricks, MLflow an asset.
• Experience in data analytics engineering, orchestrating data pipelines, performance tuning, and script and test automation development.
• Experience architecting ETL solutions for a wide variety of sources, data abstractions, and data pipelines for real-time, streaming, batch, and on-demand workloads.
• Experience with data security strategies, configuring security for reporting, and certifications such as SOC 2.

Work environment/Hours/Location

This is a full-time position based on a 40-hour work week. We are driven by objectives, however, not timesheets. Stream’s head office is location in Calgary, Alberta with remote workers located across the country in British Columbia, Ontario, and Quebec. We are happy to provide a hybrid work environment, enabling you to work remotely as needed. We encourage a strong collaborative culture and provide workspaces in office for team collaboration, design, planning sessions and social activities.

Benefits

Company benefits are available to our full-time, permanent employees and include extended health care, dental, long-term disability, AD&D, and life insurance for you and your dependents. The real benefits, in our opinion exist in the ability to become part of a flexible, passionate, and dedicated team where we continue to learn from one another every day. If you love that small-company feel with big-company goals, Stream just might be a fit for you.

Come do great things with us!