Data Engineer

InterEx Group; Noord-Holland; On-site; EUR 65,000 - 90,000; Full time

Job summary

InterEx Group is seeking a Data Engineer to extract, transform, load and cleanse data, building scalable pipelines and data structures for analysis in a financial services context.

You will collaborate with architects, data governors and data scientists to align requirements, develop performance-focused code in cloud and on-prem platforms, and contribute to a governed data catalogue.

Qualifications

• Bachelor’s or Master’s Degree in Computer Science preferred or equivalent combination of education and experience.
• Minimum 4 years of experience as a data engineer.
• Experience in (captive) financial services and understands the processes and execution models in financial services is an asset.
• Experience in Azure SQL, Azure Data Factory, Azure DevOps, Databricks, Delta Lake is a must, certification is recommended.
• Experience in working with high-volume heterogeneous data sets
• Experience in agile software processes and data-driven development.
• Experience with data lakes and dimensional models.

Responsibilities

• Works and experiments with state-of-the-art data storage and processing technologies.
• Builds large-scale batch and real-time data pipelines with data processing frameworks in cloud platforms or on-premise
• Leverages best practices in continuous integration and delivery
• Pre-process, clean and structures data to facilitate data exploration and advanced analytics/Machine Learning activities
• Learns and uses modern data preparation, integration and AI-enabled metadata management tools and techniques.
• Tracks data consumption patterns, working with data governance and information stewards...
• Performs intelligent sampling and caching.
• Recommends and implements automation in existing and future integration flows.
• Helps improve data quality by driving advancements in data acquisition, testing and tooling
• Works in cross-functional agile teams to continuously experiment, iterate and deliver on new product objectives
• Analyzes requested changes to fill in the details fitting in the complete picture and while doing that aligns with all parties involved

Skills

ETL/ELT

Data pipelines

Azure data tools

SQL & Databases

Analytics mindset

Education

Bachelor’s degree in Computer Science

Master’s degree (preferred)

Tools

Azure SQL

Azure Data Factory

Azure DevOps

Databricks

Delta Lake

Job description

The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis.

The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code in analytical and data platforms. Liaise with enterprise architects, data governors, and subject matter experts to fill any gaps in the requirements or design.

Core Responsibilities:

• Works and experiments with state-of-the-art data storage and processing technologies.
• Builds large-scale batch and real-time data pipelines with data processing frameworks in cloud platforms or on-premise
• Leverages best practices in continuous integration and delivery
• Pre-process, clean and structures data to facilitate data exploration and advanced analytics/Machine Learning activities
• Learns and uses modern data preparation, integration and AI-enabled metadata management tools and techniques.
• Tracks data consumption patterns, working with data governance and information stewards within these teams, and participates in vetting and promoting content created in the business and by data scientists to the curated data catalogue for governed reuse
• Performs intelligent sampling and caching.
• Recommends and implements automation in existing and future integration flows.
• Helps improve data quality by driving advancements in data acquisition, testing and tooling
• Works in cross-functional agile teams to continuously experiment, iterate and deliver on new product objectives
• Analyzes requested changes to fill in the details fitting in the complete picture and while doing that aligns with all parties involved (Architects, Data Governance, …), either as preparation for yourself or for data engineer colleagues, before starting the implementation.

Education & Experience:

• Bachelor’s or Master’s Degree in Computer Science preferred or equivalent combination of education and experience
• Minimum 4 years of experience as a data engineer
• Experience in (captive) financial services and understands the processes and execution models in financial services is an asset
• Experience in Azure SQL, Azure Data Factory, Azure DevOps, Databricks, Delta Lake is a must, certification is recommended
• Experience in working with high-volume heterogeneous data sets 2
• Experience in agile software processes and data-driven development
• Experience with data lakes and dimensional models

Required Competencies and Skills:

• Azure Data Engineer Associate certification is recommended
• Databricks Data Engineer Professional certification is recommendedStrive for clean, reusable and performant code
• Skilled ETL/ELT developer
• Strong understanding of data and information
• Business savvy, understand information from a business perspective and use that to more effectively develop enabling technologies
• Strong problem-solving and analytical skills
• Goal-oriented and positive attitude to deliver against targets
• Able to work independently and proactively
• Global, open, innovative, holistic and goal-oriented mindset
• Ability to set and meet deadlines
• Ability to work in high-pressure situations
• Excellent attention to detail
• Advanced level of English, oral and written