BE Data Engineer - Consolia

Consolia; United States; Remote; USD 110,000 - 170,000; Full time

Benefits offered by this job

Full private health insurance

Job summary

Consolia is seeking a Data Engineer to own the evolution of its Databricks-based analytics platform. You will design and optimize data pipelines, ingest data from various sources, and build Bronze-Silver-Gold layers while collaborating with analysts and stakeholders across Europe and North America.

The role emphasizes scalable, cost-aware cloud data engineering, governance, and delivery of trusted datasets for BI consumption, with a strong focus on performance and quality.

Qualifications

• Proven experience as a Data Engineer in a modern data environment.
• Hands-on experience with Databricks, including AI/BI Dashboards and Genie
• Strong SQL skills and experience with relational data.
• Strong Python and/or PySpark experience.
• Experience designing, building and maintaining ELT/ETL data pipelines.
• Experience with orchestration tools such as Azure Data Factory or equivalent.
• Good understanding of medallion architecture (Bronze/Silver/Gold).
• Experience with cloud-based data storage and processing technologies.
• Understanding of data modelling for BI consumption.
• Experience with multiple data sources including APIs and files.
• Knowledge of performance optimization, scalability and cost management in cloud platforms.
• Experience with Git, Azure DevOps or similar tooling.
• Excellent communication skills to convey technical topics to non-technical stakeholders.

Responsibilities

• Design, build, maintain and optimise data pipelines within Azure Databricks.
• Take technical ownership of data ingestion from source systems into the Belgium data platform.
• Build and maintain Bronze, Silver and Gold data layers using a medallion architecture.
• Add and integrate new data sources, including internal systems, external datasets, APIs and file-based feeds.
• Improve pipeline performance, reliability, monitoring and observability.
• Identify opportunities to speed up data processing and reduce platform running costs.
• Support the transition from legacy reporting to a modern Databricks Lakehouse platform.
• Work with the Data Analyst to provide trusted datasets for Power BI reporting.
• Collaborate with business users to translate data requirements into robust solutions.
• Implement data quality checks and reconciliation processes.
• Document data flows, definitions, assumptions and technical design decisions.
• Contribute to engineering standards and reusable patterns.
• Work within DevOps practices and structured delivery methods.
• Support secure development practices and non-functional requirements.
• Proactively seek innovative solutions to improve the platform.

Skills

Data Engineer experience

Databricks

SQL

Python PySpark

Data modeling

Cloud data platforms

Stakeholder engagement

Tools

Databricks

Azure Data Factory

Delta Lake

Unity Catalog

Git

Azure DevOps

Power BI

Terraform

Azure Functions

Job description

This is a fantastic opportunity to work for a well renowned software development company!

Consolia is part of an international company that operate in several European countries such as Sweden, United Kingdom, Germany and Denmark. For the past ten years Consolia has established a strong presence in Serbia with a team of experienced and skilled developers.

With a decade long experience in a software development industry, we are a reliable, well-established and trustworthy partner to many companies. We have developed a strong team-based approach and together we create innovative software products for multiple markets.

As a BI Analyst you will be working with Europe’s largest veterinary services provider, and it is the fastest growing scale player, driving consolidation of a very fragmented market composed mainly of single practices or small veterinary groups.

The Company currently operates in 19 countries across Europe and North America consisting of: +9,000 vets who treat +8M pets annually +2,000 First Opinion Practices, hospitals and crematoria a fast growing online pharmacy and digital business with significant scope for further expansion into adjacencies.

The Company is owned by EQT Private Equity, Nestle, Silverlake and a number of minority shareholders many of which work in clinics across Europe.

We are transforming the way data is managed, consumed and leveraged across the organisation.

As part of this exciting journey, we are implementing a Databricks platform that will become the foundation for analytics, reporting and data-driven decision making across Belgium.

We are looking for a Data Engineer to take ownership of the technical evolution of this platform. You will be responsible for building and optimising scalable data pipelines, integrating new data sources, ensuring data quality and reliability, and continuously improving platform performance and cost efficiency.

Working closely with the Data Analyst, business stakeholders and Group Data teams, you will play a critical role in enabling high-quality analytics while establishing best practices for data engineering, automation and governance.

You will report into the Performance & Business Development Director. (TBC)

JOB DESCRIPTION:

As a Data Engineer, you will be responsible for:

• Design, build, maintain and optimise data pipelines within Azure Databricks.
• Take technical ownership of data ingestion from source systems into the Belgium data platform.
• Build and maintain Bronze, Silver and Gold data layers using a medallion architecture.
• Add and integrate new data sources, including internal systems, external datasets, APIs and file-based feeds.
• Improve pipeline performance, reliability, monitoring and observability.
• Identify opportunities to speed up data processing and reduce platform running costs.
• Support the transition from legacy reporting and data infrastructure to a modern Databricks Lakehouse platform.
• Work closely with the Data Analyst to provide trusted, consumption-ready datasets for Power BI reporting and self-service analytics.
• Collaborate with business users and SMEs to understand data requirements and translate them into robust technical solutions.
• Implement data quality checks, validation rules and reconciliation processes.
• Support documentation of data flows, business definitions, assumptions and technical design decisions.
• Contribute to the definition of engineering standards, reusable patterns and platform best practices.
• Work within DevOps ways of working, using version control, deployment practices and structured delivery methods.
• Support secure development practices, including access management, scalability, performance and other non-functional requirements.
• Proactively seek innovative solutions to improve the platform, automate manual processes and increase the value delivered from data.

WHAT ARE WE LOOKING FOR?

We are looking for a technically skilled and business-aware Data Engineer with strong experience in modern cloud-based dataplatforms and a practical mindset for delivery.

You should be comfortable working with data storage, processing, modelling and orchestration concepts, and have experience building and maintaining production-grade data pipelines. Experience with Azure Databricks, Azure Data Factory and cloud-based data engineering is particularly important for this role.

You will act as a key interface between business stakeholders, the Belgium Data Analyst, technology teams and Group Data teams, so excellent communication and stakeholder engagement skills are essential.

You should be confident working in an environment where the data platform is evolving, priorities may change, and pragmatic decisions are needed to balance short-term delivery with long-term platform health.

REQUIREMENTS:

• Proven experience as a Data Engineer or Analytics Engineer in a modern data environment.
• Hands-on experience with Databricks, including AI/BI Dashboards and Genie
• Strong SQL skills and experience working with relational data.
• Strong Python and/or PySpark experience.
• Experience designing, building and maintaining ELT/ETL data pipelines.
• Experience with orchestration tools such as Azure Data Factory or equivalent.
• Good understanding of medallion architecture, including Bronze, Silver and Gold data layers.
• Experience working with cloud-based data storage and processing technologies.
• Understanding of data modelling and building consumption-ready datasets for BI and analytics.
• Experience integrating data from multiple structured and semi-structured sources, including APIs, JSON, databases and files.
• Good understanding of performance optimisation, scalability and cost management in cloud data platforms.
• Experience with Git, Azure DevOps or similar version control and delivery tooling.
• Strong problem-solving skills and ability to troubleshoot pipeline, data quality and performance issues.
• Excellent communication skills and ability to explain technical topics to non-technical stakeholders.
• Ability to work collaboratively with analysts, architects, infrastructure teams and business users.
• Excellent communication skills in Dutch and English
• Experience with Delta Lake and Databricks optimisation techniques.
• Experience with Unity Catalog or data governance tooling.
• Exposure to Power BI and analytical consumption patterns.
• Experience working in a multi-site, multi-country or federated data environment.
• Experience with data quality frameworks, metadata management and lineage.
• Knowledge of Infrastructure-as-Code tools such as Bicep or Terraform.
• Experience with APIs, Azure Functions or event-driven data architectures.
• Experience in healthcare, veterinary, retail or service-based organisations.
• Knowledge of Dutch, French and/or English would be an advantage, depending on the stakeholder environment.

COMPANY OFFERS:

• Challenging and interesting work in an international environment
• Opportunity to work in a small team of highly professional people
• Engaging work atmosphere with proactive, friendly and motivated colleagues
• Master and development of opportunities
• Regular team building activities
• Full private health insurance