Data Engineer (Databricks, Azure)

MW Consulting; United States; Hybrid; USD 46,000 - 72,000; Full time

Job summary

MW Consulting is seeking a Data Engineer to work on projects for Avanade Poland, focusing on Databricks and Azure to build scalable data pipelines and end-to-end ETL processes.

You will collaborate with data scientists and engineers, contribute to data architectures, and have opportunities to lead projects while exploring AI engineering paths.

Qualifications

• Experience designing scalable data pipelines and ETL processes.
• Proficient with Databricks and Azure data services.
• Strong Python and SQL skills; English B2+.

Responsibilities

• Design, build, and maintain scalable data pipelines.
• Develop data architectures and end-to-end ETL processes.
• Collaborate with data scientists and engineers for data integration.
• Apply best engineering practices across data solutions.
• Lead a project as team lead when needed.
• Develop skills in AI Engineering.

Skills

Databricks

Azure Cloud

Python

PySpark

SQL

ETL

Data modeling

Medallion architecture

Data warehousing

Data governance

Terraform

English B2

Tools

Docker

Kubernetes

easyOCR

Job description

We’re looking for specialists open to cooperate on B2B basis, for our customer – Avanade Poland.Avanade website; Privacy policy

This role is available for both specialists open to work remotely (within Poland) or in hybrid mode (offices in Wroclaw and Krakow).

Position

As Data Engineer, you will be working in projects for various domains and industries, with tech stack focused mostly around Databricks and Azure cloud.

Tasks

• Designing, building, and maintaining scalable, efficient, and reliable data pipelines


• Developing data architectures and ETL processes end to end


• Collaborating with data scientists, analysts, and engineering teams to ensure effective data integration and optimization


• Applying best engineering practices across data solutions


• Optimizing data workflows and ensuring the reliability and performance of data systems (for Senior level)


• Taking on a Team Lead role within a project when needed


• Developing your skills in AI Engineering and exploring opportunities in this area

Required skills

• Databricks


• Azure cloud and services (e.g. Azure Data Factory, Data Lake, SQL Database, Azure Databricks)


• Python and PySpark


• SQL


• Data Engineering concepts such as ETL, data modeling, medallion architecture, data warehousing, data governance


• Terraform


• English (at least B2)

Nice to have

• Docker and Kubernetes


• Knowledge of OCR and corresponding libraries (e.g. easyOCR)


• Any experience in data science area

If you want to ask any questions before applying, feel free to reach out via LinkedIn!