Data Engineer – Contractor

Jobtailor; Deutschland; Remote; EUR 70.000 - 110.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a senior data engineer to design, develop, and maintain scalable data platforms using Databricks, Spark (PySpark), SQL, and Microsoft Fabric. You will build end-to-end data pipelines, lakehouse architectures, and semantic models while ensuring data quality and governance across enterprise datasets.

In this role you will lead engineering efforts, implement CI/CD and IaC practices, optimize performance, and collaborate with stakeholders across analytics, architecture, and

Qualifikationen

• Bachelor's degree required.
• 5+ years of experience in Data Engineering or related fields.
• Strong hands-on expertise in Databricks, Spark PySpark, SQL, Microsoft Fabric, Delta Lake, and Data Warehousing.
• Experience designing data pipelines, lakehouse architectures, and data integration frameworks.
• Proficiency with Databricks Workflows, Jobs, Automation, and Orchestration.
• Experience with CI/CD pipelines using Azure DevOps, GitHub, or equivalent.

Aufgaben

• Design, develop, and maintain scalable data solutions using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake.
• Build and optimize data ingestion, transformation, and orchestration pipelines for enterprise data platforms.
• Lead data engineering activities across requirements, design, development, testing, deployment, and support.
• Develop automation frameworks for data validation, testing, monitoring, deployment, and operational support.
• Validate data quality, completeness, consistency, and business rule compliance across datasets.
• Collaborate with business stakeholders, architects, analytics teams, and platform engineers.

Kenntnisse

Databricks

Apache Spark (PySpark)

SQL

Microsoft Fabric

Data Engineering

Data Platform Development

Data Integration Frameworks

Lakehouse Architectures

CI/CD

Data Quality Management

Data Governance

Row-Level Security (RLS)

DataOps/MLOps

Infrastructure as Code

Performance Tuning

Cloud platforms (Azure/AWS/GCP)

Ausbildung

Bachelor's Degree

Tools

Azure DevOps

GitHub

Power BI

Terraform

Bicep

Jobbeschreibung

• Design, develop, and maintain scalable data solutions using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake
• Build and optimize data ingestion, transformation, and orchestration pipelines for enterprise data platforms
• Develop automation frameworks for data validation, testing, monitoring, deployment, and operational support
• Lead data engineering activities across requirements, design, development, testing, deployment, and support
• Build and maintain automated QA and reconciliation frameworks for analytical and reporting solutions
• Validate data quality, completeness, consistency, and business rule compliance across enterprise datasets
• Develop and maintain semantic models, lakehouses, warehouses, and data marts within Microsoft Fabric
• Implement and validate Row-Level Security (RLS) and data governance standards
• Collaborate with business stakeholders, architects, analytics teams, and platform engineers
• Support CI/CD, DevOps, Infrastructure-as-Code, and deployment automation initiatives
• Drive performance optimization and cost efficiency across Databricks and Fabric environments
• Establish engineering best practices, coding standards, reusable frameworks, and governance processes

Requirements

• Bachelor's Degree
• 5+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Development
• Strong hands-on expertise in Databricks, Apache Spark (PySpark), SQL, Microsoft Fabric, Delta Lake, and Data Warehousing
• Experience designing and implementing data pipelines, ETL/ELT solutions, lakehouse architectures, and data integration frameworks
• Strong experience with Databricks Workflows, Jobs, Automation, and Orchestration
• Experience developing automated testing and validation frameworks for data platforms
• Hands-on experience implementing CI/CD pipelines using Azure DevOps, GitHub, or equivalent tools
• Strong understanding of data quality, reconciliation, metadata management, and governance practices
• Experience with performance tuning and optimization of Spark workloads
• Experience with Microsoft Fabric components including Data Factory, Lakehouse, Warehouse, Semantic Models, and Power BI
• Preferred: Infrastructure as Code (Terraform, Bicep, ARM Templates), DataOps/MLOps, Power BI semantic models and DAX, enterprise data governance and security frameworks, cloud platforms such as Azure, AWS, or GCP, and technical leadership/mentoring
• Strong analytical and problem-solving skills
• Excellent communication and stakeholder management abilities
• Ability to lead initiatives independently and work effectively in cross-functional teams
• Strategic mindset to align technical solutions with business objectives

Core Competencies

Demonstrates expertise in designing and developing scalable data solutions using Databricks, SQL, and Spark, while ensuring data quality and governance. Proficient in building automated frameworks for data validation and CI/CD processes, with strong analytical skills to optimize performance and align technical solutions with business objectives.

Highest-signal resume keywords

• Databricks
• Apache Spark (PySpark)
• SQL
• Microsoft Fabric
• Data Engineering

Hard Skills

• Data Pipeline Design
• ETL/ELT Solutions
• Data Warehousing
• Data Quality Management
• Performance Tuning
• Automation Frameworks
• Data Governance
• Row-Level Security (RLS)
• DataOps/MLOps
• Infrastructure as Code

Soft Skills

• Analytical Skills
• Problem-Solving
• Communication
• Stakeholder Management
• Leadership

Certifications & Qualifications

• Bachelor's Degree

Industry Keywords

• Data Engineering
• Data Platform Development
• Data Integration Frameworks
• Lakehouse Architectures
• CI/CD

Tools & Technologies

• Azure DevOps
• GitHub
• Power BI
• Terraform
• Bicep