Data Engineer: Delta Lake & Spark Expert (Remote)

bridge351; Lisboa; Presencial; EUR 50 000 - 80 000; Tempo integral

Descrição da oferta de emprego

Knowledge and Skills

• Core Engineering Languages: Expert proficiency in SQL and Python (PySpark); strong working knowledge of Java for backend data processing or SDK integrations.
• Generic Data Engineering: Solid understanding of Software Engineering principles applied to data (DRY code, modularity, error handling, and logging).
• Data Structures: Proficiency in working with structured, semi‑structured (JSON/XML), and unstructured data formats.
• DevOps & CI/CD: Practical experience with Git, branching strategies, and automating deployments through Azure DevOps or GitHub Actions.
• Distributed Computing: Strong grasp of Apache Spark internals, memory management, and execution plans.
• Languages: Good knowledge of written and spoken English (working language). Knowledge of French is an asset.

Specific Expertise

• Fabric Delta Lake Management: Expertise in optimizing and managing transaction logs within the Fabric Lakehouse.
• Multi‑stack Integration: Proven ability to connect Fabric with non‑Microsoft tools, utilizing generic JDBC/ODBC drivers or REST APIs.
• Performance Tuning: Deep experience in identifying bottlenecks in data flows and optimizing code for low‑latency processing.
• Data Design Patterns: Practical experience implementing standard patterns such as SCD (Slowly Changing Dimensions), Change Data Capture ingestion, and idempotency in a Delta Lake environment.

Certifications & Standards

• Microsoft Focus: DP-700 (Microsoft Certified: Fabric Data Engineer Associate) – required.
• Generic/Cloud Focus: DP-203 (Azure Data Engineer) or equivalent certifications from Databricks (Data Engineer Associate) or AWS – a plus.

Benefits

We offer 100% remote opportunities and a professional, dynamic, and collaborative team environment.

• International career opportunities across Portugal, Belgium, Luxembourg, Denmark, Netherlands, Singapore, and the United States.
• Health and Life Insurance.
• Tech Visa support for non‑EU applicants.