Databricks Specialist

Qrusible Talent Network; Dubai; Ibrido; AED 180.000 - 260.000; Tempo pieno

Descrizione del lavoro

Pinnacle DXB, a Dubai-based IT solutions firm, seeks a Databricks Specialist to design and run data pipelines for a Finance Automation Program. You will ingest ERP and finance data, structure bronze/silver/gold layers, and deliver reliable datasets for Power BI, AI agents, and web apps.

Onsite in Dubai or remote, full-time; 4–7 years in data engineering with Databricks required. Strong PySpark/SQL, Delta Lake, Unity Catalog and production experience with Databricks Workflows are essential.

Competenze

• 4–7 years in data engineering with at least 2 years hands-on Databricks.
• Strong PySpark and SQL proficiency.
• Experience with Delta Lake, Unity Catalog and Databricks Workflows in production.
• Cloud experience with at least one provider (Azure preferred).
• Data modelling and ETL/ELT design across ERP/finance data (AP/AR/GL) preferred.

Mansioni

• Design, build and maintain batch and incremental pipelines on Databricks.
• Ingest data from ERP, databases, files, APIs and other finance systems.
• Organise data in bronze / silver / gold layers with Delta Lake.
• Write efficient PySpark and SQL transformations.
• Build data quality checks, monitoring, alerts and error handling into pipelines.
• Set up access control and governance with Unity Catalog.
• Orchestrate with Databricks Workflows or Azure Data Factory; tune performance and compute cost.
• Prepare curated datasets for reporting, Power BI, AI agents and the web application; use Git and CI/CD.

Conoscenze

PySpark

SQL

Delta Lake

Unity Catalog

Databricks Workflows

Git

CI/CD

Azure

Data modelling

ERP/finance data

Descrizione del lavoro

Role Overview

The Databricks Specialist builds and runs the data foundation for Pinnacle DXB's Finance Automation Program. You bring finance data from the ERP and other source systems into Databricks, clean and transform it, and prepare reliable datasets for reporting, Power BI, AI agents and the program's web applications.

Experience: 4–7 years in data engineering with at least 2 years hands‑on Databricks. Full-time, onsite in Dubai or remote. Finance-domain data experience is strongly preferred. The client wants to close this role within 2–3 weeks.

About Pinnacle DXB

Pinnacle Computer Systems LLC (Pinnacle DXB) is a Dubai-based IT solutions company founded in 1995, with 30+ years in the region and 2,000+ clients across the Middle East. It delivers ERP and business applications (SAP Business One partner), IT services and support contracts, infrastructure, security, cloud, and Power BI / business-intelligence services to sectors including hospitality, education, healthcare, retail, manufacturing, distribution, contracting, automotive, oil & gas and media. HQ: Office 204, Al Khaleej Building, Al Karama, Dubai, with a presence in Abu Dhabi. Hiring for a Finance Automation Program: dedicated full-time resources, one person per role.

What You'll Work On

• Design, build and maintain batch and incremental pipelines on Databricks
• Ingest data from the ERP, databases, files, APIs and other finance systems
• Organise data in bronze / silver / gold layers with Delta Lake
• Write efficient PySpark and SQL transformations
• Build data quality checks, monitoring, alerts and error handling into every pipeline
• Set up access control and governance with Unity Catalog
• Orchestrate with Databricks Workflows or Azure Data Factory; tune performance and compute cost
• Prepare curated datasets for reporting, Power BI, AI agents and the web application; use Git and CI/CD

Who Is Right for This Role

• 4–7 years in data engineering, including at least 2 years hands‑on Databricks
• Strong PySpark and SQL
• Delta Lake, Unity Catalog and Databricks Workflows in production
• At least one cloud (Azure preferred)
• Data modelling and ETL/ELT design
• Has integrated data from ERP or other enterprise systems; finance data (AP, AR, GL) strongly preferred
• Git and basic CI/CD

Good to Have

• Delta Live Tables (Lakeflow), Auto Loader, structured streaming
• Databricks Asset Bundles, Terraform
• SAP data extraction
• Power BI
• MLflow or Mosaic AI for ML / GenAI use cases
• Databricks Certified Data Engineer (Associate or Professional) plus an Azure or AWS data certification

Who Isn't a Fit

• Less than 2 years of hands‑on Databricks
• Only notebook / ad‑hoc analysis work, no production pipelines
• No experience with enterprise (ERP / finance) source data
• Under 4 years of data engineering

How You'll Work

• Careful with data accuracy; validates numbers with finance users
• Documents data models and pipelines
• Works closely with the finance specialist, AI automation and application engineers

Engagement

• Full-time, dedicated to the Finance Automation Program (one person per role; openings: 1)
• Location: Dubai, UAE (onsite) or Remote
• Experience: 4–7 years
• Professional English required
• ERP and cloud platform to be confirmed by the client (likely SAP / Microsoft Azure; experience with SAP, Oracle or Microsoft Dynamics and Azure or AWS is relevant)
• Compensation discussed with shortlisted candidates
• Hiring timeline: the client wants to close this role within 2–3 weeks