Platform Engineer

HCL Technologies Limited; Helsingborgs kommun; On-site; SEK 900,000 - 1,150,000; Full time

Job summary

HCL Technologies Limited is seeking a Data Architect skilled in the Databricks AI Stack to design and implement scalable lakehouse architectures on Azure and/or GCP. You will lead governance, performance tuning, and security while integrating LLMs and GenAI solutions for end-to-end AI use cases.

The role requires hands-on Databricks setup, PySpark, Delta Lake, and cloud experience across Azure and GCP ecosystems, with a focus on governance and cost control.

Qualifications

• Strong hands-on Databricks platform setup and architecture.
• Expertise in the Databricks AI Stack: MLflow, Feature Store, Model Serving, Vector Search, RAG architectures, embeddings.
• Experience with PySpark, Delta Lake, and Spark performance optimization.
• Cloud ecosystems: Azure Databricks, Azure services, GCP Databricks, BigQuery, GCS, Vertex AI.
• Hands-on integration of LLMs and Generative AI solutions.
• Knowledge of Unity Catalog, governance, data security, and access management.

Responsibilities

• Design and implement Databricks architecture on Azure and/or GCP (workspace, clusters, networking, security).
• Lead platform setup, governance, and optimization (Unity Catalog, cost control, access management).
• Build and deploy AI/ML solutions using Databricks (MLflow, Feature Store, Model Serving).
• Develop GenAI/LLM solutions (RAG pipelines, embeddings, vector search).
• Integrate LLMs (Azure OpenAI / Vertex AI / other APIs) within Databricks.
• Deliver scalable Lakehouse architecture solutions.
• Collaborate with engineering and business teams for end-to-end AI use cases.
• Ensure performance tuning, reliability, and security of workloads.

Skills

Databricks

MLflow

Feature Store

Model Serving

Vector Search

RAG Architectures

Embeddings

PySpark

Delta Lake

Spark Optimization

Azure Databricks

GCP Databricks

Vertex AI

Unity Catalog

Data Security

Access Management

LLMs

Generative AI

Cloud Architectures

Tools

BigQuery

GCS

Azure Services

Databricks Runtime

Job description

We are looking for Data architect with experise on Databricks AI Stack

Key Responsibilities

• Design and implement Databricks architecture on Azure and/or GCP (workspace, clusters, networking, security).
• Lead platform setup, governance, and optimization (Unity Catalog, cost control, access management).
• Build and deploy AI/ML solutions using Databricks (MLflow, Feature Store, Model Serving).
• Develop GenAI / LLM solutions (RAG pipelines, embeddings, vector search).
• Integrate LLMs (Azure OpenAI / Vertex AI / other APIs) within Databricks.
• Deliver scalable Lakehouse architecture solutions.
• Collaborate with engineering and business teams for end-to-end AI use cases.
• Ensure performance tuning, reliability, and security of workloads.

Skills

Strong hands-on experience in Databricks platform setup and architecture (mandatory).

Expertise in the Databricks AI Stack: MLflow AutoML Feature Store Model Serving Vector Search RAG Architectures Embeddings

Strong experience with PySpark, Delta Lake, and Spark performance optimization. Experience with cloud ecosystems:

Azure Databricks and related Azure services GCP Databricks, BigQuery, GCS, and Vertex AI Hands-on experience integrating LLMs and Generative AI solutions.

Knowledge of Unity Catalog, governance, data security, and access management.