Senior/Principal Data Engineer - Financial Services/Banking

Adecco Portugal; Jakarta Pusat; Sur place; IDR 420 000 000 - 600 000 000; Plein temps

Résumé du poste

Adecco Portugal is seeking an experienced Senior/Principal Data Engineer to design, build, and optimize scalable data platforms supporting analytics, AI, and LLM applications in Jakarta.

You will own data architecture, pipelines, and governance, integrate LLM data workflows, and collaborate with AI/ML, security, and product teams to deliver reliable, high-performance data infrastructure with Cloudflare integrations.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
• 5–8 years of experience in data engineering or related field.
• Strong proficiency in Python and SQL; experience with Spark, Kafka, or equivalent.
• Experience designing/operating modern data platforms (data lakes/warehouses/lakehouses).
• Hands-on with LLMs production data prep, embeddings, RAG pipelines, vector search.
• Cloud experience (AWS/GCP/Azure).
• Experience with Cloudflare products (API security, WAF, Workers, CDN, Zero Trust).
• CI/CD, IaC, containerization, and automated testing.
• Leadership and collaboration across engineering/product teams.

Responsabilités

• Design, develop, and maintain scalable data pipelines and data processing systems.
• Build data lakehouse architectures and ensure data quality and governance.
• Develop data pipelines for LLMs, embeddings, RAG, and vector search.
• Collaborate with AI/ML, security, product, and DevOps teams to deploy reliable platforms.

Connaissances

Python

SQL

Apache Spark

Kafka

CI/CD

Infrastructure as Code

Containerization

Cloud platforms

Security & governance

Observability

Formation

Bachelor's or Master's in CS/IT/Engineering

Outils

Cloudflare

Pinecone

Weaviate

Milvus

pgvector

Airflow

Dagster

Kubernetes

Docker

Terraform

Description du poste

We are looking for an experienced Senior / Principal Data Engineer to design, build, and optimize scalable data platforms and infrastructure supporting advanced analytics, AI, and Large Language Model (LLM) applications.

The ideal candidate combines strong data engineering expertise with hands-on experience in cloud infrastructure, AI/LLM data pipelines, and modern security and performance technologies, including Cloudflare. This role will play a key part in developing reliable, secure, and high-performance data systems that support business intelligence, machine learning, and AI-driven products.

Key Responsibilities

1. Data Architecture & Engineering

• Design, develop, and maintain scalable data pipelines, ETL/ELT workflows, and data processing systems.
• Build and optimize data architectures, including data lakes, data warehouses, and lakehouse solutions.
• Ensure data quality, consistency, reliability, governance, and availability across multiple systems.
• Optimize data processing performance, storage efficiency, and infrastructure costs.

2. AI & LLM Integration

• Develop data pipelines to support LLM applications, including data ingestion, preprocessing, transformation, and indexing.
• Build and maintain data infrastructure for Retrieval-Augmented Generation (RAG), vector databases, embeddings, and semantic search.
• Prepare and optimize structured and unstructured data for AI model training, fine-tuning, and inference.
• Collaborate with AI/ML engineers and product teams to operationalize LLM-powered applications.
• Implement data evaluation, monitoring, and quality controls to improve the reliability of AI systems.
• Design and maintain secure, scalable, and highly available cloud-based data infrastructure.
• Integrate Cloudflare services into application and data platform architectures where appropriate.
• Leverage relevant Cloudflare capabilities, such as CDN, WAF, API Shield, Zero Trust, and Workers, to improve security, performance, and API connectivity.
• Implement monitoring, access controls, network security, and performance optimization across data services.
• Collaborate with platform and DevOps teams to establish infrastructure automation and deployment best practices.

4. Technical Leadership & Collaboration

• Lead technical design discussions and establish data engineering standards and best practices.
• Mentor engineers, review code, and guide the implementation of complex data solutions.
• Work closely with software engineering, AI/ML, product, security, and business stakeholders.
• Drive technical decision-making, architecture reviews, and continuous platform improvements.

Requirements

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.
• Minimum 5–8 years of experience in data engineering, software engineering, or data platform development, with demonstrated ownership of complex technical projects.
• Strong proficiency in Python and SQL, with experience in data processing frameworks such as Apache Spark, Kafka, or equivalent technologies.
• Experience designing and operating modern data platforms, including data warehouses, data lakes, or lakehouse architectures.
• Hands-on experience integrating LLMs into production systems, particularly data preparation, embeddings, RAG pipelines, vector search, or AI data infrastructure.
• Practical experience with cloud platforms such as AWS, Google Cloud Platform, or Microsoft Azure.
• Experience with Cloudflare products or technologies, particularly API security, WAF, Workers, CDN, or Zero Trust.
• Strong understanding of distributed systems, APIs, data security, observability, and production reliability.
• Experience with CI/CD, Infrastructure as Code, containerization, and automated testing is an advantage.
• Strong problem-solving, system design, communication, and stakeholder management skills.

Additional Qualifications for Principal-Level Candidates

• Proven track record of defining data platform architecture and making high-impact technical decisions.
• Experience leading cross-functional engineering initiatives and mentoring senior engineers.
• Ability to translate business and AI product requirements into scalable technical solutions.
• Experience establishing engineering standards, reliability metrics, and long-term technology roadmaps.

Preferred Skills

• LLM application architecture and RAG implementation.
• Vector databases such as Pinecone, Weaviate, Milvus, or pgvector.
• Orchestration tools such as Airflow, Dagster, or equivalent.
• Cloudflare Workers, API Shield, WAF, and Zero Trust.
• Kubernetes, Docker, Terraform, and cloud-native infrastructure.
• Data governance, privacy, lineage, and observability.