Data Engineer

Quantuma; Kuala Lumpur; Hybrid; MYR 120,000 - 180,000; Full time

Job summary

K3 Advisory Group is seeking a Senior Data Engineer to design and operate ELT pipelines across Azure data services, delivering trusted datasets for analytics and AI across multiple brands.

You will work with the Data Lead and a cross-functional squad to implement robust data models, APIs and governance controls, while advancing data quality, lineage and security practices in a regulated professional services environment.

Qualifications

• 4+ years in modern data engineering with cloud platforms — ideally Azure.
• Strong SQL — including PostgreSQL — and Python for data transformation.
• Experience with Azure data services such as Data Factory, Fabric, Synapse or Databricks.
• Solid experience working with APIs: consuming them reliably at scale and exposing data through them.
• Experience with data transformations, reconciliation and version-controlled ETL/ELT pipeline development with CI/CD.
• Working knowledge of star schema and medallion/lakehouse modelling.
• Data governance awareness and practical PII handling experience.
• Able to communicate clearly with UK-based stakeholders and work across time zones.
• Desirable: MCP, ML/AI workloads, and governance tooling familiarity.

Responsibilities

• Develop ELT pipelines from SaaS, ERP, CRM and finance systems into the Azure data platform (Azure Data Lake, Synapse/Databricks, PostgreSQL).
• Design and implement data models (star schema and medallion/lakehouse) with semantic layers for product, dashboards and AI.
• Implement transformations using SQL and Python within version-controlled, code-reviewed workflows.
• Manage schema evolution and CDC to keep downstream models stable as sources change.
• Optimise pipeline and query performance, including PostgreSQL tuning, balancing cost, freshness and reliability.
• Build robust API integrations to ingest data from third-party and internal systems.
• Expose governed datasets to third-party systems and internal products via APIs and data views.
• Contribute to MCP-based integrations under senior guidance.
• Ingest and enrich external data sources with quality checks and reconciliation.
• Support onboarding of acquired businesses onto the Group data platform.

Skills

SQL

Python

PostgreSQL

Azure

Data Factory

Databricks

APIs

ETL/ELT

Data Modelling

Data Governance

CI/CD

PII Handling

Job description

About K3 Advisory Group

K3 Advisory Group is a UK professional services group with multiple trading subsidiaries and more than 1,200 staff. The Group spans corporate finance, tax, restructuring and insolvency, legal, financial planning, and technology-enabled advisory services.

About K3 Advisory Group

K3 Advisory Group is a UK professional services group with multiple trading subsidiaries and more than 1,200 staff. The Group spans corporate finance, tax, restructuring and insolvency, legal, financial planning, and technology-enabled advisory services. All technology delivery must balance the pace required to exploit AI, data and automation with strict expectations around governance, regulatory obligations, security-by-design, audit trails, and client confidentiality. Group Technology builds shared platforms, data foundations and AI-assisted products that scale across these businesses while accommodating local variation. Engineers in this team work close to commercial outcomes, with direct visibility of advisors, partners and clients.

Our data & AI platform

Our estate is predominantly Azure. We build on Azure-native data services — including Microsoft Fabric, Synapse, Data Factory and, on some products, Cosmos DB — with Databricks part of our wider reference stack, and we make heavy use of PostgreSQL. We keep the platform aligned with the latest Azure capabilities and the Group's technology direction, so we value engineers who are comfortable evolving an estate, not just maintaining one. Our data teams support both internal platforms and the SaaS products the Group sells, with a growing focus on integrating third-party and external data across multiple products. Governed integration is central to how we work: well-designed APIs, semantic layers and modern integration standards such as the Model Context Protocol (MCP) connect data with applications and AI-assisted products under strict access control, logging and audit. Reconciliation, pipeline reliability and the safe, permissioned exchange of data with third-party systems are core to how the platform earns trust.

Role Purpose

Build and maintain reliable ELT pipelines, data models and data products powering analytics and AI across all K3 brands. Enable data-driven decision making through high-quality, secure data access. This role delivers within the patterns, standards and governance framework set by the senior data engineers and Data Lead. Data flowing through the platform supports advisor, partner and (in places) client-facing decisions across multiple subsidiaries, including FCA-regulated entities — so quality, tenancy and auditability are baseline expectations at every level of the team, not just senior ones.

Key Responsibilities

Pipelines & Data Modelling

• Develop ELT pipelines from SaaS, ERP, CRM and finance systems into the Azure data platform (Azure Data Lake, Synapse/Databricks, PostgreSQL), following the Group's documented ingestion patterns.
• Design and implement data models (star schema and medallion/lakehouse patterns) with semantic layers that serve product, dashboard and AI consumption.
• Implement transformations using SQL and Python within version-controlled, code-reviewed workflows.
• Manage schema evolution and change data capture (CDC) so that downstream models remain stable as source systems change.
• Optimise pipeline and query performance — including PostgreSQL tuning — balancing cost, freshness and reliability.

APIs, Integration & Data Exposure

• Build robust API integrations to ingest data from third-party and internal systems: authentication, paging, rate limits, retries and idempotency handled properly.
• Expose governed datasets to third-party systems and internal products via APIs and approved data views, respecting tenancy and permissions.
• Contribute to MCP-based integrations under senior guidance — building or maintaining governed tools through which applications and AI-assisted products query and enrich approved datasets (desirable, and trained on the job).
• Ingest and enrich third-party and external data sources — applying quality assessment and reconciliation before downstream use — across internal platforms and the Group's SaaS products.
• Support onboarding of acquired businesses onto the Group data platform under the direction of senior engineers.

Quality, Reconciliation & Governance

• Implement data quality rules, reconciliation controls and freshness monitoring on the datasets you own; reconciliation against source systems is routine, not heroic.
• Maintain lineage tracking so that every metric and output can be traced back to source.
• Apply security controls — tenant/client separation, permission-aware access, PII handling and masking — as designed, and raise gaps early.
• Support cross-brand data integration while maintaining strict data segregation between subsidiaries, including FCA-regulated entities.
• Keep documentation, source mappings and runbooks current for the pipelines you deliver.

Delivery & Collaboration

• Deliver new source connectors and datasets within agreed SLAs, using repeatable, documented patterns rather than bespoke effort each time.
• Work day-to-day within a cross-functional squad alongside senior data engineers, AI/ML engineers, analysts and product owners.
• Help prepare curated datasets that support AI/LLM use cases (e.g. retrieval datasets for RAG) to specifications set by senior engineers.
• Participate in code review, pairing, incident response and post-incident learning, and help onboard junior engineers.

Required Experience & Skills

• 4+ years in modern data engineering with cloud platforms — ideally Azure, as our estate is predominantly Azure.
• Strong SQL — including hands‑on PostgreSQL — and Python (or Scala) for data transformation.
• Experience with Azure data services such as Data Factory, Microsoft Fabric, Synapse or Databricks.
• Solid experience working with APIs: consuming them reliably at scale and exposing data through them.
• Experience with data transformations, reconciliation and version-controlled ETL/ELT pipeline development with CI/CD.
• Working knowledge of star schema and medallion/lakehouse modelling.
• Data governance awareness and practical PII handling experience.
• Able to communicate clearly with UK-based stakeholders and work effectively across time zones.
• Effective use of AI-assisted engineering tools, paired with full ownership of the result: everything shipped is understood, tested and explainable. We value AI-accelerated engineers, not unreviewed AI output.

Desirable Experience

• Awareness of MCP (Model Context Protocol) and governed tool-based integration patterns — or genuine interest in learning them quickly.
• Exposure to ML or AI workloads: feature datasets, RAG/retrieval datasets, embeddings or supporting AI/ML teams.
• Experience exposing data to third-party systems (partner integrations, client reporting feeds, webhooks).
• Exposure to professional services, financial services or legal data domains (client/matter/case, finance, risk, pipeline).
• Experience with DBT, data contracts or semantic layer tooling; familiarity with catalogue/lineage tooling such as Purview.
• Experience of platform migrations (e.g. moving workloads between cloud data platforms).

Success Measures

Targets are agreed with the Data Lead and will be refined as the Group's data quality framework matures.

• Freshness: Data freshness meets agreed targets on priority datasets.
• Quality: Issues on owned datasets are detected early, reconciled against source and resolved with clear ownership; contributes to shaping the Group's data quality framework.
• Delivery: New connectors and integrations delivered within agreed SLAs.
• Governance: Data access patterns preserve tenancy, permissions and segregation across all subsidiaries, including FCA-regulated entities.

Working Environment

• Reporting line: Data Lead, working day-to-day within a cross-functional product squad (engineering, data, AI, design, product).
• Stakeholders: UK-based business leaders, operational teams, compliance and risk functions, and client-facing advisors across multiple subsidiaries.
• Delivery model: Iterative, product-led delivery with short feedback loops, paired with the governance discipline appropriate to a regulated professional services environment.
• Tooling baseline: Azure-first cloud platform, PostgreSQL and Azure-native data services, Git-based source control, CI/CD pipelines, infrastructure-as-code, observability tooling and a documented engineering handbook.
• Ways of working: Code review, pairing, design reviews, threat modelling for sensitive features, and lightweight architecture decision records (ADRs).

Governance, Security & Compliance Expectations

• Confidentiality: Client, matter, and case data is highly sensitive. Need-to-know access is the default; broad access is the exception and must be justified.
• Security by design: Threat modelling, secure defaults, secrets management, dependency scanning and least-privilege access are built into features from day one.
• Auditability: User actions, data access and administrative changes are logged in a tamper-evident, queryable form suitable for internal audit and regulatory review.
• Responsible AI: Where AI is used, model behaviour, prompts, tools and data access are versioned, evaluated and monitored. Applications, AI-assisted products and third-party systems integrate with data only through approved, governed interfaces (APIs, semantic layers, MCP servers) — never raw access. Human oversight is preserved for material decisions.
• Regulatory awareness: For features touching FCA-regulated entities (e.g. Pareto, Luna), additional controls apply around record-keeping, client communications and data handling. Engineers are expected to flag uncertainty early.
• Data protection: UK GDPR, Malaysian PDPA (where applicable) and Group data protection standards apply across all subsidiaries; data minimisation, lawful basis and retention controls are part of normal design.

Development & Progression

• Clear engineering career path: Junior → Data Engineer → Senior → Lead, with a parallel route into architecture.
• Mentoring from senior engineers and the Data Lead, with structured code review and design feedback.
• Exposure to acquisitions, integrations and greenfield product builds across multiple professional services disciplines.
• Supported learning budget, relevant certifications (e.g. Azure, Databricks, PostgreSQL) and time allocated for proof-of-concept work.
• Direct line of sight to commercial outcomes — engineers see how their work changes how advisors and clients actually operate.

Person Specification

• Reliable: Delivers well-tested, documented pipelines that others can support.
• Quality-minded: Treats data quality, reconciliation, lineage and segregation as part of the job, not an afterthought.
• Curious: Wants to understand the business meaning of the data — and how AI systems will consume it — not just move it.
• Pragmatic: Follows established patterns, and flags when a pattern doesn't fit rather than working around it silently.
• Collaborative: Works well within a squad, communicates progress and blockers openly, and asks for help early.
• Creative problem-solver: Thinks outside the box when the standard pattern doesn't fit, then lands on a pragmatic, supportable solution.

Salary & Location

• Location: Kuala Lumpur, Malaysia (hybrid working model).