Data Engineer

Tkxel; Pakistan; Sur place; PKR 2 000 000 - 3 000 000; Temps partiel

Résumé du poste

Tkxel seeks a Data Engineer for a two-month AI platform project. You will build secure batch data pipelines, connect to diverse data sources, and ensure data quality for AI workflows and backend services. Collaboration with IT/security teams is essential to deliver on-prem deployment.

The role emphasizes strong SQL skills, YAML-driven configuration, and experience with PostgreSQL and Kafka-like systems to support data integration and downstream analytics.

Qualifications

• Strong SQL skills for data extraction, transformation, and validation.
• Experience building batch data ingestion pipelines from multiple sources.
• Secure enterprise data connectivity and on-premises integration.
• Familiarity with YAML or configuration-driven setups.
• Experience with PostgreSQL and data modeling.
• Familiar with Kafka or similar messaging systems.
• Hands-on data transformation and data quality checks.
• Ability to document pipelines for handoff.

Responsabilités

• Build and configure batch data pipelines using low-code builder.
• Connect to client data sources (RDBMS, APIs, files, on-prem storage, enterprise systems).
• Write and validate SQL for extraction, transformation, and validation.
• Prepare and validate data for AI workflows and backend/frontend apps.
• Define source-to-target mappings and data schemas.
• Create YAML/config files for pipelines and connections.
• Validate data ingestion, freshness, and delivery.
• Perform data quality checks (completeness, consistency, reconciliation).
• Define how data should be stored and exposed to downstream workflows.
• Collaborate with Backend/Frontend/DevOps and client IT/security teams.
• Support testing, deployment, documentation, and handoff in on-prem env.

Connaissances

SQL

Data pipeline development

Batch data pipelines

Kafka

YAML/configuration

PostgreSQL

Secure data integration

Troubleshooting

Description du poste

Job Description:

Role Overview

We are seeking a Data Engineer to support a short-term AI application delivery project built on AI platform, which provides capabilities including MLOps, LLM/RAG, conversational UI, low-code data pipelines, and workflow orchestration.

The Data Engineer will focus on secure data integration and data pipeline development. Responsibilities include connecting to client data sources, configuring and implementing pipelines, transforming and preparing data, validating data quality, and documenting the solution for operational handoff. The role will also help define how relational, intermediate workflow, and output data should be stored, structured, and made available to downstream application, analytics, and AI workflows.

The ideal candidate will have strong SQL skills, experience with secure enterprise data connectivity, familiarity with YAML or configuration-driven setup, and hands-on experience building and integrating data pipelines. Relevant data sources may include relational databases, APIs, file-based data, and enterprise systems.

This is an approximately two-month engagement, with the final solution deployed within the client’s on-premises environment.

Key Responsibilities

• Build and configure batch data pipelines using the platform’s low-code data pipeline builder, with support for event- or message-based integration where required.
• Securely connect to client data sources, including relational databases, APIs, file-based sources, on-premises storage, enterprise systems, and Kafka or similar messaging platforms.
• Write and validate SQL for data extraction, transformation, filtering, aggregation, lookup logic, and validation.
• Prepare, transform, clean, and validate data required by AI workflows, ML models, backend services, and frontend applications.
• Define and validate source-to-target mappings, data schemas, field mappings, data types, event/message schemas, and downstream input/output structures.
• Create and maintain YAML or configuration files for pipeline setup, data mappings, data source connections, event/message connections, and workflow inputs.
• Validate data ingestion, including source formats, schemas, load completeness, duplicate and null handling, retry/reprocessing behavior, event/message delivery, data freshness, and downstream availability.
• Perform data quality checks, including completeness, consistency, reconciliation, schema validation, and event/message validation.
• Define how relational, intermediate workflow, event/message, and output data should be stored, structured, and made available to downstream workflows.
• Collaborate with Backend, Frontend, DevOps, and client IT/security teams to ensure correct data formats, secure connectivity, and successful deployment.
• Support testing, troubleshooting, deployment, documentation, and handoff of data pipeline components within the client’s on-premises environment.

Required Skills and Experience

• Strong SQL skills, including joins, aggregations, filtering, CTEs, window functions, and query optimization.
• Experience with data transformation, cleansing, source-to-target mapping, and data validation.
• Hands-on experience building batch data ingestion pipelines from relational databases, APIs, file-based sources, and shared storage, including scheduling, incremental loads, retry handling, error handling, and delivery validation.
• Experience working with structured and semi-structured data, including relational tables, JSON, CSV, Parquet, event messages, and API outputs.
• Familiarity with Kafka or similar messaging technologies, including event/message schemas and basic producer/consumer concepts.
• Familiarity with YAML or configuration-driven data pipeline setup.
• Experience with PostgreSQL, including schema design, indexing, query optimization, and relational data modeling.
• Experience with secure enterprise or on-premises data integration, including network restrictions, credentials, and access controls.
• Strong troubleshooting, testing, documentation, and independent delivery skills.

Preferred Qualifications

• Experience with data governance considerations, including access control, retention, lineage, auditability, and secure handling of client data.
• Familiarity with Python or Java for custom transformation or integration logic where needed.

Requirements: