Senior Data Engineer: Scalable Pipelines & ML Features

Convo; Islamabad; Presencial; PKR 2.000.000 - 4.500.000; Jornada completa

Descripción de la vacante

Convo is seeking a Senior Data Engineer to build reliable data pipelines and connectors powering its CPG-focused agentic platform. You’ll develop scalable batch and streaming pipelines using Python, Spark, Databricks, Kafka, dbt, and Delta Lake, transforming source data into governed RAW and CUBE layers while ensuring data quality, performance, observability, and reliability.

You will also build data-product and feature-serving interfaces that enable ML models, agents, and downstream

Formación

• Minimum experience: 5+ years in data engineering, including 3+ years delivering production Spark or Python pipelines.
• Advanced Python and production Apache Spark/Databricks development.
• Kafka or comparable event-streaming technology.
• Strong SQL performance tuning and dimensional/data-product implementation.
• dbt and Delta Lake, including incremental patterns and table optimization.
• Batch and streaming reliability patterns: idempotency, checkpointing, replay and late-arriving data.
• Automated data quality, observability and schema-contract testing.

Responsabilidades

• Develop Python/Spark ingestion and transformation pipelines from source systems into RAW and Enriched CUBE layers.
• Implement streaming paths alongside scheduled workloads.
• Create dbt/SQL transformations, data-quality controls and contract-validation checks.
• Optimize Delta Lake layout, SQL performance, partitioning and incremental processing.
• Build feature-serving and data-product interfaces for models, agents and applications.
• Instrument pipelines for lineage, freshness, throughput, failure recovery and cost.

Conocimientos

Python

Apache Spark

Databricks

Kafka

dbt

Delta Lake

SQL tuning

Data quality

Observability

Contract-validation

Descripción del empleo

Convo is seeking a Senior Data Engineer to build reliable data pipelines and connectors powering its CPG-focused agentic platform. You’ll develop scalable batch and streaming pipelines using Python, Spark, Databricks, Kafka, dbt, and Delta Lake, transforming source data into governed RAW and CUBE layers while ensuring data quality, performance, observability, and reliability.

You will also build data-product and feature-serving interfaces that enable ML models, agents, and downstream