Senior Data Engineer

Fatima Group; Lahore; On-site; PKR 1,500,000 - 2,500,000; Full time

Job summary

Osol is seeking a hands-on Sr. Data Engineer to build the data and analytics backbone. You will standardize data engineering practices, develop reusable accelerators, and lead multi-source data pipelines across cloud, on-prem, and hybrid environments. You will mentor engineers as the practice grows and guide client engagements from pilots to production.

The role emphasizes streaming, CDC, data quality, governance, and scalable architecture for industrial AI and manufacturing use cases.

Qualifications

• Production-grade data pipelines and platforms end-to-end.
• Experience with at least one modern data platform (cloud and on-prem) and another platform.
• Strong SQL, Python, and Spark expertise; hands-on delivery experience.
• Understanding of batch and real-time pipelines, change data capture (CDC).

Responsibilities

• Define standard data engineering methodology, patterns, and testing standards.
• Plan and run proofs of concept and pilots across cloud, on-premises or hybrid platforms.
• Build reusable ingestion frameworks, data quality rule libraries, and CI/CD templates.
• Create data pipelines for structured, semi-structured, and unstructured data (ERP, APIs, logs, IoT).
• Implement CDC and event-driven integration; handle late and out-of-order data.
• Incorporate security, governance, data quality checks, and monitoring of pipelines.
• Develop portable pipelines that run on-premises and in the cloud; maintain data separation per client.
• Deliver industrial AI data sets and feature pipelines from pilot to production.
• Lead technical delivery and support presales with discovery, sizing, and demos.

Skills

Production data eng

Platform depth

SQL

Python

Spark/PySpark

Batch, streaming & CDC

Semi-structured data

Data modeling

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Kafka

CI/CD

Job description

Job Description:

We are looking for a hands-on Sr. Data Engineer to build the engineering backbone of Osol's Data & Analytics practice. You will prove and standardise how we engineer data through proofs of concept, pilots and reusable accelerators and apply that standard to client engagements, industrial data platforms and the Industrial AI program we are delivering in our group's manufacturing plants.

We are platform-agnostic: our work runs on any major cloud, on premises or both, and you will ingest any kind of data batch or real-time, from ERP systems to plant sensors securely and cost-efficiently. This is a senior individual contributor role; as the practice grows, you will mentor engineers and lead a data engineering team.

Key Responsibilities:

• Methodology & standards: Define, with the Head of Data & Analytics, Osol's standard data engineering methodology reference patterns (lakehouse layering, change data capture, streaming), coding and testing standards, definition of done and estimation guides.
• Proofs of concept: Plan and run proofs of concept and pilots across cloud, on-premises and hybrid platforms against agreed success criteria, and publish evidence-based comparisons of capability, performance, cost and operability.
• Reusable accelerators: Build metadata-driven ingestion frameworks (batch, CDC, streaming and files), data quality rule libraries, CI/CD and infrastructure-as-code templates, and demo environments for presales and partners.
• Multi-source ingestion: Build pipelines for structured, semi-structured and unstructured data databases, SAP and Oracle ERP, APIs, files, documents, logs, event streams and IoT sensors in batch, micro-batch and real time, across industries such as manufacturing, finance, telecom and retail.
• Streaming & CDC: Implement change data capture and event-driven integration, handling late, duplicated and out-of-order data and backfilling long histories.
• Quality by design: Build in security (identity and access, encryption, masking), governance (catalogue, lineage, classification, retention), data quality checks, reliability and cost control, and operate pipelines with orchestration, automated testing, CI/CD and SLA-based monitoring.
• Portable, reusable pipelines: Build collection and storage patterns that run the same on premises and in the cloud, are configured rather than recoded for each plant or client, and keep each client's data separate.
• Industrial AI use cases: Deliver reliable, analysis-ready datasets and feature pipelines for use cases such as predictive maintenance, and take them from pilot into dependable production.
• Presales & delivery: Support presales with technical discovery, sizing, estimates and demos; lead technical delivery through design and code reviews; and hand over solutions clients can run themselves.

Industrial data acquisition experience is preferred: Acquire data from plant historians, control, maintenance and laboratory systems through read-only access in segmented plant networks, buffering through outages and backfilling years of history without disturbing plant systems.

Key Skills:

We do not expect in-depth experience with every tool/technology mentioned below. But we expect deep, production-grade expertise in at least one modern data platform and one leading CSP tech stack, hands-on experience in multiple others in same category, and the ability to learn a new platform and technology quickly.

• Production data engineering: Proven, hands-on design, build and operation of production data pipelines and platforms end to end.
• Platform depth: Deep, production-grade expertise in at least one modern data platform (cloud and on-premises) and practical experience with at least one other.
• Programming & processing: Expert SQL, strong Python and Apache Spark (PySpark / Spark SQL).
• Batch, streaming & CDC: Proven delivery of batch and real-time pipelines using Kafka or a cloud equivalent, with change data capture.
• Semi-structured & unstructured data: JSON, XML, documents and logs, not only relational data.
• Data modelling for engineering: Dimensional models, Data Vault and lakehouse layer design, and turning logical models into efficient physical designs (partitioning, file formats, indexing).
• Security, governance & DataOps: Access control, encryption, masking, lineage, cataloguing and automated quality checks; Git, CI/CD, infrastructure as code and pipeline monitoring.
• Technical evaluation: Running proofs of concept and writing clear, evidence-based recommendations.

Requirements:

• Bachelor's or Master's degree in Computer Science, Software Engineering, Information Technology, Engineering or a related discipline.
• 7-10 years in data engineering, data integration or data warehousing, including 3+ years as a technical lead or lead engineer.
• Has led the technical build of at least one data platform or major integration program that reached production. Multi-industry experience is strongly preferred; industrial or OT data experience is a plus.
• Data engineering certifications on a major platform (e.g., Databricks, Microsoft Fabric, Snowflake, AWS or Google Cloud) are an advantage.