Data Engineer (Lahore) – Sora Systems – Full‑Time Role 2024

Sora Systems LLC; Lahore; On-site; PKR 250,000 - 500,000; Full time

Job summary

Sora Systems LLC in Lahore is seeking a Data Engineer with 4–7 years of experience to design and maintain AWS-based data lakehouse pipelines using Python, Spark and Apache Iceberg. You will build scalable batch and streaming data flows and collaborate across engineering and data science teams.

Candidates should be proficient in Python and SQL, have hands-on AWS experience (S3, Glue, Athena) and familiarity with CI/CD tools and version control. A degree in CS or a related field is preferred.

Qualifications

• 4–7 years of professional experience in data engineering.
• Proficient in Python and SQL.
• Hands-on experience with AWS services (S3, Glue, Athena) and Apache Iceberg.
• Familiarity with PySpark or Apache Spark and data lakehouse concepts.
• Understanding of cloud security, IAM and monitoring tools.
• Experience with Git, CI/CD tools such as Jenkins and version control.
• Bachelor’s or Master’s degree in Computer Science or related field is preferred.

Responsibilities

• Design, build and maintain both batch and streaming data pipelines.
• Construct and optimise lakehouse architectures on AWS using Apache Iceberg.
• Develop ETL/ELT processes for structured and unstructured datasets.
• Work with Amazon S3, AWS Glue, Glue Data Catalog and Amazon Athena.
• Implement data partitioning, schema evolution, query optimisation and cost‑effective cloud resources.
• Monitor pipeline health, resolve failures and enhance performance.
• Ensure data quality, security, governance and appropriate access controls.
• Set up CI/CD pipelines and keep technical documentation up to date.
• Collaborate with engineering, AI and data‑science teams.

Skills

Python

SQL

PySpark

Apache Spark

IAM

Monitoring tools

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

AWS S3

AWS Glue

Amazon Athena

Apache Iceberg

Glue Data Catalog

Jenkins

Job description

Sora Systems in Lahore seeks a Data Engineer with 4‑7 years experience to create and maintain AWS‑based data lakehouse pipelines, using Python, Spark and Apache Iceberg.

Sora Systems, a technology firm based in Lahore, is hiring a full‑time Data Engineer to design and sustain scalable data pipelines and AWS‑powered lakehouse solutions.

Responsibilities

• Design, build and maintain both batch and streaming data pipelines.
• Construct and optimise lakehouse architectures on AWS using Apache Iceberg.
• Develop ETL/ELT processes for structured and unstructured datasets.
• Work with Amazon S3, AWS Glue, Glue Data Catalog and Amazon Athena.
• Implement data partitioning, schema evolution, query optimisation and cost‑effective cloud resources.
• Monitor pipeline health, resolve failures and enhance performance.
• Ensure data quality, security, governance and appropriate access controls.
• Set up CI/CD pipelines and keep technical documentation up to date.
• Collaborate with engineering, AI and data‑science teams.

Who Should Apply

• 4–7 years of professional experience in data engineering.
• Proficient in Python and SQL.
• Hands‑on experience with AWS services (S3, Glue, Athena) and Apache Iceberg.
• Familiarity with PySpark or Apache Spark and data lakehouse concepts.
• Understanding of cloud security, IAM and monitoring tools.
• Experience with Git, CI/CD tools such as Jenkins and version control.
• Bachelor’s or Master’s degree in Computer Science or related field is preferred.