We are looking for skilled Data Engineers to design, develop, implement, and maintain scalable data engineering solutions. The ideal candidate should have experience in data integration, ETL/ELT development, data pipelines, data warehousing, big data technologies, and cloud-based data platforms.
Candidates with experience in one or more relevant data engineering technologies are encouraged to apply.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data processing solutions.
- Build and optimize ETL/ELT workflows for data ingestion, transformation, integration, and loading.
- Extract, process, clean, validate, and transform structured, semi-structured, and unstructured data.
- Develop and optimize SQL queries, stored procedures, data models, and database structures.
- Work with data warehouses, data lakes, lakehouses, and enterprise data platforms.
- Implement batch processing and real-time data streaming solutions where required.
- Ensure data quality, integrity, consistency, reliability, and performance.
- Work with cloud-based data engineering services and platforms.
- Monitor, troubleshoot, and resolve data pipeline and production issues.
- Implement data security, governance, access controls, and data management best practices.
- Collaborate with data architects, analysts, data scientists, application developers, and business stakeholders.
- Participate in code reviews, technical documentation, testing, deployment, and continuous improvement.
Technical Skills
Programming & Query Languages: Python, SQL, PySpark, Scala, Java, R, Shell Scripting.
Big Data Technologies: Apache Spark, Hadoop, Hive, Kafka, Flink, HDFS.
Data Warehousing & Lakehouse: Snowflake, Databricks, Delta Lake, Microsoft Fabric, Apache Iceberg, Amazon Redshift, BigQuery, Teradata.
Data Modelling: Dimensional Modelling, Star Schema, Snowflake Schema, Data Modelling, Data Marts, Data Warehousing.
Workflow Orchestration: Apache Airflow, Azure Data Factory, AWS Step Functions, Dagster, Prefect.
Data Governance & Quality: Data Validation, Data Profiling, Data Lineage, Metadata Management, Master Data Management, Data Cataloguing, Data Security.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
- Relevant experience in Data Engineering, ETL Development, Data Integration, Data Warehousing, or Big Data.
- Strong understanding of SQL, data structures, data modelling, and data processing concepts.
- Hands-on experience with one or more programming languages and data engineering tools.
- Experience developing, maintaining, or optimizing data pipelines and data integration workflows.
- Familiarity with relational databases, data warehouses, or cloud data platforms.
- Good analytical, troubleshooting, and problem-solving skills.
- Ability to work independently and collaboratively in a team environment.
- Good communication skills and the ability to work with technical and business stakeholders.
Preferred Qualifications
- Experience with AWS, Azure, or GCP data services.
- Exposure to real-time data processing, streaming technologies, or distributed computing.
- Knowledge of data governance, security, and compliance practices.
- Experience with Agile/Scrum methodologies, CI/CD, and automated deployments.
- Relevant cloud, data engineering, or platform certifications are an advantage.