Big Data Engineer [L2]

Alef Education; Abu Dhabi; On-site; AED 260,000 - 420,000; Full time

Benefits offered by this job

Annual flight allowance

Performance bonus

Education allowance

Life insurance

Enhanced leave policy

Hybrid work model & 40 remote days

Job summary

Alef Education is seeking a Big Data Engineer in Abu Dhabi to lead data ingestion, storage, and processing initiatives using modern big data stacks.

You will design scalable pipelines, optimize performance, and contribute to production code while mentoring teammates and promoting best practices across the data platform.

Qualifications

• Degree in Computer Science, Software Engineering or related preferred.
• Proven experience with big data technologies and scalable data pipelines.
• Strong coding standards with focus on testability and maintainability.

Responsibilities

• Define, design, and develop data ingestion, storage, and management services.
• Build robust data pipelines and batch jobs for production environments.
• Architect scalable, low-latency data systems and reduce technical debt.
• Mentor developers and improve code quality within the team.
• Collaborate with stakeholders and support production deployments.

Skills

Big data

Java/Scala

Python

CI/CD

TDD/BDD

Azure

SQL/NoSQL

Education

Bachelor's degree in Computer Science

Tools

Spark

Hadoop

Azure Databricks

MongoDB

Cassandra

Kafka

Hive

Oozie

Job description

# Big Data Engineer [L2]Apply Now* Abu Dhabi, Abu Dhabi, United Arab Emirates (Alef\\_Abu\\_Dhabi)* Posted 61 Days Ago* Office - Flexible* Fulltime**Who we are**Alef Education began with a bold idea: that every learnerdeserves a personalised and meaningful education experience. What started in2016 as a small pilot programme in Abu Dhabi has evolved into one of theworld’s most dynamic EdTech companies—reshaping how millions of students engagewith learning across the globe.Today, Alef is proudly headquartered in the UAE, workinghand-in-hand with ministries of education, schools, and teachers to bringsmart, data-powered platforms into classrooms in over 14,000 schools.Supporting over 1.1 million students and 50,000 teachersacross the UAE, Indonesia & Morocco our AI-driven platforms generate 16+million data points every day, helping drive smarter learning decisions.Whether it’s improving national exam results, boosting classroom engagement, orsupporting educators with world-class tools, Alef is committed to impact atscale.In 2024, Alef made history as the first EdTech company tolist on the Abu Dhabi Securities Exchange (ADX), cementing our role as aregional innovator with global reach.**About The Role**As a Big Data Engineer you will have a strong understandingof big data technologies with an exceptional ability to code. You willprovide technical leadership, working closely with the wider team to ensurehigh quality code is delivered in line with the project goals and deliverycycles. You will work closely with other teams to deliver rapid prototypes aswell as production code for which you will ensure high accessibility standardsare upheld. We expect familiarity with modern frameworks and languages, as wellas working practices such as Clean Code, TDD, BDD, continuous integration,continuous delivery, and DevOps.**Key Responsibilities***Defining and developing services and solutions** Define, design, and develop services and solutions around large data ingestion, storage, and management such as withRDBMS, No SQL DBs, Log Files, Events.* Define, design, and run robust data pipelines/batch jobs in a production environment.* Architecting highly scalable, highly concurrent, and low latency systems*Maintain, support, and enhance current systems.** Contribute to paying down technical debt and use development approaches that minimize the growth of new technical debt.* Contribute feedback to improve the quality, readability, and testability of the code base within your team.* Mentor and train other developers in a non-line management capacity.* Build tools (One of SBT, Gradle, Maven).* Ensure all software built is robust and scalable.*Collaborating with Internal and external stakeholders** Participating in sprint planning to work with developers and project teams to ensure projects are deployable and monitorable from the outside.* Work with third-party and other internal providers to support a variety of integrations.* As part of the team, you may be expected to participate in some of the 2nd line in-house support and Out-of-Hours support rotas.* Proactively advise on best practices.**To Be The Right Fit, You'll Need*** Degree in Computer Science, Software Engineering or related preferred* Minimum of 5 years experience in a Big Data* Follow Clean Code/Solid principles* Adhere and use TDD/BDD.* Outstanding ability to develop efficient, readable, highly optimized/maintainable and clear code.* Highly Proficient in either Functional Java or Scala, Python* Knowledge of Azure Big Data/Analytics services – ADLS (Azure Data Lake Storage), HDInsight, Azure Data Factory, Azure Synapse Analytics, Azure Fabric, Azure Event Hubs, Azure Stream Analytics, Azure Databricks* Experience of Storing Data in systems such as Hadoop HDFS, ADLS, Event Hubs* Experience of designing, setting up and running big data tech stacks such as Hadoop, Azure Databricks, Spark and distributed datastores such as Cassandra, DocumentDBs, MongoDB, Event Hubs* In-depth knowledge of Hadoop technology ecosystem – HDFS, Spark, Hive, HBase, Event Hubs, Flume, Sqoop, Oozie, SPARK, Avro, Parquet* Experience debugging a complex multi-server service.* In depth knowledge and experience in IaaS/PaaS solutions (eg AWS Infrastructure hosting and managed services)* Familiarity with network protocols - TCP/IP, HTTP, SSL, etc.* Knowledge of relational and non-relational database systems* Understanding continuous integration and delivery.* Mocking (any of the following Mockito, ScalaTest Spock, Jasmine, Mocha).* IDE Intellij or Eclipse.* Build tools (One of SBT, Gradle, Maven).* Ensure all software built is robust and scalable.* An ability to communicate technical concepts to a non-technical audience.* Working knowledge of unix-like operating systems such as Linux and/or Mac OS X.* Knowledge of the git version control system.* Ability to quickly research and learn new programming tools and techniques.**What we offer**We believe our people are our greatest advantage. We value curiosity, purpose, and the drive to make a real difference. When you join Alef, you join a mission — and a team that’s just as passionate about transforming education as you are. Here’s a glimpse of what we offer:* Annual flight allowance* Annual performance bonus* Education allowance* Life insurance* Enhanced leave policy* A hybrid work model and up to 40 remote working days per year to work from anywhere (with flexibility built in)At Alef, we don’t just imagine a better future for education.We build it — together.