Data Engineer

Bybit; Abu Dhabi; On-site; AED 120,000 - 200,000; Full time

Job summary

A leading company in the cryptocurrency sector seeks a Big Data Engineer to manage and develop robust data pipelines and ensure data consistency. Candidates should possess a degree in Computer Science, proficiency in Hive, Spark, and Python, and have at least 3 years of experience in big data engineering. This role offers an opportunity to work on innovative projects that shape the future of finance.

Qualifications

• At least 3 years of experience in big data engineering.
• Business analysis experience is a plus.
• Knowledge of Crypto or Web3 related experience is a plus.

Responsibilities

• Design, construct, and maintain efficient data pipelines.
• Develop and maintain ETL processes for data integrity.
• Collaborate with analysts to understand data requirements.

Skills

Proficient in Hive

Proficient in Spark

Proficient in Presto

Strong ETL development

Proficient in Python

Familiar with data visualization tools

Education

Bachelor’s degree in Computer Science or related field

Tools

Tableau

Power BI

Docker

Kubernetes

Bash scripting

Job description

Job description

Responsibility:

1. Design , Construct, and Maintain efficient and reliable data Pipeline and Warehouse from diverse data .
2. Utilize Hive, Spark, and Presto for big data processing and analysis
3. Develop and maintain ETL (Extract, Transform, Load) processes to ensure data integrity and consistency
4. Collaborate with data analysts and business units to understand data requirements and provide technical support .
5. Build a resilient data architecture from complex data sources, ensuring alignment and accuracy in data calculations
6. Optimize data processing workflows to improve efficiency and performance
7. Support business-related data push and dashboard development requirements
8. Support the collection of competitor-related data and build the infrastructure for competitive analysis
9. Write and maintain technical documentation to ensure code maintainability and scalability

Requirement:

1. Bachelor’s degree or higher in Computer Science, Information Technology, Data Science, or a related field
2. At least 3 years of experience in big data engineering . Business analysis experience is a plus.
3. Proficient in Hive, Spark, and Presto for data processing and analysis
4. Strong ETL development experience and familiar with common data transformation tools
5. Proficient in data visualization tools such as Tableau, Power BI, or similar
6. Strong programming skills in Python (Required)； Java, or Scala is a plus
7. Knowledge of Docker, Kubernetes, and Bash scripting is a plus
8. Crypto or Web3 related working experience is a plus

About Bybit:

Established in March 2018, Bybit is one of the fastest growing cryptocurrency derivatives exchanges, with more than 40 million registered users. We aim to continue revolutionizing the industry by fusing the best of cryptocurrency and traditional finance. Our innovative, highly advanced, user-friendly platform has been designed from the ground-up using best-in-class infrastructure to provide our users with the industry's safest, fastest, fairest, and most transparent trading experience. Built on customer-centric values, we endeavour to provide a professional, 24/7 multi-language customer support to help in a timely manner.

Responsibility:

1. Design , Construct, and Maintain efficient and reliable data Pipeline and Warehouse from diverse data .
2. Utilize Hive, Spark, and Presto for big data processing and analysis
3. Develop and maintain ETL (Extract, Transform, Load) processes to ensure data integrity and consistency
4. Collaborate with data analysts and business units to understand data requirements and provide technical support .
5. Build a resilient data architecture from complex data sources, ensuring alignment and accuracy in data calculations
6. Optimize data processing workflows to improve efficiency and performance
7. Support business-related data push and dashboard development requirements
8. Support the collection of competitor-related data and build the infrastructure for competitive analysis
9. Write and maintain technical documentation to ensure code maintainability and scalability

Requirement:

1. Bachelor’s degree or higher in Computer Science, Information Technology, Data Science, or a related field
2. At least 3 years of experience in big data engineering . Business analysis experience is a plus.
3. Proficient in Hive, Spark, and Presto for data processing and analysis
4. Strong ETL development experience and familiar with common data transformation tools
5. Proficient in data visualization tools such as Tableau, Power BI, or similar
6. Strong programming skills in Python (Required)； Java, or Scala is a plus
7. Knowledge of Docker, Kubernetes, and Bash scripting is a plus
8. Crypto or Web3 related working experience is a plus

About Bybit:

Established in March 2018, Bybit is one of the fastest growing cryptocurrency derivatives exchanges, with more than 40 million registered users. We aim to continue revolutionizing the industry by fusing the best of cryptocurrency and traditional finance. Our innovative, highly advanced, user-friendly platform has been designed from the ground-up using best-in-class infrastructure to provide our users with the industry's safest, fastest, fairest, and most transparent trading experience. Built on customer-centric values, we endeavour to provide a professional, 24/7 multi-language customer support to help in a timely manner.