Big Data Engineer

Virtusa; Dubai; On-site; AED 120,000 - 200,000; Full time

Job summary

A leading IT services company in Dubai is looking for a Senior Lead Software Engineer specializing in Big Data. Candidates should have over 7 years of experience working with Cloudera, Spark, Hive, and Kafka, along with proficiency in API development. The role involves optimizing data pipelines and deploying solutions on cloud platforms. Join a dynamic team and engage in exciting digital transformation projects in the Middle East.

Qualifications

• 7 plus years of experience in Big Data engineering.
• Hands on with Cloudera, Spark, Hive, Kafka, Python, Hue, and Ranger.
• Strong understanding of distributed systems and cloud data services.

Responsibilities

• Develop and optimize data pipelines using Spark, Hive, and Python on Cloudera.
• Develop real-time data workflows using Kafka.
• Design and develop APIs for data access and integration.

Skills

Big Data engineering

Cloudera

Spark

Hive

Kafka

Python

Hue

Ranger

Job description

Role: Senior Lead Software Engineer

Skill: Big Data Engineer

Experience : 7 Years

Strong functional knowledge/Experience of the domain Banking.

Develop and optimize data pipelines using Spark Hive and Python on Cloudera.

Develop real time data workflows using Kafka.

Design and develop APIs for data access and integration.

Work with Hue Oozie and other Cloudera tools for job orchestration and data access.

Deploy solutions on cloud platforms AWS Azure or GCP

Required Skills and Experience:

7 plus years of experience in Big Data engineering

Hands on with Cloudera Spark Hive Kafka Python Hue and Ranger

Strong understanding of distributed systems and cloud data services

Proficient in API development and data security controls

About Virtusa

Add the Middle East to your global professional experience and have the opportunity to work on some of the leading Digital Transformation programs.

Virtusa is one of the fastest growing IT Services companies in the Middle East with a growing client base in the UAE KSA Qatar & Oman and we work with the leading Banking and Financial Services Travel Telecom and Enterprises firms in the region. We have partnered with our clients to win awards from Gartner IDC WfMC and other analyst for the work that we have delivered to our clients in the region.

Teamwork quality of life professional and personal development: values that Virtusa is proud to embody. When you join us you join a team of 30000 people globally that cares about your growth one that seeks to provide you with exciting projects opportunities and work with state of the art technologies throughout your career with us.