Big Data Engineer

Virtusa; Dubai; On-site; AED 120,000 - 200,000; Full time

Job summary

A leading IT services company in Dubai is seeking a Senior Lead Software Engineer specializing in Big Data engineering. This role involves optimizing data pipelines using tools like Spark and Hive, designing APIs, and deploying solutions on cloud platforms. With over 7 years of experience required, candidates should also have a strong grasp of the Banking domain. This position offers a dynamic work environment with a focus on personal and professional development.

Qualifications

• 7 plus years of experience in Big Data engineering.
• Hands on with Cloudera, Spark, Hive, Kafka, Python, Hue and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

Responsibilities

• Strong functional knowledge of Banking domain.
• Develop and optimize data pipelines using Spark, Hive and Python on Cloudera.
• Develop real time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Work with Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms AWS, Azure or GCP.

Skills

Big Data engineering

Cloudera

Spark

Hive

Kafka

Python

Hue

Job description

Role

Senior Lead Software Engineer

Skill

Big Data Engineer

Experience

7 Years

Responsibilities

• Strong functional knowledge/Experience of the domain Banking.
• Develop and optimize data pipelines using Spark, Hive and Python on Cloudera.
• Develop real time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Work with Hue, Oozie and other Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms AWS, Azure or GCP.

Required Skills and Experience

• 7 plus years of experience in Big Data engineering.
• Hands on with Cloudera, Spark, Hive, Kafka, Python, Hue and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

About Virtusa

Add the Middle East to your global professional experience and have the opportunity to work on some of the leading Digital Transformation programs. Virtusa is one of the fastest growing IT Services companies in the Middle East with a growing client base in the UAE, KSA, Qatar & Oman and we work with the leading Banking and Financial Services, Travel, Telecom and Enterprise firms in the region. We have partnered with our clients to win awards from Gartner, IDC, WfMC and other analysts for the work that we have delivered to our clients in the region.

Benefits and Values

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us you join a team of 30,000 people globally that cares about your growth and seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.