Data Engineer

Cloudxtreme; Hyderabad, Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Cloudxtreme is seeking a data engineering professional to design and optimize data pipelines, integrating diverse data sources and applying best practices in data governance. The role emphasizes Spark-based processing, Azure cloud services, and collaboration with cross-functional teams.

The candidate will work on ETL processes, data modeling, and scalable data architectures to support analytics and reporting initiatives within the cloud environment.

Qualifications

• Experience with Azure cloud, Azure Data Bricks, SQL, Spark, Scala or Python.

Skills

Python

SQL

Scala

Scripting & automation

Tools

Azure Cloud

Azure Data Bricks

Apache Spark

Apache Kafka

Apache Airflow

Talend

Hadoop

Hive

HBase

Cassandra

Jira

GitHub

DevOps tools

Job description

Strong Programming and Scripting Skills: Proficiency in programming languages such as Python, SQL, or Scala, as well as experience with scripting and automation tools, for data manipulation, ETL processes, and pipeline development.

• Experience with Azure cloud, Azure data bricks, SQL , Spark, Scala or Python
• Experience in Jira, Github, DevOps tools
• Data Integration and ETL Tools: Familiarity with tools and frameworks for data integration and ETL processes, such as Apache Kafka, Apache Spark, Apache Airflow, or Talend.
• Data Modeling and Database Design: Knowledge of data modeling concepts, dimensional modeling, and schema design to create efficient and scalable database structures.
• Big Data Technologies: Experience with big data platforms and technologies like Apache Hadoop, Apache Hive, Apache HBase, or Apache Cassandra for handling large volumes of data.
• Cloud Platforms: Familiarity with cloud platforms and related services for data storage, processing, and analytics.
• Data Governance and Security: Understanding of data governance principles, data security best practices, and compliance requirements to ensure data integrity, privacy, and regulatory compliance.
• Problem-solving and Analytical Thinking: Strong problem-solving skills, attention to detail, and analytical thinking to identify and resolve data-related issues and optimize data pipelines.
• Communication and Collaboration: Excellent communication skills to effectively collaborate with cross-functional teams, including data analysts, data scientists, and stakeholders, and to explain complex technical concepts to non-technical stakeholders.