Data Engineer

Magellanic Cloud Limited; Bengaluru; Hybrid; INR 2,500,000 - 4,500,000; Full time

Job summary

Magellanic Cloud Limited in Bengaluru is seeking an experienced Data Software Engineer to join our hybrid, full-time team. The role focuses on building large-scale data solutions using Spark, Python, and cloud services, with an emphasis on performance and reliability.

You will work with a cross-functional team, mentor juniors, and contribute to design and implementation of data pipelines, streaming jobs, and data platforms for enterprise clients.

Qualifications

• 5–12 years of Big Data and data-related technology experience.
• Expert understanding of distributed computing principles.
• Expert knowledge of Apache Spark.
• Hands-on Python programming experience.
• Experience with Hadoop v2, MapReduce, HDFS, Sqoop.
• Experience building stream-processing systems (Storm or Spark-Streaming).
• Good SQL, joins, stored procedures, relational schemas understanding.
• Experience with NoSQL databases (HBase, Cassandra, MongoDB).
• Knowledge of ETL techniques and frameworks.
• Performance tuning of Spark jobs.

Responsibilities

• Lead design and implementation of big data solutions for complex data needs.
• Develop and optimize Spark-based data pipelines and workloads.
• Collaborate with cross-functional teams to deliver scalable data platforms.
• Mentor junior engineers and promote best practices in data engineering.
• Ensure reliability, scalability and performance of data systems.

Skills

Python

Pyspark

AWS

Databricks

Kafka

Snowflake

Job description

Hi Everyone

We do have openings for Data Software Engineering

Exp: 9-15 Years

Location: Bengaluru

Notice Period : Immediate or Serving NP or one month official

Note: Please dont share any 90 days 60 days np without serving

Work Mode: Hybrid-Fulltime Opportunity with MNC client

Interview Mode : Face to Face only

Responsibilities:

Mandatory Skills: Python, Pyspark, Aws,Databricks, Kafka, Snowflake

1. 5-12 Years of in Big Data & Data related technology experience
2. Expert level understanding of distributed computing principles
3. Expert level knowledge and experience in Apache Spark
4. Hands on programming with Python
5. Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop
6. Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming
7. Good understanding of Big Data querying tools, such as Hive, and Impala
8. Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files
9. Good understanding of SQL queries, joins, stored procedures, relational schemas
10. Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
11. Knowledge of ETL techniques and frameworks
12. Performance tuning of Spark Jobs
13. Experience with native Cloud data services AWS or AZURE
14. Ability to lead a team efficiently
15. Experience with designing and implementing Big data solutions
16. Practitioner of AGILE methodology