Spark Engineer

Tudiptech Tudip Technologies Pvt Ltd; Pune District; On-site; INR 400,000 - 600,000; Full time

Job summary

A technology firm in Pune District is seeking an entry-level Spark Engineer to join their Data Engineering team. The ideal candidate will have at least 1 year of experience working with Apache Spark. Responsibilities include monitoring Spark jobs, collaborating on design and optimization with senior engineers, and resolving performance issues. A BE/BTech (CS/IT) or MCA from a reputed institute is preferred. This role offers an excellent opportunity to enhance your skills in a dynamic and fast-paced environment.

Qualifications

• At least 1 year of experience working with Apache Spark.
• Hands-on experience with Spark SQL, DataFrames, and RDDs.
• Knowledge of big data processing and distributed computing.

Responsibilities

• Monitor and support Spark-based data processing jobs.
• Collaborate with engineers to design and optimize Spark jobs.
• Analyze logs to identify and fix performance issues.

Skills

Spark SQL

DataFrames

RDDs

Java

Scala

Python

Problem-solving skills

Git

Education

BE/BTech (CS/IT), MCA

Tools

Apache Spark

YARN

Mesos

Kubernetes

Job description

About Company

Tudip Technologies Pvt. Ltd is a CMMI Level 5 extreme technology company. Careers at Tudip Technologies are not just jobs, but a promise of a bright and dynamic future. Tudip provides ample opportunities to grow within the company technically as well as a technocrat by promoting entrepreneurship. Tudip Technologies’ careers will enable you to help clients enhance and improve while you build your career. Tudip is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We are a place which defines Integrity, Innovation, and Serenity. Tudip provides you a better platform that translates an individual into an experienced and immensely skilled professional through an ethical and vibrant business environment. We are here for effective client servicing, taking care of our employees’ needs, and creating a success story to remember.

Position Summary

We are seeking an entry-level Spark Engineer with at least 1 year of experience working with Apache Spark to join our Data engineering team. In this role, you will assist in developing, optimizing, and supporting Spark-based applications and data pipelines. You will work alongside senior engineers and Data scientists to ensure the effective use of Spark for large-scale data processing and analytics. This is a great opportunity to enhance your skills while supporting Spark jobs in a dynamic and fast-paced environment.

Please read the job criteria below and drop us an email at [e-mail] OR create an account at our Recruitment Portal to get started.

Roles & Responsibilities

• Assist in monitoring, troubleshooting, and supporting Spark-based data processing jobs, ensuring optimal performance and reliability.
• Collaborate with senior engineers to design, build, and optimize Spark jobs and data pipelines for large-scale data processing and analytics tasks.
• Identify and resolve performance bottlenecks, errors, and failures in Spark applications by analyzing logs and job execution data.
• Monitor the health and performance of Spark clusters, recommend improvements, and implement fixes as needed.
• Work with data engineers and data scientists to understand data processing requirements and provide technical support for data-driven applications.
• Assist in documenting Spark job configurations, processes, and troubleshooting steps for knowledge sharing and team collaboration.
• Keep up with new developments in Apache Spark and related big data technologies, and apply best practices for performance, scalability, and data integrity.

Job Requirements/Qualifications

• At least 1 year of experience working with Apache Spark in a production or development environment.
• Educational Qualification: BE/BTech (CS/IT), MCA from a reputed institute.
• Hands-on experience with Spark SQL, DataFrames, and RDDs.
• Knowledge of big data processing concepts and distributed computing.
• Familiarity with Spark cluster management tools like YARN, Mesos, or Kubernetes.
• Experience with Java, Scala, or Python for Spark job development.
• Understanding of data formats such as Parquet, Avro, ORC, and JSON.
• Basic knowledge of ETL processes, data ingestion, and data transformation pipelines.
• Strong problem-solving skills and ability to troubleshoot issues in Spark jobs.
• Familiarity with Git or other version control tools for managing code.