Data Engineer

Impetus; Bengaluru; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

A leading digital engineering company in Bengaluru is seeking a skilled Big Data Engineer to develop and optimize data pipelines using technologies such as Spark and PySpark. Candidates should have at least 4 years of relevant experience, strong optimization skills, and proficiency in Python and Java. This full-time role offers opportunities to work closely with data scientists and contribute to innovative data solutions.

Qualifications

• Strong experience with Big Data technologies - Spark, PySpark, Hive.
• Expertise in Spark job optimization and performance tuning.
• Proficient in Python and Java for building scalable data applications.
• Solid understanding of distributed computing and data processing frameworks.
• At least 4 years of relevant experience is required.

Responsibilities

• Develop, maintain, and optimize data pipelines using Big Data technologies.
• Implement performance tuning techniques for Spark jobs.
• Work closely with data scientists and analysts to deliver scalable data solutions.
• Perform data validation, transformation, and aggregation.
• Participate in design reviews and maintain high-quality coding standards.
• Monitor and troubleshoot data workflows in production environments.

Skills

Spark

PySpark

Hive

Python

Java

Education

Bachelor's or Master's degree in Computer Science, IT or related field

Tools

Google Cloud Platform (GCP)

Git

Job description

Direct message the job poster from Impetus

Talent Acquisition Professional || IT & Non-IT Recruitments

About the Organization-

Impetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.

Founded in 1991, we are cloud and data engineering leaders providing solutions to fortune 100 enterprises, headquartered in Los Gatos, California, with development centers in NOIDA, Indore, Gurugram, Bengaluru, Pune, and Hyderabad with over 3000 global team members. We also have offices in Canada and Australia and collaborate with a number of established companies, including American Express, Bank of America, Capital One, Toyota, United Airlines, and Verizon.

Job Description

We are looking for a skilled and experienced Big Data Engineer to join our team. The ideal candidate should have a strong background in Big Data technologies, particularly PySpark, Hive, and Spark optimization. Experience with GCP is a strong plus.

• Develop, maintain, and optimize data pipelines using Big Data technologies (PySpark, Hive, Spark).
• Implement performance tuning techniques for Spark jobs to ensure optimal performance.
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver scalable data solutions.
• Perform data validation, transformation, and aggregation to prepare datasets for analytics.
• Participate in design reviews and code reviews to maintain high-quality coding standards.
• Monitor and troubleshoot data workflows in production environments.
• Maintain documentation related to data engineering processes, architecture, and data flow.

Required Skills (Mandatory)

• Strong experience with Big Data technologies – Spark, PySpark, Hive.
• Expertise in Spark job optimization and performance tuning.
• Proficient in Python and Java for building scalable data applications.
• Solid understanding of distributed computing and data processing frameworks.
• At least 4 years of relevant experience is required.

Good to Have

• Experience with Google Cloud Platform (GCP) and its data services (e.g., BigQuery, Dataflow, Dataproc).
• Familiarity with CI/CD pipelines and version control tools (e.g., Git).
• Experience working in Agile/Scrum environments.

Education

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.

For Quick Response- Interested Candidates can directly share their resume along with the details like Notice Period, Current CTC and Expected CTC at [e-mail]

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Consulting and Engineering

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Impetus by 2x

Sign in to set job alerts for “Data Engineer” roles.

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.