Big Data Engineer

Purview Services; Surat; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Purview Services is seeking a skilled Big Data Engineer to design, develop, and implement data solutions using Hadoop, Spark, and NoSQL technologies. You will collaborate with cross-functional teams to translate business requirements into scalable pipelines and architectures.

The role focuses on maintaining data quality, validating data, and optimizing performance to handle large datasets in a fast-paced environment. Strong programming skills and Agile experience are essential.

Qualifications

• Strong understanding of big data concepts and technologies.
• Experience with large datasets and scalable big data solutions.
• Proficiency in Java, Python, or Scala.
• Familiarity with Git and agile methodologies.

Responsibilities

• Design and implement big data solutions using Hadoop, Spark, and NoSQL.
• Collaborate with cross-functional teams to gather requirements.
• Develop and maintain large-scale data pipelines and architectures.
• Ensure data quality and integrity through testing and validation.
• Optimize systems for performance and scalability.

Skills

Big data concepts

Java

Python

Scala

Team collaboration

Stakeholder communication

Agile methodologies

Tools

Hadoop

Spark

NoSQL databases

Git

Job description

Roles and Responsibility

• Design, develop, and implement big data solutions using various technologies.
• Collaborate with cross-functional teams to identify business requirements and develop solutions.
• Develop and maintain large-scale data pipelines and architectures.
• Ensure data quality and integrity by implementing data validation and testing procedures.
• Optimize big data systems for performance and scalability.
• Stay updated with industry trends and emerging technologies in big data engineering.

Job Requirements

• Strong understanding of big data concepts and technologies such as Hadoop, Spark, and NoSQL databases.
• Experience working with large datasets and developing scalable big data solutions.
• Strong programming skills in languages such as Java, Python, or Scala.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment and communicate effectively with stakeholders.
• Familiarity with agile development methodologies and version control systems like Git.