Job Opportunity with Collabera PAN India Location_Big Data Engineer

Job Title: Big Data Engineer

Location: PAN India

Experience: 5-15 Years
Mode of Work: Onsite/Hybrid

Roles and Responsibilities:

• Develop and deploy data processing applications using Big Data frameworks such as Hadoop, Spark, Kafka, or similar technologies.
• Write efficient and optimized code in programming languages like Java, Scala, Python, or SQL to manipulate and analyze data.
• Implement data security and privacy measures to protect sensitive information and comply with regulatory requirements.
• Collaborate with cross-functional teams to integrate data solutions with existing systems and applications.
• Conduct testing and validation of data pipelines and analytical solutions to ensure accuracy, reliability, and performance.
• Document technical specifications, deployment procedures, and operational guidelines for data solutions.
• Stay updated on industry trends, emerging technologies, and Big Data development and analytics best practices.

Additional Responsibilities:

• Design, develop, and maintain high-performance, distributed data pipelines using Apache Spark, Scala, and Kafka.
• Implement data ingestion, transformation, and processing logic using Spark and Scala.
• Build and maintain real-time data streaming applications using Kafka.
• Optimize and tune Spark applications for performance and scalability.
• Ensure data quality and integrity through rigorous testing and monitoring.
• Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver effective solutions.
• Troubleshoot and resolve issues related to data pipelines and infrastructure.
• Stay up-to-date with the latest technologies and trends in big data processing.
• Contribute to the development of best practices and coding standards.
• Participate in code reviews and knowledge sharing sessions.
• Deploy and manage applications in cloud environments (e.g., AWS, Azure, GCP).
• Proven experience in developing data pipelines using Spark, Scala, and Kafka.
• Strong understanding of distributed computing concepts and principles.
• Proficiency in Scala programming language.
• Expertise in Apache Spark framework, including Spark Core, Spark SQL, Spark Streaming.
• Experience with Kafka ecosystem, including Kafka Connect, Kafka Streams.
• Knowledge of SQL and NoSQL databases.
• Familiarity with other big data technologies such as Hadoop, Hive, and/or HBase.
• Experience with cloud computing platforms (e.g., AWS, Azure, GCP) is a plus.
• Experience with version control systems (e.g., Git).
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.

If Interested, Please Share the Following Details:

• Full Name:
• Contact Number:
• Email Address:
• Total Experience:
• Relevant Experience:
• Current Company:
• Current Location:
• Preferred Location:
• Current CTC:
• Expected CTC:
• Notice Period:
• Updated Resume: