Big Data Engineer

Ness Digital Engineering; Hyderabad; Hybrid; INR 2,500,000 - 4,000,000; Full time

Job summary

Ness Digital Engineering in Hyderabad and Bangalore is seeking a Data Research Analyst to build and optimize data processing solutions using Spark (Python/Scala) within the Hadoop ecosystem. The role requires strong SQL/NoSQL, Unix/Linux, and DevOps exposure; Agile delivery is mandatory.

Ideal candidates hold a Master’s in CS/IS/Engineering or equivalent, with 8+ years IT experience and 5+ years in Spark. Leadership experience and cloud/data-architecture familiarity are a plus. On-site in India.

Qualifications

• Masters degree in Computer Science, Information Systems or Engineering, or demonstrable equivalence.
• 8+ years of IT experience in application development and support.
• 5+ years of experience with Apache/Pi Spark jobs using Python and Scala.
• Strong experience with Oracle and related technologies.
• Strong SQL and NoSQL database experience.
• Unix/Linux Operating System experience.
• Experience using system tools, source control, utilities and third-party products.
• Experience with scheduling technologies (Autosys).
• Experience with DevOps tools (Git, Jenkins, Ansible, etc.).
• Agile development experience is MUST.
• Experience with Cloud solutions, data models & architecture is a plus.
• Experience with Change Management and defect tracking tools.
• Experience leading small teams.
• Java technologies exposure is a plus.
• Financial domain experience is preferred.

Responsibilities

• Hands on development in Apache/Pi Spark using Python and Scala.
• Deep knowledge of the Hadoop ecosystem.
• Design and develop applications for Ratings Advance analytics and data systems.
• Participate in coding, testing, debugging and documentation processes.
• Develop software based on clear business specs.
• Work on new initiatives and support existing Ratings apps.
• Performance tuning and troubleshooting across applications and systems.
• Integrate in-house apps with vendor platforms.
• Set up development environments/sandboxes.
• Check-in code changes into source repository.
• Perform unit testing and fix errors; interact with customers and IT staff.

Skills

Spark (PySpark/Scala)

SQL/NoSQL

Unix/Linux

Agile

DevOps concepts

Team leadership

Cloud concepts

Change management

Java basics

Data architecture

Education

Master's degree in Computer Science, Information Systems or Engineering

Tools

Git

Jenkins

Ansible

Autosys

Oracle

Job description

Greetings from Ness!

We @Ness, hiring candidates for Data Research analyst role for Hyderabad and Bangalore locations.

JD:

Responsibilities:

• Hands on development experience in writing Apache/Pi Spark jobs using Python and Scala
• Complete knowledge of the Hadoop ecosystem
• Design and development of applications for Ratings Advance analytical, data innovation systems
• Participate in multiple software development processes including Coding, Testing, De-bugging & Documentation.
• Develop software applications based on clear business specifications.
• Work on new initiatives and support existing Ratings applications.
• Perform Application & System Performance tuning and troubleshoot performance issues.
• Integrate in-house applications with various vendor software platforms.
• Setup development environment / sandbox for application development.
• Check-in application code changes into the source repository.
• Perform unit testing of application code and fix errors.Effectively interact with customers, business users and IT staff.

Qualifications:

• Masters degree in Computer Science, Information Systems or Engineering is required, or in lieu, a demonstrated equivalence in work experience.
• 8 + years of IT experience in application development and support.
• 5+ years of Strong experience with Apache/Pi Spark jobs using Python and Scala
• Strong experience with Oracle and related technologies
• Strong experience with SQL and no SQL database technologies.
• Experience with Unix/Linux Operating System.
• Experience using system tools, source control systems, utilities and third-party products.
• Experience using scheduling technologies (i.e. Autosys)
• Experience with DevOps Tools i.e. Git, Jenkins, Ansible etc
• Experience working in an Agile development environment is MUST
• Experience with Cloud Solutions, Design patterns, data models & data architecture & design is a plus.
• Experience with Change Management and defect tracking tools
• Experience in leading and managing small teams.
• Experience with Java Technologies is a plus
• Financial experience is preferred