Hadoop & PySpark Data Engineer for Big Data

Infosys; Plano (TX); On-site; USD 82,000 - 127,000; Full time

Benefits offered by this job

Medical/Dental/Vision/Life Insurance

Long-term/Short-term Disability

401(k) plan

Job summary

Infosys seeks a Technology Consultant 2 to contribute to requirements elicitation, design discussions, and coding across large-scale Hadoop-based data platforms in the US. The role supports production systems, engages in testing, and helps establish robust data pipelines while ensuring reliability.

You will collaborate with cross-functional teams, document solutions, and train users on new software applications, enabling smooth adoption and ongoing improvements in a dynamic financial services

Qualifications

• Experience in data warehousing architectural approaches.
• Sound understanding and experience with Hadoop ecosystem (Cloudera).
• Experience in working with a Big Data implementation in production environment.
• Experience with Big Data technologies like Hadoop, Hive, Spark, python, etc. Experience in python and Unix shell scripting.
• Experience in orchestration tool like Autosys, Airflow.
• Sound knowledge of relational databases (SQL) and experience with large SQL based systems.
• Understanding of Agile methodologies and technologies.

Responsibilities

• Contribute to the requirements elicitation process by documenting assigned parts of business requirements, in line with guidance provided.
• Facilitate software application design discussions, and document design decisions to guide the technical team towards building software solutions.
• Participate in coding and integrate new features or updates into existing applications, with a focus on maintaining system stability.
• Conduct code reviews, do changes to the codebase and maintain code repositories.
• Implement test strategies, analyse results, and coordinate bug fixes to uphold the software quality standards.
• Develop user training programs, documentation, and support frameworks to ensure a smooth transition to new software applications.
• Actively participate in resolving production issues and recommend preventive strategies to enhance system reliability.
• Maintain detailed records of code, testing techniques, and support activities to enrich the knowledge base and assist other similar projects.

Skills

Communication skills

End-to-end SDLC

Team collaboration

Education

Bachelor’s degree

Tools

Hadoop (Cloudera)

Airflow

Autosys

SQL

Tableau

Job description

Infosys seeks a Technology Consultant 2 to contribute to requirements elicitation, design discussions, and coding across large-scale Hadoop-based data platforms in the US. The role supports production systems, engages in testing, and helps establish robust data pipelines while ensuring reliability.

You will collaborate with cross-functional teams, document solutions, and train users on new software applications, enabling smooth adoption and ongoing improvements in a dynamic financial services