Hadoop & PySpark Data Engineer

Avance Consulting; Plano (TX); On-site; USD 95,000 - 135,000; Full time

Job summary

Avance Consulting in Plano, TX seeks a Big Data Developer to contribute to requirements elicitation, design discussions, coding, and production issue resolution. The role emphasizes end-to-end SDLC, data warehouses, and Big Data stack experience.

The ideal candidate has hands-on Hadoop, Spark, PySpark, Python, SQL, and Unix shell scripting skills, with production Big Data experience in a fast-paced environment.

Qualifications

• Experience in Hadoop, Python, and Spark.
• Experience end-to-end implementation of data warehouse and data marts; PySpark development.
• Knowledge and hands-on experience in SQL, Unix shell scripting.
• Experience in working with a Big Data implementation in production environment.
• Experience with Big Data technologies like Hadoop, Hive, Spark, Python, etc.
• Experience in Python and Unix shell scripting.

Responsibilities

• Contribute to requirements elicitation by documenting business requirements.
• Facilitate software design discussions and document design decisions for the technical team.
• Participate in coding and integrate features into existing applications while maintaining stability.
• Conduct code reviews and maintain code repositories.
• Implement test strategies, analyze results, and coordinate bug fixes.
• Develop user training programs, documentation, and support frameworks.
• Resolve production issues and recommend preventive measures to improve reliability.
• Maintain detailed records of code, testing techniques, and support activities.

Skills

End-to-end SDLC

Cross-functional collaboration

Effective communication

Testing and troubleshooting

Tools

Hadoop

Hive

Spark

PySpark

Python

SQL

Unix shell

Job description

Avance Consulting in Plano, TX seeks a Big Data Developer to contribute to requirements elicitation, design discussions, coding, and production issue resolution. The role emphasizes end-to-end SDLC, data warehouses, and Big Data stack experience.

The ideal candidate has hands-on Hadoop, Spark, PySpark, Python, SQL, and Unix shell scripting skills, with production Big Data experience in a fast-paced environment.