Hadoop Developer

Veriipro; Addison (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

Veriipro is seeking a Senior Hadoop Developer to design, build, and maintain scalable data pipelines using Hadoop, Spark, and Python in a production environment.

The role requires hands-on expertise with HDFS, Hive, and SQL, plus experience in ETL, data ingestion, and performance tuning. Collaboration with cross-functional teams and adherence to Agile processes are essential.

Qualifications

• 4+ years of experience in Big Data and data warehousing.
• Hands-on with Hadoop/Cloudera, HDFS, Hive, and Spark/PySpark.
• Proficiency in Python, Scala, SQL, and Unix/Shell scripting.
• Strong ETL, data ingestion, transformation and integration experience.
• Experience with complex SQL query optimization and performance tuning.
• Experience with enterprise scheduling tools like AutoSys.
• Strong understanding of distributed systems and large-scale data processing.

Responsibilities

• Develop and maintain scalable Hadoop/Spark-based data pipelines.
• Support data ingestion from multiple source systems.
• Develop ETL and data transformation solutions using Hive, Spark, Python, and SQL.
• Optimize SQL queries and Big Data jobs for performance.
• Troubleshoot production issues and perform root-cause analysis; support scheduling, monitoring, and operational processes.
• Collaborate with architects, developers, analysts, and business teams.
• Evaluate and adopt emerging Big Data technologies where appropriate.

Skills

Big Data

Hadoop/Cloudera

Spark/PySpark

Python/SQL

ETL/Data warehousing

Unix/Shell scripting

Agile methodologies

Analytical skills

Communication & collaboration

Tools

Hadoop

HDFS

Hive

Spark

PySpark

SQL

AutoSys

Job description

We are looking for aSenior Hadoop Developerwith strong experience inBig Data, Hadoop/Cloudera, Spark, Python, SQL, and ETLto design, develop, and support scalable data solutions in a production environment.

Roles and Requirements

• 4+ years of experience inBig Data and data warehousing.
• Strong hands-on experience withHadoop/Cloudera, HDFS, Hive, and Spark/PySpark.
• Proficiency inPython, Scala, SQL, and Unix/Shell scripting.
• Strong experience inETL, data ingestion, transformation, and integration.
• Experience withcomplex SQL query optimization and performance tuning.
• Experience withAutoSysor similar enterprise scheduling tools.
• Strong understanding ofdistributed systems and large-scale data processing.
• Experience supportingproduction Big Data applications and troubleshooting critical issues.
• Understanding ofAgile methodologiesand enterprise development/operational processes.
• Strong analytical, problem-solving, communication, and collaboration skills.
• Experience designinganalytical/data warehouse environments and BI solutions.
• Financial services/banking experience preferred.

Primary Responsibilities

• Develop and maintain scalableHadoop/Spark-based data pipelines.
• Support data ingestion from multiple source systems.
• Develop ETL and data transformation solutions usingHive, Spark, Python, and SQL.
• Optimize SQL queries and Big Data jobs for performance.
• Troubleshoot production issues and perform root-cause analysis.Support scheduling, monitoring, and operational processes.
• Collaborate with architects, developers, analysts, and business teams.
• Evaluate and adopt emerging Big Data technologies where appropriate.