Data Engineer

Tekskills; Hyderabad; Hybrid; INR 900,000 - 1,600,000; Full time

Job summary

Tekskills in Hyderabad is seeking an experienced Hadoop/Spark data engineer to design, develop and optimize big data pipelines that power analytics. You will implement Scala/Java transformations, work with Hive, Spark streaming, and CI/CD practices while ensuring high-quality, scalable solutions.

You will collaborate with analysts and fellow engineers to validate results, write unit tests, and document designs for future operations.

Qualifications

• Total 4–9 years of experience with Hadoop and Spark ecosystems.
• Strong knowledge of data warehousing methodologies.
• Experience with CI/CD, Git, Jenkins and Agile practices.

Responsibilities

• Create Scala/Spark/PySpark jobs for data transformation; produce unit tests.
• Read parquet data with Spark/Spark-SQL; create tables in Hive via Scala API.
• Collaborate with BI analysts to review results and obtain sign-off.
• Prepare design/operations documentation for future use.
• Perform code quality reviews and gatekeeping; pair programming typically.
• Work in collaborative teams; deliver high-quality code.
• Demonstrate understanding of data structures, distributed processing, and automation.
• Familiar with Oracle, Spark streaming, Kafka, ML.
• Develop applications using Hadoop tech stack; deliver on time and cost-effectively.
• Ensure smooth production deployments and post-production verification.
• Hands-on role to develop quality applications within timelines; handle team queries.

Skills

Spark

Hive

SQL

Hadoop

Scala

Java

CI/CD

Git

Jenkins

Agile Methodologies

DevOps

Data warehousing

Tools

Parquet

HDFS

Job description

Experience: 4+Years

Location: Hyderabad

Primary Skillsets: Spark , Hive ,SQL and Hadoop technologies

experience and should have strong experience in Hadoop, Spark, Scala, Java, Hive, Impala, CI/CD, Git, Jenkins, Agile Methodologies, DevOps, Cloudera Distribution. Strong Knowledge in data warehousing Methodology

Roles & Responsibilities:

• 1. Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce unit tests for Spark transformations and helper methods Used Spark and Spark-SQL to read the parquet data and create the tables in hive using the Scala API.
• 2. Work closely with Business Analysts team to review the test results and obtain sign off
• 3. Prepare necessary design/operations documentation for future usage Perform peers
• 4. Code quality review and be gatekeeper for quality checks Hands-on coding, usually in a pair programming environment
• 5. Working in highly collaborative teams and building quality code
• 6. The candidate must exhibit a good understanding of data structures, data manipulation, distributed processing, application development, and automation.
• 7. Familiar with Oracle, Spark streaming, Kafka, ML.
• 8. To develop an application by using Hadoop tech stack and delivered effectively, efficiently, on-time, in-specification and in a cost-effective manner.
• 9. Ensure smooth production deployments as per plan and post-production deployment verification.
• 10. This Hadoop Developer will play a hands-on role to develop quality applications within the desired timeframes and resolving team queries.
• 11. Requirements Hadoop data engineer with total (4 - 6 years) & ( 6 -9 Years) of experience and should have strong experience in Hadoop, Spark, Scala, Java, Hive, Impala, CI/CD, Git, Jenkins, Agile Methodologies, DevOps, Cloudera Distribution. Strong Knowledge in data warehousing Methodology