Data Engineer

ValueLabs; Bengaluru Urban; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

ValueLabs in Bangalore seeks a seasoned Data Engineer to design, build, and maintain data pipelines using PySpark and Python. The role focuses on data warehousing, ETL development, and handling large-scale data across banking domains.

The candidate should have 5+ years of commercial data experience, strong SQL/No-SQL skills, and a solid grasp of SDLC, UAT, and production deployments. On-site work in Bangalore is required.

Qualifications

• 5+ years of commercial experience in a data-driven role.
• Hands-on experience building data Marts/ETL Pipelines using PySpark and Python.
• End-to-end SDLC activities including UAT, production deployment, and post-production support.
• Knowledge of data warehousing concepts and working with structured/semi-structured/unstructured data.

Responsibilities

• Build data marts and ETL pipelines using PySpark and Python.
• Lead end-to-end SDLC, including UAT and production deployments.
• Collaborate with cross-functional teams to deliver banking data warehousing solutions.

Skills

PySpark

Python

Hadoop

SQL

Airflow

Tools

Jupyter

Git

No-SQL DBMS

Job description

Experience: 5+ years

Primary Skill: PySpark, Python, Hadoop, SQL, Airflow

Location - Bangalore (WFO)

Key responsibilities include:

5+ years of previous commercial experience in a data-driven role, hands-on experience building data Marts/ ETL Pipelines Pyspark Python expert in ETL scripting Ability to write clean, maintainable, and robust code in Python End to END SDLC, build UAT, UAT bug fix, Production deployment activities and postproduction Support Analytical expert in Oracle query and data analysis, also in pyspark code debugging Understanding and expertise of software engineering concepts and best practices pipelines in production and ability to work across, structured, semi-structured and unstructured data Banking Clients/ Banking Domain Knowledge Data Warehousing strong knowledge base.

• Hands-on experience of technologies like Python, Spark/Pyspark, Hadoop/MapReduce/HIVE, Pandas etc.
• Familiarity with query languages and database technologies, CI/CD, testing and validation of data Tech stack and activities that you would use and perform on a daily basis- o Python o Spark (PySpark) o Jupyter o SQL and No-SQL DBMS o Git (as source code versioning and CI/CD) o Data Analysis Expert o Imputation Techniques o Data Linking / Cleansing o Feature Engineering o Apache Airflow/ Oozie, Jenkins Pipeline knowledge
• Collaborative - able to coordinate with multiple teams and resolve the dependencies and deliver the end goal.
• Excellent interpersonal, and communication skills
• Ability to communicate effectively with technical and non-technical audience
• Ability to work under pressure with a solid sense for setting priorities
• Ability to lead technical work with strong sense of ownership
• Strong command of English language (both verbal and written)