Engineer - Data Analytics

HireWand Technologies; Karnataka; Vor Ort; INR 600.000 - 1.200.000; Vollzeit

Zusammenfassung

HireWand Technologies is seeking a Data Engineer to design, build, and maintain scalable data pipelines, manage databases, and ensure data reliability for analytics.

You will collaborate with data scientists and analysts, optimize SQL queries, work with PySpark, and manage cloud infrastructure across AWS, Azure, and GCP from Bengaluru.

Qualifikationen

• Bachelor’s degree in CS, Engineering, or related field.
• Strong SQL skills and PostgreSQL experience.
• Hands-on Python for data processing.
• Experience with Spark or similar big data frameworks.
• Experience with ETL/data warehousing.
• Familiarity with cloud data platforms (AWS, Azure, GCP).

Aufgaben

• Design, develop, and maintain robust data pipelines using Python and ETL tools.
• Hands-on experience with PySpark Framework.
• Build and optimize data models in PostgreSQL or other relational databases.
• Write Complex SQL queries for data extraction, transformation, and loading.
• Have good experience in Optimizing the Queries for better performance.
• Implement and manage data lakes and related storage solutions.
• Utilize distributed SQL engines such as Trino (PrestoSQL) for querying across data sources.
• Develop Python scripts for data processing, automation, and API integration.
• Ensure data quality, consistency, and accuracy through monitoring and checks.
• Collaborate with data scientists and analysts to fulfill data requirements.
• Maintain data documentation and dictionaries.
• Automate repetitive tasks and improve the efficiency of data processes.
• Troubleshoot and resolve data-related issues.
• Manage and optimize cloud-based data infrastructure (AWS, Azure, GCP).

Kenntnisse

SQL

Python

PySpark

Data pipelines

Data warehousing

PostgreSQL

Cloud platforms

Debugging

Communication

Ausbildung

Bachelor's degree in Computer Science, Engineering, or related field

Tools

PostgreSQL

PySpark

Trino/Presto

Apache Spark

AWS

Azure

GCP

Apache Iceberg

Delta Lake

Apache Hudi

SQL

API

Jobbeschreibung

About the Role

We are looking for a skilled Data Engineer to join our growing data team. In this role, you will design, build, and maintain scalable data pipelines, manage databases, and ensure data is reliable and accessible for analytics and reporting. You will work closely with data scientists, analysts, and other engineers to enable data-driven decision-making across the organization.

Key Responsibilities

• Design, develop, and maintain robust data pipelines using Python and modern ETL tools.
• Better understanding and hands-on experience in PySpark Framework
• Build and optimize data models in PostgreSQL or other relational databases.
• Write Complex SQL queries for data extraction, transformation, and loading.
• Have good experience in Optimizing the Queries for better performance.
• Implement and manage data lakes and related storage solutions.
• Utilize distributed SQL engines such as Trino (PrestoSQL) for querying across data sources.
• Develop Python scripts for data processing, automation, and API integration.
• Ensure data quality, consistency, and accuracy through monitoring and checks.
• Collaborate with data scientists and analysts to fulfill data requirements.
• Maintain data documentation and dictionaries.
• Automate repetitive tasks and improve the efficiency of data processes.
• Troubleshoot and resolve data-related issues.
• Manage and optimize cloud-based data infrastructure (AWS, Azure, GCP).

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Strong SQL skills and experience with relational databases preferably PostgreSQL
• Hands-on experience with Python for data processing.
• Experience with Apache Spark or similar big data processing frameworks.
• Experience with data pipelines, ETL, and data warehousing.
• Familiarity with cloud data platforms (AWS, Azure, GCP).
• Strong problem-solving, analytical, and communication skills.

Preferred / Bonus Skills

• Experience with Apache Iceberg
• Knowledge of distributed query engines like Trino/Presto.
• Experience with data modeling and schema design for large-scale data platforms.
• Familiarity with data security and governance policies.

Must Have Skills

• Strong SQL skills with experience in writing complex queries
• Query optimization and performance tuning
• Hands-on experience with Python for data processing and automation
• Experience with PySpark / Apache Spark framework
• Experience in designing and maintaining ETL/data pipelines
• Experience with relational databases, especially PostgreSQL
• Understanding of data warehousing concepts
• Experience with cloud platforms (AWS / Azure / GCP)
• Data modeling and schema design basics
• Troubleshooting and debugging data-related issuesGood analytical and problem-solving skills

Good to Have Skills

• Experience with Data Lake technologies
• Apache Iceberg / Delta Lake / Apache Hudi
• Trino / Presto distributed SQL engines
• API integration using Python
• Data quality monitoring and validation frameworksAutomation of data workflows/processes
• Experience with large-scale distributed data platforms
• Knowledge of data governance and security policies
• Documentation and data dictionary maintenance
• Collaboration experience with data scientists and analytics teams

Experience: 0-3 Years

Location: Bangalore

Skills

sql, pyspark / spark, python, postgresql, trino, apache iceberg, delta lake, apache hudi, presto, api