AWS Data Engineer

TechNET; India; Remote; INR 1,378,000 - 2,480,000; Part time

Job summary

TechNET is seeking a skilled Data Engineer for a remote contract role in India. The role focuses on cloud-based data engineering, building and maintaining large-scale data pipelines using Python, SQL, PySpark, and Apache Spark.

Candidates should have practical experience with AWS data services (S3, Glue, EMR, Redshift), Databricks, and workflow orchestration with Airflow. The position involves designing data models, lakes, and warehouses for analytics and reporting.

Qualifications

• Hands-on data engineering experience in cloud-based data platforms.
• Strong Python and SQL proficiency and experience with PySpark.
• Experience building end-to-end ETL/ELT pipelines.
• Experience with AWS data services (S3, Glue, EMR, Redshift).
• Knowledge of data warehousing, data lakes, and data modeling.
• Familiarity with Databricks and workflow orchestration (Airflow).
• Experience maintaining production data pipelines in enterprise environments.
• Understanding of data governance and security best practices.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for ingestion and processing.
• Build data processing solutions using Python, PySpark, SQL, and Spark.
• Work with AWS data services (S3, Glue, EMR, Redshift) to support data solutions.
• Develop workflows with Databricks and Airflow for orchestration.
• Integrate data from databases, APIs, files, and enterprise systems.
• Perform data cleansing, validation, and quality checks.
• Design and maintain data models, lakes, and warehouses.
• Troubleshoot production issues and optimize pipelines for performance.
• Implement data security, governance, monitoring, and best practices.
• Collaborate with architects, analysts, and stakeholders to deliver data solutions.
• Follow software engineering practices: versioning, testing, docs, CI/CD.

Skills

Python

SQL

PySpark

Apache Spark

ETL/ELT pipelines

AWS data services

Databricks

Data warehousing

Airflow

Data modeling

Tools

AWS S3

AWS Glue

AWS EMR

Amazon Redshift

Job description

Experience: 4–10 Years

Location: Remote – India

Engagement: Contract

Duration: 6+ months, with possibility of extension

Job Overview

• We are looking for a skilled Data Engineer with hands‑on experience in cloud‑based data engineering, big data technologies, and large‑scale data pipeline development. The ideal candidate will have strong proficiency in Python, SQL, PySpark, ETL/ELT processes, and cloud data services, with experience building, optimizing, and maintaining reliable data solutions for enterprise environments.
• Candidates should have practical experience with AWS data services and distributed data processing technologies. Experience with Databricks, data warehousing, workflow orchestration, and production data pipelines will be highly valued.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, processing, and integration.
• Build data processing solutions using Python, PySpark, SQL, and Apache Spark.
• Work with cloud data services, including AWS S3, Glue, EMR, and Redshift, to develop and maintain enterprise data solutions.
• Develop and optimize data workflows using Databricks and orchestration tools such as Apache Airflow.
• Integrate data from multiple sources, including databases, APIs, files, and other enterprise systems.
• Perform data cleansing, transformation, validation, and quality checks to ensure data accuracy and consistency.
• Design and maintain data models, data lakes, and data warehouse structures to support analytics and reporting.
• Troubleshoot production issues and optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Implement data security, governance, monitoring, and development best practices.
• Collaborate with cross‑functional teams, including architects, analysts, and business stakeholders, to understand requirements and deliver effective data solutions.
• Follow software engineering best practices, including version control, testing, documentation, and CI/CD.

Required Skills and Qualifications

• Hands‑on experience in Data Engineering, Big Data Engineering, or a related technical role.
• Strong proficiency in Python and SQL.
• Experience developing ETL/ELT pipelines and end‑to‑end data processing workflows.
• Hands‑on experience with PySpark and Apache Spark for distributed data processing.
• Practical experience with AWS data services, particularly S3, AWS Glue, EMR, and/or Amazon Redshift.
• Understanding of data warehousing, data modeling, data lake architecture, and distributed processing concepts.
• Experience processing and transforming large volumes of structured and semi‑structured data.
• Ability to troubleshoot data pipeline failures and optimize queries, jobs, and workflows.
• Experience working with production data pipelines in enterprise environments.