Data Engineer

Sunday Labs Tech, Inc.; Mumbai; In loco; INR 1.500.000 - 3.000.000; Tempo pieno

Descrizione del lavoro

Sunday Labs Tech, Inc. in Mumbai is seeking a data engineer to design, build and maintain data pipelines for analytics. You will develop ETL workflows with Python and SQL, run PySpark jobs at scale, and leverage AWS Redshift, Databricks or Snowflake, Airflow, and data lake/warehouse concepts to deliver reliable data platforms.

Ideal candidates have hands-on experience with production AWS services and a fintech/financial services background.

Competenze

• Proficiency in Python and SQL, demonstrated through real projects.
• Hands-on experience building and optimizing PySpark applications.
• Proven track record designing and running ETL and data pipeline systems.
• Deep experience working with AWS services in production environments.
• Practical exposure to AWS Redshift for data warehousing and analytics.
• Familiarity with Databricks or Snowflake platforms.
• Understanding of data lake and data warehouse concepts and implementation.
• Knowledge of Airflow for workflow orchestration is valued.
• Background in FinTech, Banking, BFSI, or Financial Services

Mansioni

• Design, build and maintain data pipelines moving data from sources into centralized storage.
• ETL workflows using Python and SQL at scale.
• PySpark jobs for distributed processing across large datasets.
• Use AWS services to build robust data architectures, with depth in Redshift.
• Design data lake and data warehouse architectures.
• Orchestrate workflows with Airflow.
• Work with Databricks or Snowflake for transformations.

Conoscenze

Python

SQL

Strumenti

PySpark

Airflow

AWS

Redshift

Databricks

Snowflake

Descrizione del lavoro

Batch: 2022/2023/2024.

About the role

• - You will design, build and maintain data pipelines that move information reliably from multiple sources into centralized storage systems
• - Your work spans extracting raw data, transforming it to match business requirements, and loading it into repositories ready for analysis
• - You will ensure data quality and consistency across the infrastructure you own
• - You will monitor pipeline performance and troubleshoot failures to keep data flowing without interruption

What you'll work on

• - ETL workflows using Python and SQL to process structured and unstructured data at scale
• - PySpark jobs for distributed data processing across large datasets
• - AWS services to build robust data architectures, with particular depth in Redshift for analytics
• - Data lake and data warehouse designs that organize information efficiently
• - Orchestration of complex workflows using tools like Airflow
• - Modern data platforms including Databricks or Snowflake for advanced transformations

What we're looking for

• - Proficiency in Python and SQL, demonstrated through real projects
• - Hands-on experience building and optimizing PySpark applications
• - Proven track record designing and running ETL and data pipeline systems
• - Deep experience working with AWS services in production environments
• - Practical exposure to AWS Redshift for data warehousing and analytics
• - Familiarity with Databricks or Snowflake platforms
• - Understanding of data lake and data warehouse concepts and implementation
• - Knowledge of Airflow for workflow orchestration is valued
• - Background in FinTech, Banking, BFSI, or Financial Services

Good to know

• - Immediate joiners only