Data Engineer

Weekday AI (YC W21); Bengaluru; On-site; INR 700,000 - 1,500,000; Full time

Job summary

Weekday AI in Bengaluru, Karnataka is seeking a Data Engineer to design, build, and support modern cloud-based data solutions, handling large datasets and implementing reliable data pipelines. You will work with GCP, Databricks, BigQuery, Python, SQL and Spark/PySpark, collaborating with data engineers, BI teams and architects to enable reporting and analytics.

The role involves batch and near-real-time processing, with a focus on data quality, platform reliability, and continuous engineering

Qualifications

• 2+ years of hands-on data engineering experience.
• Experience with GCP, Azure or AWS.
• Proficiency in Python and SQL.
• Experience with ETL/ELT and data warehousing concepts.
• Familiarity with Apache Airflow or Cloud Composer.

Responsibilities

• Design, develop, and maintain scalable data pipelines on cloud platforms (GCP, Databricks, etc.).
• Build data processing solutions using Python, SQL, Spark, and PySpark.
• Develop and manage data workloads using Databricks Notebooks and Workflows.
• Work with BigQuery and Google Cloud Storage for storage and processing.
• Support batch and near-real-time data ingestion and processing requirements.
• Develop reliable ETL/ELT workflows following data engineering and warehousing best practices.
• Implement monitoring, validation, and quality checks to ensure pipeline reliability.
• Prepare high-quality datasets for BI and analytics teams.
• Collaborate with engineers and architects to improve data platforms and practices.
• Troubleshoot data pipeline issues and optimize workloads for performance.
• Follow development, version-control, CI/CD, and Agile practices.

Skills

Data Engineering

Python

SQL

Spark/PySpark

Databricks

BigQuery

GCP

Airflow

CI/CD

Git

Job description

Job Description:

This role is for one of the Weekdays clients

Salary range: Rs 700000 - Rs 1500000 (ie INR 7-15 LPA)

Experience: 2+ yrs

Location: Bengaluru, Karnataka, India

Job Type: Full-time

We are looking for a skilledData Engineerto design, build, and support modern cloud-based data solutions. This role is ideal for someone who enjoys working with large and complex datasets, developing reliable data pipelines, and transforming raw data into high-quality, analytics-ready information.

You will work across cloud platforms and modern data engineering technologies, with a strong focus onGCP, Databricks, BigQuery, Python, SQL, and Spark/PySpark. You will collaborate closely with data engineers, architects, BI teams, and other technical stakeholders to build scalable data platforms that support reporting, analytics, and business decision-making.

The role offers an opportunity to work across batch and near-real-time data processing while contributing to data quality, platform reliability, and continuous improvements in engineering practices.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ingestion workflows usingGCP, Databricks, or other major cloud platforms.


• Build data processing and transformation solutions usingPython, SQL, Spark, and PySpark.


• Develop and manage data workloads usingDatabricks Notebooks and Workflows.


• Work extensively withBigQuery and Google Cloud Storagefor data storage and processing.


• Support scheduled, batch, and near-real-time data ingestion and processing requirements.


• Develop reliable ETL/ELT workflows while following data engineering and data warehousing best practices.


• Implement monitoring, validation, and quality checks to ensure pipeline reliability and data accuracy.


• Prepare and maintain high-quality datasets for BI, reporting, and analytics teams.


• Collaborate with engineers and architects to improve data platforms, architecture, and engineering practices.


• Troubleshoot data pipeline issues and optimize workloads for performance and scalability.


• Follow established development, version-control, CI/CD, and Agile practices.

What Makes You a Great Fit

• 2+ years of hands-on experience in Data Engineeringor a closely related role.


• Strong practical knowledge of at least one major cloud platform such asGCP, Azure, or AWS.


• Hands-on experience withDatabricks, BigQuery, and cloud storage technologies.


• Strong proficiency inPython and SQL.


• Solid understanding ofETL/ELT processes, data pipelines, and data warehousing concepts.


• Experience working with both structured and unstructured data.


• Familiarity withApache Airflow or Cloud Composer.


• Exposure toAzure Data Factory (ADF)is an advantage.


• Knowledge ofGit, CI/CD, and Agile development methodologies.


• Exposure toKafka, Google Pub/Sub, or other streaming technologiesis a plus.


• Strong analytical and problem-solving abilities with a proactive approach to troubleshooting.


• Good communication skills and the ability to collaborate effectively with technical and cross-functional teams.


• A strong sense of ownership and enthusiasm for learning and working with modern cloud and data technologies.

Requirements: