Senior Data Engineer

Weekday 1; Bengaluru; On-site; INR 8,000,000 - 20,000,000; Full time

Job summary

Weekday 1 in Bengaluru is seeking an experienced Senior Data Engineer with 5–15 years of experience to design, build, and optimize scalable data systems and pipelines. The role focuses on data mining, pattern identification, large-scale data processing, and distributed architectures.

You will collaborate with engineering, analytics, product, and business teams. Strong knowledge of ETL/ELT pipelines, SQL, and programming in Python, Java, or Scala is required, with a passion for reliable data

Qualifications

• Experience building ETL/ELT pipelines.
• Strong data mining and pattern identification.
• Experience with large-scale data systems.
• Proficiency in Python, Java, or Scala.

Responsibilities

• Design, develop, and maintain scalable data pipelines, ETL/ELT workflows, and data processing systems.
• Perform advanced data mining across large datasets to identify trends, relationships, anomalies, and insights.
• Analyze datasets to identify patterns and correlations to support business decisions.
• Build and optimize data architectures able to handle large-scale data volumes and high-throughput workloads.
• Develop reliable batch and real-time data processing solutions using distributed computing technologies.
• Collaborate with Data Scientists, Analysts, and Engineers to understand data requirements.
• Design and maintain data models, schemas, and integration frameworks.
• Troubleshoot data and pipeline issues and improve system reliability.

Skills

Data Mining

Pattern identification

Large Scale Systems

SQL

ETL/ELT pipelines

Programming: Python/Java/Scala

Data modeling

Data warehousing

Distributed processing

Data architectures

Job description

This role is for one of Weekday’s clients
Salary range: Rs 8000000 - Rs 20000000 (ie INR 80 - 200 LPA)

Min Experience: 5+ years
Location: Bengaluru
JobType: full-time

We are seeking an experienced Senior Data Engineer with 5–15 years of experience to design, build, and optimize scalable data systems and pipelines. The ideal candidate will have strong expertise in data mining, pattern identification, large-scale systems, data processing, and distributed architectures. You will work closely with engineering, analytics, product, and business teams to transform complex datasets into reliable, actionable data solutions.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable data pipelines, ETL/ELT workflows, and data processing systems.
• Perform advanced data mining across large and complex datasets to identify trends, relationships, anomalies, and valuable insights.
• Analyze datasets to identify meaningful patterns and correlations that can support business and product decisions.
• Build and optimize data architectures capable of handling large-scale data volumes and high-throughput workloads.
• Develop reliable batch and real-time data processing solutions using distributed computing technologies.
• Work with structured and unstructured data from multiple sources and ensure data is accurately processed and transformed.
• Optimize data pipelines, queries, storage, and processing frameworks for performance, scalability, and reliability.
• Implement data quality, validation, monitoring, and error-handling mechanisms across data workflows.
• Collaborate with Data Scientists, Analysts, Software Engineers, Product Managers, and other stakeholders to understand data requirements.
• Design and maintain data models, schemas, and data integration frameworks.
• Troubleshoot complex data and pipeline issues and drive improvements in system reliability and efficiency.
• Contribute to technical architecture decisions and establish best practices for large-scale data engineering.
• Mentor junior and mid-level engineers and contribute to improving engineering standards and processes.

Must-Have Skills

• Data Mining
• Strong understanding of pattern identification, trend analysis, and data exploration
• Experience working with Large Scale Systems
• Strong knowledge of data structures, algorithms, and database concepts
• Hands-on experience building ETL/ELT pipelines and data processing workflows
• Experience with SQL and relational databases
• Proficiency in at least one programming language such as Python, Java, or Scala
• Experience with distributed data processing and large-scale data platforms
• Strong understanding of data modeling, data warehousing, and data architecture
• Experience optimizing data pipelines and processing workloads for performance and scalability
• Strong analytical and problem-solving abilities

Good-to-Have Skills

• Experience with Apache Spark, Kafka, Hadoop, Flink, or similar distributed technologies
• Exposure to cloud platforms such as AWS, Microsoft Azure, or Google Cloud
• Experience with data lakes, lakehouses, and modern data warehouse technologies
• Knowledge of real-time and streaming data architectures
• Experience with data governance, observability, and data quality frameworks
• Familiarity with machine learning or analytics workflows is a plus
• Experience working with high-volume, high-velocity, or complex datasets