Data Engineer – Senior Level

WebSenor Ltd; Bengaluru, Udaipur District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

WebSenor Ltd is seeking a Senior Data Engineer based in Bengaluru, India. This role involves designing and building scalable data pipelines, and implementing Data Lake and Data Warehouse solutions. Ideal candidates have a Bachelor's degree in Computer Science and over 8 years of experience in data engineering, with in-depth knowledge of AWS services.

The role requires strong SQL skills along with proficiency in modern data engineering tools and frameworks.

Qualifications

• Minimum of 8+ years of experience in data engineering.
• Deep experience in designing and building robust, scalable data pipelines.
• Proficiency in SQL and familiarity with various file formats.

Responsibilities

• Design, develop and own robust data pipelines.
• Implement Data Lake, Data Warehouse, and Lakehouse solutions.
• Ensure data quality and governance across the data lifecycle.

Skills

data warehouse

Architectural Patterns

Modern data engineering tools and framework

AWS

SQL

File Formats

Education

Bachelor's degree in computer science, Information Systems, or a related field

Master's degree

Tools

AWS Data Services (S3, Glue, Athena, EMR, Kinesis)

Apache Flink

Kafka Streams

Apache Airflow

pySpark

data integration tools

Job description

Skills: data warehouse, Architectural Patterns, Modern data engineering tools and framework, AWS, SQL, File Formats

Responsibilities

• Design, develop and own robust data pipelines, ensuring optimal performance, scalability, and maintainability.
• Design and implement Data Lake, Data Warehouse and Lakehouse solutions with different architecture patterns.
• Ensure data quality, integrity and governance across all stages of data lifecycle.
• Monitor and optimize performance of data engineering pipelines.
• Contribute to design principles, best practices, and documentation.
• Collaborate closely with cross-functional teams to deeply understand business requirements, translating them into effective technical design, implementations that support the organization's data-driven initiatives.
• Provide mentorship and guidance to other team members of the data engineering team, promoting knowledge transfer, a culture of continuous learning and skills development.

We are looking for Senior Data Engineer

• Bachelor's degree in computer science, Information Systems, or a related field. Master's degree is a plus.
• A seasoned Data Engineer with a minimum of 8+ years of experience.
• Deep experience in designing and building robust, scalable data pipelines – both batch and real-time using modern data engineering tools and frameworks.
• Proficiency in AWS Data Services (S3, Glue, Athena, EMR, Kinesis etc.).
• Strong grip on SQL queries, various file formats like Apache Parquet, Delta Lake, Apache Iceberg or Hudi and CDC patterns.
• Experience in stream processing frameworks like Apache Flink or Kafka Streams or any other distributed data processing frameworks like pySpark.
• Expertise in workflow orchestration using Apache Airflow.
• Strong analytical and problem-solving skills, with the ability to work independently in a fast-paced environment.
• In-depth knowledge of database systems (both relational and NoSQL) and experience with data warehousing concepts.
• Hands-on experience with data integration tools and a strong familiarity with cloud-based data warehousing and processing is highly desirable.
• Excellent communication and interpersonal skills, facilitating effective collaboration with both technical and non-technical stakeholders.
• A strong desire to stay current with emerging technologies and industry best practices in data landscape.