Sr. Data Engineer

Manatal; Bengaluru; On-site; INR 800,000 - 1,500,000; Full time

Job summary

Manatal is seeking an experienced Data Engineer to join their team in Bengaluru, India. You will be responsible for building and maintaining Data Lake, Data Warehouse, and APIs, while collaborating closely with Product and Engineering teams.

The ideal candidate should have at least 6 years of relevant experience, proficiency in Python, and a strong understanding of data architecture and integration tools. Knowledge of the Agriculture Technology domain and Agile methodologies will be an advantage.

Qualifications

• 6+ years experience as a data engineer with hands-on experience in data engineering projects.
• Knowledge of data architecture concepts and data warehousing.
• Experience with Kafka, Kafka Connect, and event-driven architectures.

Responsibilities

• Build and maintain Data Lake, Data Warehouse, and Data Catalog.
• Develop ETL specifications for data integration.
• Run proof-of-concepts for new tools/software.

Skills

Data engineering

SQL queries

API provisioning

Kubernetes

Python

Data warehousing

Data integration tools

Agile methodologies

Education

Bachelor's degree in computer science or related field

Tools

AWS

Github

Jira

Kafka

Job description

• You shall be part of a growing team of Data Engineers, Data Scientists, Data Architect and Data Steward within Farm & Field Data Products (FFDP - Data specialization for Farm & Field initiatives within Digital Ag Solutions)
• Your core deliverables will be to build and maintain Data Lake, Data Warehouse, Data Catalog, Data & Analytics APIs, experimentation workbench for Data Science, Geospatial Analysis, and Business Intelligence
• You shall work closely with different Product and Engineering teams within DAS to build data flows, pipelines with traceable linkage, and lineage
• You shall study the input data source to develop ETL specifications for data integration development and implementation
• Input Data Source could be Kafka Events, streaming data from IoT Sensors, Weather APIs, CSV Files, Image Files, Bulk Data Stores, Batch-loading
• Proficient in data transformations on source data, perform joins and aggregations, build Data Warehouse, feature engineering for Machine Learning and Business Intelligence
• Participate in Data Profiling, Data Cleaning and Quality Management to support Data Stewards and Data Scientists in the team
• You would help run proof-of-concepts to evaluate new tools/software to recommend as potential features to Yara Farm and Field Data Products

Desired Candidate Requirements

• Bachelor's degree in computer science, information technology, or a field related to computer science
• 6+ years experience working as a data engineer with hands‑on experience in data engineering projects such as writing SQL queries, data pipelining, API provisioning, database optimization/maintenance, etc.
• Knowledge of data architecture concepts such as database systems, schemas, schema registry, data warehousing, etc.
• Worked with scalable architectures (micro-services, event-driven, data‑mesh, Kubernetes, etc.) Able to code in Python, our preferred tech stack
• Experience with Kafka, Kafka Connect, KSQL is preferred
• Comfortable working with our tools: AWS, Github, Helmcharts, Jira
• Aligned with our principles: agile, customer‑centric, lean, service‑oriented architecture, self‑organization, transparency
• Hands‑on experience in metadata management, data modelling, data analysis, data integration tools
• Preferable experience in the Agriculture Technology domain
• Good communication and presentation skills
• Experience in Agile (Scrum) way of working
• Experience in working with cross‑geography teams