Data Engineer & Architect, Digital Products

Better Cotton Initiative; Delhi; On-site; INR 2,445,000 - 2,988,000; Full time

Benefits offered by this job

Hybrid working in Delhi/Ahmedabad

Global travel up to 1 month/year

Flexible core hours 10:00–16:00

Pension scheme

Generous leave package

Parental benefits

Job summary

Better Cotton Initiative is seeking a Data Engineer & Architect, Digital Products, to turn fragmented data into a governed, production-grade data platform. You will define the target-state lakehouse and build pipelines, data models and controls to support field tools, supply chains, traceability and AI-enabled services.

The role focuses on data governance, security, and scalable architecture, enabling evidence-based decisions across Better Cotton and supporting its mission and vision for Change.

Qualifications

• A degree in Computer Science, Engineering or a related field, or equivalent practical experience.
• Strong hands-on data engineering, architecture and production platform delivery experience.
• Experience with Microsoft data ecosystem (Fabric, Azure Synapse, Azure Data Lake, Power BI) and Databricks.
• Experience with distributed systems, large-scale data processing, and optimization.

Responsibilities

• Build and operate ingestion and ETL/ELT pipelines across field tools and systems.
• Design and own the target-state lakehouse and data warehouse architectures.
• Lead migration of high-volume historical data with minimal disruption.
• Establish platform environments, IAM, encryption, and audit logging.
• Implement data catalog, metadata management, lineage and data-quality monitoring.
• Create reusable data products for reporting, traceability, compliance, and AI/ML use cases.

Skills

SQL

Python

Power BI

Databricks

Azure Synapse

Azure Data Lake Storage

Medallion lakehouse

Data governance

English fluency

Education

Degree in Computer Science/Engineering or related field

Tools

Databricks

Azure Synapse

Power BI semantics

Azure Data Lake Storage

Delta Lake

Databricks Workflows

dbt

Job description

Starting Date: December 2026/ January 2027

Contract type: Full-time and open-ended

Salary: From 2,716,737 INR

Location: India; Delhi, Ahmedabad (Hybrid)

Application closing date: 15th October 2026

If we receive a high volume of applications, we may close this vacancy earlier than the advertised deadline, so we encourage you to apply as soon as possible.

About The Job

The Data Engineer & Architect, Digital Products exists to turn Better Cotton's fragmented organisational data into governed, production-grade data infrastructure. Combining architecture with hands-on delivery, the role defines the target-state lakehouse and then builds the platform, pipelines, data models and controls that make it operational, so that field tools, supply chain systems, traceability, compliance, impact reporting and future AI-enabled services can rely on trusted, reusable data. In doing so, the role reduces manual reconciliation, strengthens data quality, security and governance, and enables evidence-based decision-making across Better Cotton in support of its mission and Vision for Change.

The Challenge

Looking ahead to 2030, we are targeting reducing GHG emissions from cotton production by half, big cuts in synthetic pesticide use, improve soil health across our network, improve farm income and support women's empowerment among farmers and farm workers.

Responsibilities

• Build and operate ingestion and ETL/ELT pipelines across field tools, supply chain systems, middleware APIs, legacy systems and partner platforms, so that priority data is delivered reliably, incrementally and observably.
• Design and own the target-state lakehouse architecture, enterprise data warehouse, data marts and data models, ensuring a scalable, secure and maintainable platform that separates transactional and analytical workloads and meets data residency, privacy and retention requirements.
• Lead the phased extraction and migration of high-volume historical data from legacy systems, reconciling large and complex datasets without disrupting live operations.
• Establish and operate platform environments, infrastructure as code, CI/CD, identity, access controls, encryption and audit logging together with IT and Security, keeping the platform secure, repeatable and cost-efficiency.
• Implement the data catalogue, data dictionary, metadata management, automated lineage and data-quality monitoring, translating governance policies into measurable and enforceable platform controls.
• Build trusted, reusable data products for reporting, traceability, compliance, impact measurement, member services and AI/ML use cases, enabling Power BI and analytics teams to work from governed data.

Experience

the following skills, knowledge, and experience:

Essential

• A degree in Computer Science, Engineering or a related field, or equivalent practical experience.
• Strong hands-on experience in data engineering, architecture and production data-platform delivery, including responsibility for ingestion and ETL/ELT pipelines, medallion (bronze/silver/gold) lakehouse, data warehouse and data mart design, distributed processing at scale, and the migration and reconciliation of large or complex datasets, with a track record of delivering platforms that operate in production.
• Experience across the Microsoft data ecosystem, including Microsoft Fabric, Azure Synapse, Azure Data Lake Storage and Power BI semantic models, alongside Databricks and other cloud-native platforms.
• Experience with distributed systems and large-scale data processing, including query optimisation, partitioning and clustering, file and storage optimisation, caching and cost control on consumption-based platforms.
• Experience with orchestration and transformation tooling such as Databricks Workflows, Apache Airflow, Dagster or dbt, applying medallion architecture and ELT-first patterns.
• Experience working in international, multicultural, or mission-driven environments is an asset.
• Professional certifications in Databricks, cloud platforms (AWS or Azure or GCP) or data architecture are desirable but not required.
• Applies expert SQL and strong Python to model, transform and reconcile large datasets efficiently.
• Builds and operates production lakehouse workloads and pipelines using Spark/PySpark, Delta Lake, Unity Catalog and Databricks Workflows, applying medallion architecture.
• Fluency in written and spoken English is essential.

Desirable

• Experience designing and implementing lakehouse, data warehouse and data mart architectures, with a strong understanding of data modelling principles.
• Experience with cloud platforms (Azure, AWS or GCP) and CI/CD practices.
• Knowledge of data governance, security and data quality practices, including access controls, lineage, cataloguing and monitoring frameworks.
• Proficiency in Hindi, Gujarati, Marathi, Tamil or other Indian languages is desirable to support collaboration with local teams, partners and field-level stakeholders.

What We Offer

• Hybrid working opportunity, with the office based in Delhi.
• The opportunity to work from anywhere in the world for up to one month per year
• Flexible working, with core hours from 10 am to 4 pm local time
• Continuous learning and development
• Pension scheme
• A generous annual leave package, including paid holiday, public holidays, additional paid days off during the festive season, study and exam days, and extra discretionary leave for life events such as family weddings or moving home
• Enhanced parental benefits
• The opportunity to make your mark and make a difference.

Working arrangements

The position is full-time (40 hours per week) and will be based in Delhi or Ahmedabad, requiring travel between the two. Better Cotton Initiative offers flexible working, with core hours being 10 am - 4 pm Better Cotton Initiative offers flexible working, with core hours being 10 am - 4 pm. The position may require limited international travel.

Our hiring process

Initial Screening

Applicant screening is a crucial step in our recruitment process. This stage may involve a brief phone interview or video call to evaluate your suitability for the position. During this screening, we review resumes and cover letters, and may conduct initial background checks to narrow down the pool of applicant. Internal candidates are encouraged to inform their line manager about their application to promote transparency and help facilitate a smooth transition if selected.

Interviews

If you successfully pass the initial screening, you will be invited for interviews. The interview process at Better Cotton Initiative may include one or more rounds of interviews with various stakeholders, such as the hiring manager, team members, and potentially senior management. As an internal candidate, you may have discussions regarding your current responsibilities and how they will transition to the new position.

Assessment And Testing

Depending on the role, you might be required to complete assessments, technical tests, or skills evaluations to gauge your competency and suitability for the position. Internal candidates may have the option to demonstrate their existing skills through current job performance or past project outcomes.

Reference Checks

Once you progress further in the process, reference checks are conducted to validate your work history, skills and qualifications, and for some roles, your education and criminal history. Internal candidates may be asked for references from current or previous supervisors within Better Cotton Initiative to provide insights into their performance and capabilities.

Offer Stage

After completing interviews, assessments, and reference checks, the hiring team evaluates your performance and fit for the role. If you are selected and your references are relevant and satisfactory, you will receive a formal job offer letter outlining the terms of employment, compensation, benefits, and any other relevant details. For internal candidates, the offer process may include a discussion about how your transition to the new role will be managed and any support needed for a smooth handover.

About Better Cotton Initiative

Better Cotton Initiative is aglobal not-for-profit organisationsupporting theproduction of moreequitableandsustainable cotton at farmlevel. We exist to catalise the cotton farming communitywithpartners,governments and others to continuously drive positive impact by setting standards and facilitating the supply and demand of more sustainable cotton from farm tobrand. In just 15 years we have aligned more than a fifth of the world’s cotton with our standard and helped farmers and farming communities thrive. Join us in building a better, more sustainable future for cotton worldwide.

About Our Values

• Trustworthy (including honest, transparent, credible)
• As having Integrity (including responsible, authentic)
• Positive (including problem-solving, pragmatic)
• Engaging (including adaptable, inclusive, holistic)
• Daring (including courageous, innovative, game-changing)

Being you @ Better Cotton Initiative

Better Cotton Initiative is committed to creating a diverse environment and is proud to be an equal opportunity employer with a strong commitment to good practice and transparency in the management of natural, human, and financial resources.

We have a zero-tolerance approach to any attitudes or behaviours that put children or adults at risk of harm. Safeguarding incidents are acts of serious misconduct and are grounds for disciplinary action, up to and including, dismissal and referral to relevant authorities for criminal prosecution. Therefore, all offers of employment are subject to satisfactory references and appropriate screening checks.