Data Engineer – Cloud Analytics

Finopsly Inc.; India; On-site; INR 900,000 - 1,300,000; Full time

Job summary

FinOpsly, an AI-native Value-Control platform, is seeking a Data Engineer in India to build high-performance pipelines powering our Explainable AI engine. You will ingest, normalize, and model complex billing metadata from AWS, Azure, GCP, and Snowflake, ensuring autonomous agents act on accurate data.

You do not need FinOps background to succeed; strong data engineering fundamentals, pipeline design, data modeling, and data quality are essential as you tackle multi-cloud data flows and

Qualifications

• 3–5 years in a Data Engineering role.
• Hands-on experience building pipelines in Azure Data Factory.
• Proficiency in Snowflake — warehouses, clustering keys, streams, tasks, and RBAC.
• Expert-level SQL for complex transformations; strong Python for API integrations and data validation.
• Ability to design Star Schemas, handle SCDs (Type 1 & 2), and model hierarchical data.
• Experience calling REST APIs with pagination, auth, and incremental loads.
• Experience implementing data quality checks, reconciliation logic, and freshness monitoring.
• Familiarity with AWS, Azure, or GCP environments; billing knowledge is a plus.

Responsibilities

• Ingest raw structured and semi-structured data from customer cloud environments into Snowflake staging layers.
• Configure ADF Linked Services, Integration Runtimes, and pipeline triggers for multi-cloud data movement.
• Optimize pipeline performance for high-volume datasets with low-latency delivery.
• Extract data from REST APIs and third-party sources as required by product needs.
• Transform and normalize API responses to produce clean, structured Snowflake tables.
• Ensure transformed data is ready for front-end consumption and BI tools.
• Build reusable Python utilities for API pagination, error handling, and incremental loads.
• Collaborate with product/domain experts to gather business requirements for new modules.
• Translate business requirements into clean, scalable data models (cost hierarchies, tagging, namespaces).
• Design analytics-ready schemas (Star Schema, dimensional models) for ML and frontend use.
• Implement data quality frameworks for accuracy and completeness across pipelines.
• Propose refresh strategies (SCD Type 1 vs Type 2) for tables.
• Implement automated testing and watermarking to prevent data gaps/double counting in reporting.
• Instrument observability: row counts, null rates, freshness, SLAs, anomaly alerts.
• Collaborate with ML engineers and analytics teams to align data models with usage needs.
• Write data dictionaries, pipeline runbooks, and transformation specs.

Skills

SQL

Python

Data modeling

SCDs (Type 1 & 2)

REST API integrations

Data quality checks

Cloud platforms (AWS/Azure/GCP)

Tools

Azure Data Factory

Snowflake

Job description

FinOpslyis an AI-native Value-Control™ platformfor cloud (AWS, AZ, GCP), data (Snowflake, Databricks), and AI economics, built to help enterprises move beyond passive cost visibility to active, outcome-driven control. The platform unifies technology spend across cloud infrastructure (AWS, Azure, GCP), data platforms (Snowflake, Databricks), and AI workloads into a single system of action—combining planning, optimization, automation, and financial operations.

We are seeking a Data Engineer with strong technical fundamentals to build the high-performance pipelines that power our Explainable AI engine. You will be responsible for ingesting, normalizing, and modeling complex billing metadata from AWS, Azure, GCP, and Snowflake, ensuring our autonomous agents act on a bedrock of high-fidelity data.

You do not need a FinOps background to succeed here. Our domain is learnable, and our team will ramp you up. What matters is that you are a strong, principled data engineer — someone who thinks carefully about pipeline design, data modeling, and data quality, and who can apply those skills to new problem spaces quickly.

What You’ll Do

• Ingest raw structured and semi-structured data from customer cloud environments (AWS, Azure, GCP) into Snowflake staging layers.
• Configure ADF Linked Services, Integration Runtimes, and pipeline triggers to reliably move data across multi-cloud boundaries.
• Optimize pipeline performance for high-volume datasets, ensuring reliable, low-latency delivery to downstream transformation layers.
• Pull data from REST APIs (cloud providers, SaaS platforms) and third-party data sources as required by product needs.
• Transform and normalize API responses using Python and SQL to produce clean, structured tables in Snowflake.
• Ensure transformed data is structured and optimized for direct consumption by front-end applications and BI tools.
• Build reusable Python utilities for API pagination, error handling, and incremental data extraction.
• Work with product and domain experts to gather and analyze business requirements for new product modules — you will learn about the domain context on the job.
• Translate business requirements (e.g., cost hierarchies, tagging taxonomies, account/namespace structures) into clean, scalable data models.
• Design analytics-ready schemas (Star Schema, dimensional models) that serve both ML for feature engineering and front-end consumption.
• Design and implement data quality frameworks that ensure accuracy, completeness, and consistency across all pipeline layers.
• Propose the right refresh strategies for each table — identifying when SCD Type 1 (overwrite) vs. SCD Type 2 (history-tracking) is appropriate.
• Implement automated testing and watermarking logic to prevent data gaps or double counting in Financial Reporting
• Instrument observability metadata: row counts, null rates, freshness, SLAs, and anomaly alerting.
• Work closely with ML engineers, analytics engineers, and product managers to align data models with consumption needs.
• Write clear technical documentation — data dictionaries, pipeline runbooks, transformation specs — that enables the team to be self-sufficient.
• Participate actively in design reviews, propose improvements proactively, and share knowledge across the engineering team.
• Mentor junior engineers and contribute to establishing best practices for pipeline development and data modeling.

What You Bring

• 3–5 years in a Data Engineering role. SaaS or analytics product experience is a plus; FinOps domain knowledge is not required.
• Hands‑on experience building and managing pipelines in Azure Data Factory (Linked Services, IR, triggers, parameterization).
• Proficiency in Snowflake — virtual warehouses, clustering keys, streams, tasks, and role‑based access.
• Expert‑level SQL for complex transformations; strong Python for API integrations, pipeline utilities, and data validation scripts.
• Ability to design Star Schemas, handle SCDs (Type 1 & 2), and model hierarchical structured data — domain context will be provided.
• Practical experience calling REST APIs, handling pagination, auth (OAuth/API keys), and incremental loads.
• Experience implementing data quality checks, reconciliation logic, uniqueness constraints, and freshness monitoring.
• General familiarity with AWS, Azure, or GCP environments. Billing‑specific knowledge is a nice‑to‑have, not a requirement.

Core Competencies

• Engineering Fundamentals First
• Analytical Thinking

Nice to Have

• Infrastructure as Code (IaC): Experience with Terraform or CloudFormation to manage data resources programmatically.
• Performance Engineering: Knowledge of query optimization, partitioning strategies, and indexing in distributed environments.
• Stream Processing: Exposure to real‑time data tools like Kafka, Kinesis, or Spark Streaming.
• API Development: Experience building or consuming RESTful APIs or ecosystem integration.
• Containerization: Familiarity with Docker and Kubernetes from a data workload perspective.

What You’ll Gain

• Exposure to enterprise‑scale cloud billing datasets
• Mastery of Multi-Cloud FinOps: Become a subject matter expert in the financial logic of AWS, Azure, GCP, and Snowflake.
• AI‑Native Product Building: Gain hands‑on experience architecting data for 'Agentic AI' and autonomous systems.
• Architectural Ownership: Autonomy to help define data standards and influence the long‑term roadmap in a high-growth startup.
• Business & Financial Acumen: Learn to bridge the gap between technical infrastructure and business ROI, a high‑value skill for executive leadership.

FinOpsly is an Equal Opportunity Employer. Join us in building the future of Autonomous ROI.