Data Engineer - Cloud Analytics

Cintrifuse; Bengaluru; Presencial; INR 1 200 000 - 1 800 000; Tempo integral

Resumo da oferta

FinOpsly in Bengaluru is seeking a Data Engineer to build high-performance pipelines for its Explainable AI engine. You will ingest, normalize, and model billing metadata from AWS/Azure/GCP and Snowflake, ensuring data enables autonomous agents to act on high-fidelity information.

You don’t need a FinOps background; the team will ramp you up. Strong data modeling, pipeline design, and data quality focus will help you succeed in this multi-cloud, data-centric role.

Qualificações

• 3–5 years in a Data Engineering role.
• Hands-on experience with Azure Data Factory.
• Proficiency in Snowflake, SQL, and Python.
• Experience with Star Schema and SCD types.
• Knowledge of REST APIs and data integration.

Responsabilidades

• Ingest raw data from cloud environments into Snowflake staging layers.
• Configure ADF Linked Services, IRs, and pipeline triggers.
• Optimize pipelines for high-volume data workloads.
• Transform and normalize API responses using Python and SQL.
• Design analytics-ready schemas and data models.
• Build data quality frameworks and observability.

Conhecimentos

Data modeling

Data quality assurance

Troubleshooting

Communication

Team collaboration

Formação académica

Bachelor's degree in CS/Engineering

Ferramentas

Azure Data Factory

Snowflake

Python

SQL

Kubernetes

Descrição da oferta de emprego

FinOpsly is an AI-native Value-Control™ platform for cloud (AWS, AZ, GCP), data (Snowflake, Databricks), and AI economics, built to help enterprises move beyond passive cost visibility to active, outcome-driven control. The platform unifies technology spend across cloud infrastructure (AWS, Azure, GCP), data platforms (Snowflake, Databricks), and AI workloads into a single system of action—combining planning, optimization, automation, and financial operations.

We are seeking a Data Engineer with strong technical fundamentals to build the high-performance pipelines that power our Explainable AI engine. You will be responsible for ingesting, normalizing, and modeling complex billing metadata from AWS, Azure, GCP, and Snowflake, ensuring our autonomous agents act on a bedrock of high-fidelity data.

You do not need a FinOps background to succeed here. Our domain is learnable, and our team will ramp you up. What matters is that you are a strong, principled data engineer — someone who thinks carefully about pipeline design, data modeling, and data quality, and who can apply those skills to new problem spaces quickly.

What You’ll Do

1. Ingest raw structured and semi-structured data from customer cloud environments (AWS, Azure, GCP) into Snowflake staging layers.
2. Configure ADF Linked Services, Integration Runtimes, and pipeline triggers to reliably move data across multi-cloud boundaries.
3. Optimize pipeline performance for high-volume datasets, ensuring reliable, low-latency delivery to downstream transformation layers.
4. Pull data from REST APIs (cloud providers, SaaS platforms) and third-party data sources as required by product needs.
5. Transform and normalize API responses using Python and SQL to produce clean, structured tables in Snowflake.
6. Ensure transformed data is structured and optimized for direct consumption by front-end applications and BI tools.
7. Build reusable Python utilities for API pagination, error handling, and incremental data extraction.
8. Work with product and domain experts to gather and analyze business requirements for new product modules — you will learn about the domain context on the job.
9. Translate business requirements (e.g., cost hierarchies, tagging taxonomies, account/namespace structures) into clean, scalable data models.
10. Design analytics-ready schemas (Star Schema, dimensional models) that serve both ML for feature engineering and front-end consumption.
11. Design and implement data quality frameworks that ensure accuracy, completeness, and consistency across all pipeline layers.
12. Propose the right refresh strategies for each table — identifying when SCD Type 1 (overwrite) vs. SCD Type 2 (history-tracking) is appropriate.
13. Implement automated testing and watermarking logic to prevent data gaps or double counting in Financial Reporting
14. Instrument observability metadata: row counts, null rates, freshness, SLAs, and anomaly alerting.

Collaboration & Team Empowerment

1. Work closely with ML engineers, analytics engineers, and product managers to align data models with consumption needs.
2. Write clear technical documentation — data dictionaries, pipeline runbooks, transformation specs — that enables the team to be self-sufficient.
3. Participate actively in design reviews, propose improvements proactively, and share knowledge across the engineering team.
4. Mentor junior engineers and contribute to establishing best practices for pipeline development and data modeling.

What You Bring

1. 3–5 years in a Data Engineering role. SaaS or analytics product experience is a plus; FinOps domain knowledge is not required.
2. Hands-on experience building and managing pipelines in Azure Data Factory (Linked Services, IR, triggers, parameterization).
3. Proficiency in Snowflake — virtual warehouses, clustering keys, streams, tasks, and role-based access.
4. Expert-level SQL for complex transformations; strong Python for API integrations, pipeline utilities, and data validation scripts.
5. Ability to design Star Schemas, handle SCDs (Type 1 & 2), and model hierarchical structured data — domain context will be provided.
6. Practical experience calling REST APIs, handling pagination, auth (OAuth/API keys), and incremental loads.
7. Experience implementing data quality checks, reconciliation logic, uniqueness constraints, and freshness monitoring.
8. General familiarity with AWS, Azure, or GCP environments. Billing-specific knowledge is a nice-to-have, not a requirement.

Core Competencies

• Engineering Fundamentals First
• Analytical Thinking
• Intellectual Curiosity & Fast Learning
• Detail Orientation & Data Integrity
• Team Empowerment
• Communication

Nice to Have

• Infrastructure as Code (IaC): Experience with Terraform or CloudFormation to manage data resources programmatically.
• Performance Engineering: Knowledge of query optimization, partitioning strategies, and indexing in distributed environments.
• Stream Processing: Exposure to real-time data tools like Kafka, Kinesis, or Spark Streaming.
• API Development: Experience building or consuming RESTful APIs or ecosystem integration.
• Containerization: Familiarity with Docker and Kubernetes from a data workload perspective.

What You’ll Gain

• Exposure to enterprise-scale cloud billing datasets
• Mastery of Multi-Cloud FinOps: Become a subject matter expert in the financial logic of AWS, Azure, GCP, and Snowflake.
• AI-Native Product Building: Gain hands-on experience architecting data for 'Agentic AI' and autonomous systems.
• Architectural Ownership: Autonomy to help define data standards and influence the long-term roadmap in a high-growth startup.
• Business & Financial Acumen: Learn to bridge the gap between technical infrastructure and business ROI, a high-value skill for executive leadership.

FinOpsly is an Equal Opportunity Employer. Join us in building the future of Autonomous ROI.