Data Engineer – Ratings Engine & Billing Transformation

Talpro; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Talpro is seeking a Data Engineer to join a large-scale Billing Transformation programme. You will source telemetry data, build scalable data pipelines, and support rating logic to prepare invoice-ready outputs for SAP invoicing.

The role requires hands-on experience with AWS Glue, Snowflake, Python, PySpark, SQL, and CI/CD practices, along with data quality checks, reconciliation controls, and operational monitoring.

Qualifications

• Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
• Experience with AWS Glue, Snowflake, PySpark, Python, SQL, and CI/CD practices.
• Experience with batch and event-driven ingestion patterns.
• Understanding of data modelling, partitioning, performance tuning, and cost optimization.

Responsibilities

• Build and maintain data ingestion pipelines from telemetry and usage data sources.
• Develop scalable ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL.
• Support both batch and event-driven data ingestion patterns.
• Prepare billing-ready datasets for invoice processing and SAP invoicing.
• Design data models and optimize queries in Snowflake.

Skills

Data engineering

ETL/ELT development

AWS Glue

PySpark

Python

SQL

CI/CD

Batch ingestion

Event-driven ingestion

Snowflake

Tools

GitHub Actions

GitLab CI

Jenkins

AWS CodePipeline

Job description

Data Engineer – Ratings Engine & Billing Transformation

Talpro is leading the way in transforming the talent acquisition landscape. We deliver innovative, sustainable, and cost-effective recruitment solutions tailored to today’s business needs. Our mission is to offer comprehensive hiring strategies that address immediate recruitment demands while laying the groundwork for long-term success.

Job Description

Data Engineer – Ratings Engine & Billing Transformation

• Role: Data Engineer – Ratings Engine & Billing Transformation
• Primary Skills: AWS Glue, Snowflake, Python, PySpark, SQL, CI/CD
• Domain Exposure: Usage-Based Billing, Rating Engine, SAP Invoice Feeds, Financial Data Processing

Role Overview

We are seeking an experienced Data Engineer to join a large-scale Billing Transformation programme. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing.

This role requires strong hands-on experience in AWS Glue, Snowflake, Python, PySpark, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, exception handling, and operational monitoring to ensure billing accuracy.

This is a critical engineering role supporting the transformation of raw telemetry usage data into rated, validated, and invoice-ready billing data.

Key Responsibilities

• Build and maintain data ingestion pipelines from telemetry and usage data sources.
• Develop scalable ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL.
• Support both batch and event-driven data ingestion patterns.
• Ensure data pipelines are reliable, performant, scalable, and cost-efficient.

Snowflake Data Engineering

• Create and maintain curated usage datasets in Snowflake.
• Design efficient data models, tables, views, and processing layers for downstream billing use cases.
• Optimize SQL queries, warehouse usage, partitioning strategies, and performance.
• Support analytics, validation, and reconciliation use cases on Snowflake.
• Support implementation of rating logic for usage-based billing.
• Prepare billing-ready datasets for invoice processing.
• Generate data outputs and feeds required for SAP invoicing.
• Work on pricing, usage calculation, billing rules, and financial data transformation logic.
• Ensure raw telemetry data is transformed into accurate, validated, and invoice-ready records.

Data Quality, Reconciliation & Audit Controls

• Design and implement data quality checks across ingestion, transformation, rating, and billing layers.
• Build reconciliation controls to validate data completeness, accuracy, and consistency.
• Implement audit trails for billing data movement and transformation.
• Create exception-handling and error-management processes for failed or inconsistent records.
• Support root cause analysis for billing discrepancies and data issues.

CI/CD, Monitoring & Operations

• Build and support CI/CD pipelines for data engineering deployments.
• Work with tools such as GitHub Actions, GitLab CI, Jenkins, AWS CodePipeline, or similar.
• Implement operational monitoring, alerts, dashboards, and pipeline health checks.
• Maintain technical documentation, runbooks, and support procedures.
• Collaborate with engineering, finance, billing, and operations teams to ensure smooth delivery.

Required Skills & Experience

Technical Skills

• Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.
• Experience with:
  • AWS Glue
  • PySpark
  • Python
  • SQL
  • Cloud-based data processing
• Strong experience working with Snowflake as a data warehouse or enterprise data platform.
• Advanced SQL skills for:
  • Data analysis
  • Validation
  • Troubleshooting
  • Query optimization
  • Performance tuning
• Experience with batch and event-driven ingestion patterns.
• Understanding of:
  • Data modelling
  • Partitioning
  • Performance tuning
  • Cost optimization
  • Scalable pipeline design

Billing & Financial Data Skills

• Good understanding of one or more of the following:
  • Usage-based billing
  • Rating engines
  • Invoice feeds
  • SAP invoice integration
  • Financial data processing
  • Billing reconciliation
• Experience transforming usage/telemetry data into billing-ready outputs will be strongly preferred.

Data Quality & Governance

• Experience building:
  • Data quality checks
  • Reconciliation frameworks
  • Audit controls
  • Exception handling processes
  • Error logging and monitoring workflows
• Ability to ensure data accuracy, traceability, and reliability across the billing lifecycle.
• Hands-on experience with CI/CD tools such as:
  • GitHub Actions
  • GitLab CI
  • Jenkins
  • AWS CodePipeline
  • Similar deployment tools
• Familiarity with Git-based workflows and release management.
• Ability to write clean, maintainable, and well-documented code.
• Experience working in Agile delivery environments.

Preferred Skills

• Experience working on large-scale Billing Transformation, Finance Transformation, or Order-to-Cash initiatives.
• Exposure to SAP invoicing, SAP billing integration, or downstream finance systems.
• Experience with operational dashboards and alerting tools.
• Knowledge of data governance, lineage, and metadata management.
• Experience handling high-volume telemetry or usage datasets.

Soft Skills

• Strong analytical and problem-solving skills.
• High attention to detail, especially around billing accuracy and reconciliation.
• Good communication and stakeholder collaboration skills.
• Ability to work with technical, finance, billing, and operations teams.
• Ownership mindset with the ability to deliver in transformation-driven environments.