Cloud Infrastructure & Platform Specialist

Cornerstone Global Partners (CGP Group); India; Remote; INR 1,800,000 - 2,400,000; Full time

Benefits offered by this job

Health insurance coverage

Job summary

Cornerstone Global Partners (CGP Group) is seeking a remote Data Platform Engineer based in India (Bangalore preferred). You will own Databricks-based data estate infrastructure on Azure with AWS integration, handling IAM, access controls, and private connectivity to keep teams productive.

The role collaborates with Data Engineering, Data Science, and MLOps teams; you’ll implement least-privilege access, create service principals, and maintain secure environments across DEV, QA, PROD.

Qualifications

• 3-5 years of cloud infra/platform engineering experience.
• Strong IAM provisioning, RBAC, SSO, SCIM, and SAML experience.
• Databricks administration and Unity Catalog experience.
• Hands-on Azure and AWS cloud experience.
• Infrastructure-as-Code with Terraform; other templates valued.

Responsibilities

• Provision, configure, and manage the core cloud infrastructure behind a Databricks-based data estate.
• Manage IAM roles, service principals, and access policies across AWS, Azure, and Databricks.
• Establish secure DEV/QA/PROD environments with guardrails.
• Automate provisioning with IaC and maintain runbooks.
• Oversee security governance and auditing across platform.

Skills

Databricks

IAM & RBAC

Terraform

Azure

AWS

Networking basics

Security governance

Tools

Databricks

Terraform

Azure ARM/Bicep

AWS CloudFormation

Job description

About Our Client

Our client is a growing SaaS company providing cloud-based software that helps energy, utility, telecom, and infrastructure companies protect their critical network infrastructure. With nearly three decades of industry experience, the company serves customers across North America and continues to expand its platform with new data-driven and AI-powered capabilities.

About Our Client

Our client is a growing SaaS company providing cloud-based software that helps energy, utility, telecom, and infrastructure companies protect their critical network infrastructure. With nearly three decades of industry experience, the company serves customers across North America and continues to expand its platform with new data-driven and AI-powered capabilities.

About The Role

Our client is building a governed, multi-cloud data estate on Databricks to power cross-product insights. This role owns the cloud infrastructure and platform side that the Data Engineering, Data Science, MLOps, and Data Architecture teams depend on. You will provision and configure what these teams need in Databricks, in Azure (the backend for all Databricks solutions), and in AWS. Identity and Access Management is the core of the role, from IAM roles and access policies to the service principals that let other applications connect to Databricks. Requests often arrive through help desk tickets, but this is not a support position: your work is what allows the teams to move forward, faster and securely. The role is remote, based in India (Bangalore preferred, Pune also an option), Monday to Friday during standard business hours, with some overlap with the US team a few days a week.

Key Responsibilities

• Provision, configure, and manage the core cloud infrastructure behind a Databricks-based data estate, with Azure as the primary backend for Databricks and additional work in AWS.
• Make Azure configuration changes required for Databricks resources and settings to work as expected.
• Manage infrastructure components such as VPCs/VNets, subnets, route tables, network connectivity, S3 buckets and Azure Storage Accounts, compute resources, and private endpoints.
• Establish and maintain secure DEV, QA, and PROD environments with proper isolation and platform guardrails.
• Build, version, test, and deploy infrastructure changes through Infrastructure-as-Code, mainly Terraform (Azure ARM/Bicep and AWS CloudFormation also apply).
• Provision and manage IAM roles, service principals, managed identities, groups, and access policies across AWS, Azure, and Databricks.
• Create service principals so other applications can connect to Databricks and exchange results.
• Implement least-privilege access and role-based access control (RBAC), and support identity federation and automated identity lifecycle management (SSO, SCIM, SAML, OAuth).
• Review and process elevated or administrative access requests through controlled, auditable procedures, and take part in periodic access reviews.
• Create, configure, and administer Databricks workspaces, including Unity Catalog metastores, catalogs, schemas, storage credentials, external locations, and permissions.
• Define and maintain Databricks cluster policies, instance profiles, and storage credentials for secure, cost-controlled data access.
• Onboard Data Engineering, Data Science, MLOps, and Data Architecture projects by setting up the right catalogs, schemas, access, and infrastructure ahead of time.
• Resolve onboarding and access blockers such as missing admin privileges, cluster-policy conflicts, permission problems, storage-access failures, and authentication issues.
• Act as the primary escalation point for Databricks platform and infrastructure issues.
• Monitor security and governance across the platform, and flag and correct any security issue a team may introduce without realizing it.
• Implement secure private connectivity (AWS PrivateLink/VPC endpoints, Azure Private Link/private endpoints, VNet/VPC peering, network security groups, firewall controls, and egress restrictions).
• Implement encryption and key management using AWS KMS, AWS Secrets Manager, Azure Key Vault, or equivalent services.
• Maintain audit logging and provide evidence for access reviews, administrative activity, infrastructure changes, and security reviews.
• Enforce tagging standards, monitor cloud and Databricks spend, and apply policies that prevent unnecessary spend.
• Automate recurring work such as workspace provisioning, catalog and schema setup, IAM role assignment, and standard onboarding.
• Maintain clear runbooks, architecture documentation, access procedures, and onboarding guides.

Requirements

• 3-5 years of experience in cloud infrastructure, platform engineering, cloud administration, or a closely related role. Strong data engineers with solid Databricks experience and a genuine interest in moving into platform work are also considered.
• Strong, hands-on Identity and Access Management experience: IAM role provisioning, service principals, managed identities, RBAC, least-privilege access, SSO, SCIM, and SAML.
• Hands-on Databricks administration: workspace provisioning and configuration, Unity Catalog, metastores, catalogs and schemas, cluster policies, and data-access controls.
• Hands-on Microsoft Azure experience, as the backend for Databricks.
• Hands-on AWS experience, used alongside Azure.
• Working knowledge of Infrastructure-as-Code, mainly Terraform. Azure ARM/Bicep or AWS CloudFormation is also valued.
• Solid understanding of security and governance, with the judgment to spot and correct security issues across the platform.
• Familiarity with cloud networking and security fundamentals: VPC/VNet architecture, subnets and routing, private connectivity, network segmentation, secrets management, encryption, and key management.
• Ability to quickly troubleshoot infrastructure, networking, identity, permission, and platform-access issues.
• Strong communication skills, with the ability to explain infrastructure and access issues clearly to technical teams.
• Familiarity with enterprise compliance and security frameworks (SOC 2, ISO 27001, GDPR), audit logging, and access reviews.
• Based in India, with Bangalore preferred and Pune also an option, and able to overlap with US hours a few days a week.

Nice-to-Have

• Some data engineering background or experience.
• Databricks certification (e.g., Databricks Platform Administrator).
• Other cloud certifications, such as Azure Solutions Architect, Azure Administrator, or AWS Solutions Architect.
• Infrastructure and platform CI/CD experience, including GitHub Actions, Databricks Asset Bundles (DABs), and DEV to QA to PROD promotion.
• FinOps and cloud cost-management practices.
• Oil & gas, pipeline, utility, energy, or asset-integrity industry experience, including GIS or geospatial data.

What We Offer

• Be part of a dynamic and growing company that is well-respected in its industry.
• Competitive compensation based on experience and qualifications.
• Health insurance coverage.