Senior Databricks Administrator

EXL; United States; Vor Ort; USD 120.000 - 180.000; Vollzeit

Zusammenfassung

EXL Analytics seeks an experienced Databricks Administrator to own and operate our Databricks Lakehouse environments end to end on Azure. You will manage Unity Catalog governance, security, access controls, and compute resources, while partnering with data engineering, data science, and cloud infrastructure teams to deliver a secure, scalable analytics platform.

You will also implement IaC with Terraform, maintain cost controls, and drive performance improvements through workload tuning and

Qualifikationen

• Bachelor's degree in a technical field or equivalent experience.
• Proven hands-on experience administering Databricks in production on Azure.
• Strong knowledge of Unity Catalog and Hive metastore migrations.

Aufgaben

• Own and operate Databricks Lakehouse environments end to end.
• Manage Unity Catalog governance, security, and access control.
• Configure network security, data protection, and cost optimization.
• Collaborate with data engineering, data science, and cloud infra teams.
• Maintain platform documentation and runbooks.

Kenntnisse

Databricks administration

Azure Databricks

Unity Catalog

Security & IAM Entra ID

Terraform / IaC

SQL / Python / Spark

Ausbildung

Bachelor's degree in Computer Science

Tools

Terraform

Databricks CLI

GitHub Actions

Jobbeschreibung

EXL (NASDAQ:EXLS) is a leading operations management and analytics company that helps businesses enhance growth and profitability in the face of relentless competition and continuous disruption. Using our proprietary, award-winning Business EXLerator Framework™, which integrates analytics, automation, benchmarking, BPO, consulting, industry best practices and technology platforms, we look deeper to help companies improve global operations, enhance data-driven insights, increase customer satisfaction, and manage risk and compliance. EXL serves the insurance, healthcare, banking and financial services, utilities, travel, transportation and logistics industries. Headquartered in New York, New York, EXL has more than 73000 professionals in locations throughout the United States, Europe, Asia (primarily India and Philippines), Latin America, Australia and South Africa.

EXL Analytics provides data-driven, action-oriented solutions to business problems through statistical data mining, innovative analytics techniques and a consultative approach. Leveraging proprietary methodology and best-of-breed technology, EXL Analytics takes an industry-specific approach to transform our clients’ decision-making and embed analytics more deeply into their business processes. Our global footprint of nearly 2,000 data scientists and analysts assist client organizations with complex risk minimization methods, advanced marketing, pricing and CRM strategies, internal cost analysis, and cost and resource optimization within the organization. EXL Analytics serves the insurance, healthcare, banking, capital markets, utilities, retail and e-commerce, travel, transportation and logistics industries.

Please visit www.exlservice.com for more information about EXL Analytics.

Role Overview:

We are seeking an experienced Databricks Administrator with deep platform administration expertise to own and operate our Databricks Lakehouse environments end to end. In this role you will be responsible for workspace configuration, Unity Catalog governance, security and access management, compute administration, performance and cost optimization, platform monitoring, and overall platform governance across development, test and production. You will partner closely with data engineering, data science and cloud infrastructure teams to deliver a secure, reliable, well-governed and cost-efficient analytics platform.

Key Responsibilities & Skillsets:

• Administer and configure Databricks workspaces across all environments - account console setup, workspace provisioning, metastore assignment, admin settings, workspace-level features and entitlements.
• Own Unity Catalog administration and governance - metastores, catalogs, schemas, external locations, storage credentials, volumes, data lineage, tagging and fine-grained access controls; lead migration from legacy Hive metastore.
• Manage platform security and access - Entra ID (Azure AD) SSO and SCIM provisioning, groups and entitlements, service principals, personal access token governance, secret scopes, row/column-level security and credential rotation.
• Configure and maintain network and data protection controls - Private Link, VNet injection, IP access lists, encryption with customer-managed keys, and audit log delivery to meet enterprise security and compliance standards.
• Design and administer compute - cluster policies, instance pools, job compute, SQL warehouses, serverless, Photon, autoscaling and Databricks Runtime version standards and upgrade cycles.
• Drive platform performance and cost optimization - workload and query tuning, Delta table maintenance (OPTIMIZE, VACUUM, liquid clustering), tagging, chargeback reporting, budget policies and cost alerts.
• Build platform monitoring and observability using system tables, audit logs, cluster and query metrics, Azure Monitor, Log Analytics and custom alerting for availability, performance, capacity and spend.
• Establish and enforce platform governance - naming, tagging and environment promotion standards, infrastructure-as-code with Terraform and Databricks Asset Bundles, and CI/CD for platform configuration.
• Support day-to-day operations - Workflows and job orchestration, Delta Sharing, incident response, root cause analysis, troubleshooting for engineering and data science teams, and capacity planning.
• Maintain platform documentation, runbooks, disaster recovery and backup procedures, and release management; advise stakeholders on Databricks best practices and new platform capabilities.

Candidate Profile:

• A Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or a similar discipline.
• Strong hands-on experience administering Databricks in production on Microsoft Azure (AWS or GCP exposure a plus), at both account and workspace level.
• Proven experience implementing and administering Unity Catalog, including metastore setup, external locations, storage credentials, privilege models and Hive metastore migration.
• Deep expertise in platform security and access management - Entra ID SSO, SCIM, RBAC, service principals, secrets management, token governance, Private Link and network isolation.
• Strong compute administration skills - cluster policies, pools, SQL warehouses, serverless, runtime upgrades, and workload isolation across shared multi-tenant environments.
• Proficiency in SQL, Python and Apache Spark fundamentals sufficient to troubleshoot jobs and tune performance; working knowledge of Delta Lake internals.
• Experience with automation and infrastructure-as-code - Terraform Databricks provider, Databricks CLI and REST APIs, Asset Bundles, Git and CI/CD pipelines (Azure DevOps or GitHub Actions).
• Hands-on monitoring, logging and cost governance experience using system tables, audit logs, Azure Monitor, Log Analytics, tagging and chargeback models.
• Familiarity with the surrounding Azure data ecosystem - ADLS Gen2, Azure Data Factory, Key Vault, Synapse or Microsoft Fabric, and Microsoft Purview.
• Excellent troubleshooting, documentation and stakeholder communication skills; Databricks certifications (Platform Administrator accreditation, Data Engineer Associate/Professional) and Azure certifications (AZ-104, DP-203) are a plus.

What we offer:

• EXL Analytics offers an exciting, fast paced and innovative environment, which brings together a group of sharp and entrepreneurial professionals who are eager to influence business decisions. From your very first day, you get an opportunity to work closely with highly experienced, world-class analytics consultants.
• You can expect to learn many aspects of businesses that our clients engage in. You will also learn effective teamwork and time-management skills - key aspects for personal and professional growth
• Analytics requires different skill sets at different levels within the organization. At EXL Analytics, we invest heavily in training you in all aspects of analytics as well as in leading analytical tools and techniques.
• We provide guidance/ coaching to every employee through our mentoring program wherein every junior level employee is assigned a senior level professional as advisors.
• Sky is the limit for our team members. The unique experiences gathered at EXL Analytics sets the stage for further growth and development in our company and beyond.