Platform Infrastructure Engineer

E-Solutions; Scottsdale (AZ); On-site; USD 120,000 - 190,000; Full time

Job summary

Data Platform Infrastructure Engineer at E-Solutions is tasked with designing, building, and managing scalable data platforms across on-prem and cloud environments. You will work with cluster technologies, IaC, and automation to ensure reliability and high performance for data workloads.

Responsibilities include deploying data platform infrastructure (Cloudera, AWS), managing distributed clusters, implementing Terraform and Ansible, and collaborating with data engineers and DevOps to enable

Qualifications

• Design, deploy, and manage data platform infrastructure across on-prem and cloud.
• Build and optimize distributed data clusters for high availability.
• Automate provisioning using Terraform and Ansible.
• Manage Cloudera Hadoop ecosystems and Databricks workspaces.
• Ensure security, governance, and migrations to cloud.

Responsibilities

• Design, deploy, and manage data platform infrastructure across on-prem (Cloudera) and cloud (AWS, Databricks) environments.
• Build and maintain distributed data clusters, ensuring high availability and performance.
• Automate infrastructure provisioning using Terraform and Ansible.
• Manage Cloudera Hadoop ecosystems (HDFS, Hive, Spark, YARN).
• Deploy Databricks workspaces, clusters, and integrations on AWS.
• Implement IaC and configuration management best practices.
• Monitor cluster performance, troubleshoot, and ensure reliability.
• Collaborate with data engineers and DevOps to support pipelines and analytics workloads.
• Ensure security, compliance, and governance across data platforms.
• Support migration from on-prem to cloud-based platforms.

Skills

Data platform infra

Cluster management

DevOps collaboration

Infrastructure automation

Tools

Cloudera CDH/CDP

Databricks

AWS

Terraform

Ansible

Job description

We are seeking a highly skilled Data Platform Infrastructure Engineer to design, build, and manage scalable data platforms across on-premise and cloud environments. The role involves working with cluster technologies, infrastructure automation, and modern data ecosystems to enable reliable and high-performing data platforms.

Key Responsibilities

• Design, deploy, and manage data platform infrastructure across on-prem (Cloudera) and cloud (AWS, Databricks) environments
• Build and maintain distributed data clusters ensuring high availability, scalability, and performance
• Automate infrastructure provisioning using Terraform and Ansible
• Manage and optimize Cloudera Hadoop ecosystems (HDFS, Hive, Spark, YARN, etc.)
• Deploy and manage Databricks workspaces, clusters, and integrations on AWS
• Implement infrastructure-as-code (IaC) and configuration management best practices
• Monitor cluster performance, troubleshoot issues, and ensure system reliability
• Collaborate with data engineers, architects, and DevOps teams to support data pipelines and analytics workloads
• Ensure security, compliance, and governance across data platforms
• Support migration from on-prem to cloud-based data platforms

Technical Skills Required

Core Technologies

• Strong experience in Cloudera (CDH/CDP) cluster setup and administration
• Hands-on experience with Databricks (cluster management, jobs, notebooks)
• Strong exposure to AWS (EC2, S3, IAM, VPC, EMR, networking concepts)

Infrastructure & Automation

• Expertise in Terraform (mandatory) for infrastructure provisioning
• Proficiency in Ansible for configuration management and automation
• Experience with CI/CD pipelines for infrastructure deployments
• Strong understanding of:
• Spark performance tuning and cluster optimization
• Knowledge of containerization (Docker/Kubernetes) is a plus