Data Platform Infrastructure Engineer

Nova Web Technologies LLC; Scottsdale (AZ); On-site; USD 140,000 - 190,000; Full time

Job summary

Nova Web Technologies LLC is seeking a senior Data Platform Infrastructure Engineer to design, deploy, and manage scalable data infrastructure across on-premises and cloud environments. You will work with Cloudera, Databricks, AWS, Terraform, and Ansible to build reliable, secure analytics platforms and data pipelines.

The role requires 8+ years of experience, strong administration of Cloudera Hadoop ecosystems, and a proven track record in IaC, automation, and collaboration with

Qualifications

• 8+ years of experience in Data Platform Infrastructure, Platform Engineering, or DevOps.
• Strong experience administering Cloudera Hadoop environments (HDFS, Hive, Spark, YARN).
• Hands-on experience with Databricks administration and cluster management.
• Strong AWS infrastructure experience.
• Expertise in Terraform and Ansible for infrastructure automation.
• Experience managing distributed data clusters in production environments.
• Strong understanding of Infrastructure-as-Code (IaC) principles.
• Experience with Linux system administration and shell scripting.
• Excellent troubleshooting and performance tuning skills.
• Strong communication and collaboration skills.

Responsibilities

• Design, deploy, and manage data platform infrastructure across on-premises (Cloudera) and cloud (AWS, Databricks) environments.
• Build, configure, and maintain distributed data clusters, ensuring high availability, scalability, and performance.
• Automate infrastructure provisioning and configuration using Terraform and Ansible.
• Manage and optimize Cloudera Hadoop ecosystems, including HDFS, Hive, Spark, and YARN.
• Deploy, configure, and administer Databricks workspaces, clusters, and integrations on AWS.
• Implement Infrastructure-as-Code (IaC) and configuration management best practices.
• Monitor platform performance, troubleshoot infrastructure issues, and ensure system reliability.
• Collaborate with Data Engineering, Architecture, and DevOps teams to support enterprise data pipelines and analytics workloads.
• Ensure platform security, governance, compliance, and operational excellence across data environments.

Skills

Cloudera Hadoop

Databricks

AWS

Linux administration

Shell scripting

Troubleshooting

Collaboration

Education

Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field

Tools

Terraform

Ansible

HDFS

Hive

Spark

YARN

Job description

Position Overview
We are seeking a highly skilled

Location: Scottsdale, AZ (100% Onsite)

Experience: 8+ Years

Employment Type: Full-Time / Contract

We are seeking a highly skilled Data Platform Infrastructure Engineer to design, deploy, and manage scalable data platform infrastructure across both on-premises and cloud environments. The ideal candidate will have strong expertise in Cloudera, Databricks, AWS, Terraform, and Ansible, with hands-on experience managing distributed data platforms and infrastructure automation.

This role is responsible for building reliable, secure, and high-performing data infrastructure that supports enterprise-scale analytics and data engineering workloads.

Key Responsibilities

• Design, deploy, and manage data platform infrastructure across on-premises (Cloudera) and cloud (AWS, Databricks) environments.
• Build, configure, and maintain distributed data clusters, ensuring high availability, scalability, and performance.
• Automate infrastructure provisioning and configuration using Terraform and Ansible.
• Manage and optimize Cloudera Hadoop ecosystems, including HDFS, Hive, Spark, and YARN.
• Deploy, configure, and administer Databricks workspaces, clusters, and integrations on AWS.
• Implement Infrastructure-as-Code (IaC) and configuration management best practices.
• Monitor platform performance, troubleshoot infrastructure issues, and ensure system reliability.
• Collaborate with Data Engineering, Architecture, and DevOps teams to support enterprise data pipelines and analytics workloads.
• Ensure platform security, governance, compliance, and operational excellence across data environments.

Must-Have Skills

• 8+ years of experience in Data Platform Infrastructure, Platform Engineering, or DevOps.
• Strong experience administering Cloudera Hadoop environments (HDFS, Hive, Spark, YARN).
• Hands-on experience with Databricks administration and cluster management.
• Strong AWS infrastructure experience.
• Expertise in Terraform and Ansible for infrastructure automation.
• Experience managing distributed data clusters in production environments.
• Strong understanding of Infrastructure-as-Code (IaC) principles.
• Experience with Linux system administration and shell scripting.
• Excellent troubleshooting and performance tuning skills.
• Strong communication and collaboration skills.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Proven experience supporting enterprise-scale data infrastructure.
• Experience working in Agile/Scrum environments.
• Ability to manage multiple priorities while maintaining platform reliability and performance.

Preferred Qualifications

• Experience with Kubernetes and containerized platforms.
• Knowledge of CI/CD pipelines and DevOps best practices.
• Experience with monitoring and observability tools.
• Familiarity with enterprise security, governance, and compliance frameworks.
• AWS or Cloudera certifications are a plus.