Senior DataOps Engineer - Remote/Hybrid Databricks Platform

Amtex Enterprises Inc; Charlotte (NC); Hybrid; USD 110,208 - 117,096; Full time

Job summary

Amtex Enterprises Inc. seeks a Senior DataOps Engineer to design, build, and optimize enterprise-scale data platforms using Databricks and AWS. You will drive platform automation, IaC, and DataOps practices while supporting analytics and AI/ML initiatives.

The role emphasizes administering Databricks environments, CI/CD pipelines, and governance across Bronze-Silver-Gold lakehouse architectures with emphasis on performance and reliability.

Qualifications

• 8+ years of data engineering or data platform experience in enterprise settings.
• 5+ years hands-on Databricks administration in production environments.
• Strong Python, PySpark, Spark SQL, and SQL proficiency.
• Deep knowledge of Delta Lake, Unity Catalog, and cluster tuning.
• Terraform and IaC for cloud infrastructure provisioning.
• CI/CD and DevOps/DataOps practices in modern data platforms.
• Extensive AWS services experience: AWS Glue, Kinesis, Firehose, S3, IAM.
• Experience with Lakehouse/Medallion architectures and governance.

Responsibilities

• Design, build, and support enterprise-scale ETL/ELT pipelines in Databricks.
• Develop batch and streaming data pipelines using PySpark and Delta Lake.
• Configure Databricks clusters for performance, cost, and reliability.
• Administer Unity Catalog, governance, access control, lineage, and security.
• Implement Delta Lake best practices, partitioning, and schema evolution.
• Build and maintain IaC with Terraform for Databricks and AWS infra.
• Develop CI/CD pipelines and automate deployments for data workloads.
• Collaborate with analytics, AI/ML, and engineering teams on data solutions.
• Monitor, troubleshoot, and optimize Databricks jobs and workflows.
• Provide technical leadership and mentoring in DataOps best practices.

Skills

Databricks

Python

PySpark

Spark SQL

SQL

Delta Lake

Unity Catalog

Terraform

IaC

CI/CD

AWS

Kinesis

Firehose

S3

IAM

DataOps

Tools

None

Job description

Amtex Enterprises Inc. seeks a Senior DataOps Engineer to design, build, and optimize enterprise-scale data platforms using Databricks and AWS. You will drive platform automation, IaC, and DataOps practices while supporting analytics and AI/ML initiatives.

The role emphasizes administering Databricks environments, CI/CD pipelines, and governance across Bronze-Silver-Gold lakehouse architectures with emphasis on performance and reliability.