Lead AWS Cloud Engineer - Databricks Platform

Greymatter Innovationz; Princeton (NJ); Hybrid; USD 120,000 - 150,000; Full time

Benefits offered by this job

Professional development opportunities

Job summary

A cloud solutions company in Princeton, NJ, is seeking an experienced AWS Cloud Engineer with extensive knowledge of Databricks. This role requires leading cloud solution implementations, managing data pipelines, and mentoring engineers. Candidates should have at least 5 years of AWS experience and strong skills in automation and leadership. The position is hybrid, requiring 3 days onsite.

Qualifications

• 5+ years of experience in AWS cloud engineering.
• 3+ years hands-on experience with Databricks platform.
• 2+ years leading or mentoring engineering teams.

Responsibilities

• Lead design and implementation of AWS solutions.
• Leverage Databricks for data pipelines and analytics.
• Guide team in cloud solutions and best practices.

Skills

AWS Cloud Engineering

Databricks Expertise

Leadership

Automation

Team Mentoring

Education

Bachelor’s degree in Computer Science or related field

AWS Certified Solutions Architect – Professional

Databricks Certified Associate Developer

Tools

AWS Services (EC2, S3, Lambda)

Databricks

Terraform

Docker

Kubernetes

Job description

Direct message the job poster from Greymatter Innovationz

Greymatter Innovationz helps you stay digitally relevant across domains, technologies, and skillsets, every day.

We are looking for:

Location: Princeton, NJ (Hybrid - 3 days onsite)

Duration: Contract Or FTE

Position Overview

We are seeking a highly skilled and experienced AWS Cloud Engineer with a strong background in Databricks to join our dynamic cloud engineering team. The ideal candidate will have extensive experience in designing, deploying, and managing cloud infrastructure on AWS, along with in-depth expertise in the Databricks platform for data engineering, analytics, and machine learning workloads. The candidate should also have prior leadership experience, as they will be responsible for guiding a team of engineers in cloud-related initiatives, best practices, and ensuring the successful delivery of data-driven solutions.

Key Responsibilities

• Lead the design, implementation, and optimization of scalable and secure AWS cloud solutions.
• Utilize AWS services such as EC2, S3, Lambda, RDS, VPC, IAM, and others for building robust cloud-based architectures.
• Work closely with the architecture team to define infrastructure requirements and recommend suitable AWS technologies.
• Databricks Platform Expertise:
• Leverage Databricks for building and managing data pipelines, running Spark-based analytics workloads, and orchestrating machine learning models.
• Design, optimize, and maintain Databricks clusters and notebooks for data processing, transformation, and analysis.
• Integrate Databricks with other AWS services (e.g., S3, Redshift, Glue) to build comprehensive end-to-end data solutions.
• Provide leadership in best practices for working with Databricks to ensure reliability, scalability, and efficiency of data workflows.
• Lead a team of cloud engineers, guiding them in implementing solutions that align with organizational goals and industry best practices.
• Mentor junior team members and provide technical guidance on cloud-related projects, Databricks integration, and infrastructure optimization.
• Facilitate cross-functional collaboration between engineering, data science, and business teams.
• Automation & Optimization:
• Automate cloud infrastructure provisioning and management using Infrastructure-as-Code tools (e.g., Terraform, CloudFormation).
• Continuously monitor and optimize AWS resources to improve performance and reduce costs.
• Troubleshooting & Support:
• Provide expert-level troubleshooting and support for cloud infrastructure, Databricks platform issues, and production systems.
• Identify and resolve performance bottlenecks, security risks, and system inefficiencies across AWS and Databricks environments.
• Documentation & Reporting:
• Document cloud infrastructure, Databricks configurations, and solutions for internal knowledge sharing.
• Prepare regular reports on system performance, cost management, and the status of cloud initiatives.

Key Requirements

• Experience:
• 5+ years of experience in AWS cloud engineering, with a proven track record of delivering cloud-based solutions.
• 3+ years of hands-on experience with Databricks platform, including building and managing data pipelines, notebooks, and Spark-based workloads.
• 2+ years of experience leading or mentoring a team of engineers, with strong leadership and team-building skills.
• Technical Skills:
• Proficient in AWS cloud services (EC2, S3, Lambda, RDS, VPC, IAM, CloudWatch, etc.).
• Strong knowledge of Databricks, Apache Spark, and data engineering best practices.
• Experience with containerization and orchestration tools (Docker, Kubernetes).
• Familiar with Infrastructure-as-Code tools (Terraform, CloudFormation).
• Experience with data processing frameworks, data lakes, and analytics pipelines.
• Additional Skills:
• Strong understanding of security best practices in cloud environments.
• Ability to manage complex cloud projects with multiple stakeholders.
• Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders.
• Experience with Agile methodologies is a plus.

Education & Certifications

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field (or equivalent experience).
• AWS Certified Solutions Architect – Professional (preferred).
• Databricks Certified Associate Developer (preferred, but not required).

We offer:

Help you to upgrade yourself to the next level.