Lead Data Bricks Platform and Service Engineering Specialist

Greymatter Innovationz; Princeton (NJ); On-site; USD 120,000 - 150,000; Full time

Job summary

A leading company is seeking a Lead Data Bricks Platform and Service Engineering Specialist in Princeton, NJ. This role involves optimizing Data Bricks environments and ensuring high performance and security in data workflows. The ideal candidate has over 10 years of experience and will work closely with various teams to enhance data-driven solutions across platforms.

Benefits

Professional development opportunities

Collaborative work environment

Qualifications

• 10+ years of overall experience; 3-4 years with Databricks preferred.
• Hands-on experience with AWS or Azure required.
• Data Bricks Certified Associate Developer or Architect certification preferred.

Responsibilities

• Design and optimize Data Bricks environments across cloud providers.
• Implement Infrastructure as Code solutions for automated management.
• Collaborate with data engineers to deploy and monitor data pipelines.

Skills

Data Bricks environments management

Infrastructure as Code (IaC)

Cloud platforms expertise (AWS, Azure)

Programming in Python, Scala, SQL

Data security and compliance

Problem-solving for distributed systems

Education

Bachelor's in Computer Science or related field

Tools

Terraform

CloudFormation

Git

Jenkins

Databricks CLI

Job description

Lead Data Bricks Platform and Service Engineering Specialist

Lead Data Bricks Platform and Service Engineering Specialist

1 week ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Greymatter Innovationz

Greymatter Innovationz helps you stay digitally relevant across domains, technologies, and skillsets, every day.

We are looking for:

Lead Data Bricks Platform and Service Engineering Specialist

Location: Princeton, NJ

Duration: Long Term

Data Bricks Platform and Service Engineering Specialist

The Data Bricks Platform and Service Engineering Specialist is a key role focused on designing, managing, and optimizing Data Bricks environments and the services running on them. This specialist will bridge the gap between infrastructure management and data engineering by providing scalable, secure, and high-performing solutions on the Data Bricks platform. The role involves working closely with data engineers, platform teams, and business stakeholders to ensure seamless data workflows, robust analytics, and reliable platform operations.

This role is ideal for someone passionate about merging platform engineering with data-driven solutions and enabling teams to unlock the full potential of the Data Bricks platform. An over all experience of 10+ years and Databricks experience of at least 3-4 years is preferred.

• Design, build, and maintain the Data Bricks platform infrastructure across cloud providers (AWS or Azure), ensuring scalability, high availability, and cost efficiency.
• Develop Infrastructure as Code (IaC) solutions (e.g., Terraform, CloudFormation) to automate cluster provisioning, workspace creation, and resource management.
• Implement and manage platform security, including role-based access control (RBAC), service principals, and data encryption.
• Monitor, troubleshoot, and optimize platform performance, leveraging tools like Databricks CLI, REST APIs, and third-party observability platforms.
• Collaborate with data engineering teams to deploy, monitor, and optimize data pipelines, streaming applications, and ML models on Data Bricks.
• Develop and implement strategies for integrating Delta Lake with existing data warehouses, lakes, or real-time streaming systems.
• Provide support for Spark-based applications, ensuring efficient resource utilization and troubleshooting slow-performing jobs.
• Enable seamless CI/CD workflows for notebooks, jobs, and ML models, integrating tools like Jenkins, GitHub Actions, or Azure DevOps.

Governance and Compliance

• Enforce data governance policies using Unity Catalog and other tools to ensure data quality, lineage, and regulatory compliance.
• Design and implement robust logging, monitoring, and alerting systems for platform operations.
• Work closely with stakeholders to establish governance around workspace provisioning, cluster configurations, and data access controls.

Collaboration and Leadership

• Serve as the subject matter expert for the Data Bricks platform, providing guidance to data engineers and analysts on best practices.
• Work cross-functionally with DevOps, IT, and Data Science teams to ensure platform alignment with organizational goals.
• Document platform architecture, operational procedures, and troubleshooting guides to streamline platform management and onboarding.

Required Skills and Qualifications

Technical Expertise

• Extensive experience in managing and optimizing Data Bricks environments and Spark-based workloads.
• Hands-on experience with cloud platforms (AWS, Azure, GCP) and their respective Data Bricks offerings.
• Proficiency in Infrastructure as Code (IaC) tools like Terraform, CloudFormation, or Pulumi.
• Strong programming skills in Python, Scala, and SQL.
• In-depth knowledge of Delta Lake, data partitioning, and schema evolution.
• Familiarity with MLOps tools such as MLflow for tracking experiments and managing models.

Platform and Service Management

• Expertise in CI/CD pipelines and version control systems (e.g., Git, Jenkins, Azure DevOps).
• Solid understanding of access control, data security, and compliance standards (e.g., GDPR, HIPAA).
• Experience with monitoring and observability tools (e.g., Prometheus, Datadog, CloudWatch).

Soft Skills

• Excellent problem-solving and troubleshooting skills for distributed systems and data pipelines.
• Strong communication and collaboration skills to work with diverse technical and non-technical teams.
• Ability to document technical processes and create knowledge-sharing materials.

Preferred Qualifications

• Data Bricks Certified Associate Developer or Architect certification.
• Knowledge of advanced data governance tools such as Alation or Collibra.
• Familiarity with containerization and orchestration (e.g., Docker, Kubernetes).

We offer:

Help you to upgrade yourself to the next level.

And More!!!

Seniority level

• Seniority level

  Executive

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Greymatter Innovationz by 2x

Get notified about new Data Specialist jobs in Princeton, NJ.

Trenton, NJ $61,152.00-$82,992.00 1 month ago

Trenton, NJ $78,624.00-$113,568.00 1 month ago

New Brunswick, NJ $38.00-$38.00 3 months ago

New Jersey, United States $65,000.00-$75,000.00 23 hours ago

Trenton, NJ $79,700.00-$113,000.00 1 hour ago

Iselin, NJ $80,000.00-$90,000.00 1 hour ago

Oncology Data Specialist - ODS - $2500 Sign on Bonus/ FULLY REMOTE!!!

New Jersey, United States $115,000.00-$130,000.00 3 weeks ago

Trenton, NJ $116,480.00-$149,760.00 1 month ago

Trenton, NJ $112,597.33-$152,810.66 1 month ago

Trenton, NJ $116,480.00-$149,760.00 1 month ago

Data Center Engineer - Elite FinTech - $80,000-$150,000 + Bonus

New Jersey, United States $80,000.00-$160,000.00 2 days ago

Skillman, NJ $80,000.00-$90,000.00 1 week ago

Trenton, NJ $58,866.01-$82,829.67 1 hour ago

Princeton, NJ $120,000.00-$150,000.00 2 weeks ago

Data Center Engineer Role || Henrico-Virginia Onsite

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.