Databricks Data Platform Engineer | Scale & Security

Intetics; Town of Poland (NY); On-site; USD 120,000 - 160,000; Full time

Job summary

Intetics Inc. is seeking a data engineer to own Databricks production support across our data platform. You will monitor, alert, and respond to incidents while maintaining SLA performance for data pipelines in Azure and AWS environments.

The role emphasizes migrating legacy ETL/ELT to Databricks, optimizing costs, and building scalable, secure, multi-tenant data pipelines. Collaboration with data science, product, and infrastructure teams is essential near daily operations.

Qualifications

• 4+ years of data engineering experience.
• 2+ years with Databricks or Spark in Azure/AWS.
• Proficiency in PySpark, SQL, and Python on production pipelines.
• Hands-on experience with Delta Lake and ACID transactions.
• Experience with pipeline performance tuning and compute optimization.
• Knowledge of PostgreSQL in production data pipelines.
• Experience supporting multi-tenant architectures and tenant isolation.

Responsibilities

• Own Databricks production support across the data platform, including monitoring and incident response.
• Maintain and report SLA performance metrics for data pipeline delivery.
• Identify and implement pipeline optimizations to reduce compute costs and improve throughput.
• Migrate legacy ETL/ELT pipelines to Databricks and build automation tooling.
• Support onboarding by provisioning and hardening tenant data pipelines.
• Design and build high-performance Databricks pipelines ingesting ERP and CRM data at scale.
• Own Delta Lake architecture including schema design, partitioning, data quality enforcement, and incremental processing.
• Enforce data security best practices across Databricks environments (RBAC, secrets management).
• Implement data quality monitoring and observability across pipelines and ML inputs.
• Apply multi-tenant data isolation patterns and ensure reliable data delivery across customers.
• Collaborate with Enterprise Architecture to integrate pipelines with AI/analytics ecosystem.
• Participate in on-call rotation for global operations and incident response.
• Maintain technical documentation and runbooks for operational readiness.
• Apply CI/CD best practices to data pipeline development.

Skills

Databricks

Apache Spark

PySpark

SQL

Python

Delta Lake

PostgreSQL

Multi-tenant

Tools

SSIS

Informatica

Terraform

Azure

AWS

Job description

Intetics Inc. is seeking a data engineer to own Databricks production support across our data platform. You will monitor, alert, and respond to incidents while maintaining SLA performance for data pipelines in Azure and AWS environments.

The role emphasizes migrating legacy ETL/ELT to Databricks, optimizing costs, and building scalable, secure, multi-tenant data pipelines. Collaboration with data science, product, and infrastructure teams is essential near daily operations.