Senior Data Engineer, Databricks & AI-Driven CRM

Intetics; Peru (IL); On-site; USD 120,000 - 180,000; Full time

Job summary

Intetics Inc. is seeking a Senior Data Engineer to own Databricks pipelines, ensuring production reliability and cost efficiency across Azure and AWS.

You will collaborate with ML engineers, product teams, and enterprise architecture to keep the data backbone fast, clean, and scalable for ERP and CRM data. The role emphasizes on-call rotation, incident response, and driving end-to-end data delivery in a distributed environment.

Qualifications

• Proficient in PySpark, SQL, and Python with production-grade pipelines.
• Hands-on with Delta Lake including schema evolution and ACID transactions.
• Experience with Databricks on Azure and/or AWS.
• Strong knowledge of PostgreSQL in production pipelines.
• Experience supporting multi-tenant architectures and data privacy.

Responsibilities

• Own Databricks production support for the predictive data platform, including monitoring and incident response.
• Maintain SLA metrics for data pipeline delivery and platform health.
• Optimize Databricks workloads to reduce compute costs and improve throughput.
• Migrate legacy ETL/ELT pipelines to Databricks and automate data delivery.
• Onboard new customers by provisioning tenant data pipelines with isolation.
• Design and build Databricks pipelines ingesting ERP and CRM data at scale across Azure and AWS.
• Own Delta Lake architecture including schema design and partitioning strategies.
• Enforce data security and access control across Databricks environments.
• Implement data quality monitoring and observability for pipelines and ML inputs.
• Apply CI/CD practices to data pipelines with version control and automated tests.

Skills

PySpark

SQL

Python

Databricks

Delta Lake

PostgreSQL

Data governance

Multi-tenant

Tools

SSIS

Informatica

SQL Server

Job description

Intetics Inc. is seeking a Senior Data Engineer to own Databricks pipelines, ensuring production reliability and cost efficiency across Azure and AWS.

You will collaborate with ML engineers, product teams, and enterprise architecture to keep the data backbone fast, clean, and scalable for ERP and CRM data. The role emphasizes on-call rotation, incident response, and driving end-to-end data delivery in a distributed environment.