Data Integration Analyst: AI-Ready Data & Databricks ETL

Softek; Piscataway Township (NJ); On-site; USD 140,000 - 180,000; Full time

Job summary

Softek is seeking a senior data engineer with deep database design and migration experience to lead complex transformation projects. You will work with the Databricks stack, building scalable pipelines and ensuring data quality, lineage, and metadata governance.

Proficiency in SQL, Python, and PySpark is required, along with experience enabling AI-ready data through feature engineering and curated datasets.

Qualifications

• 7+ years of experience in database design and data engineering.
• Experience with database transformation and migration projects for software apps.
• Proficiency with Databricks Delta Lake, Unity Catalog and related tools.
• Strong SQL, Python and PySpark skills.

Responsibilities

• Design and manage database transformation and migration initiatives.
• Build and maintain data pipelines using the Databricks stack.
• Apply NLP/LLM techniques for metadata tagging, classification, and extraction.
• Prepare AI-ready data with feature engineering and labeled datasets.

Skills

SQL

Python

PySpark

Data modeling

Data integration

ETL/ELT

Streaming

API data exchange

Data quality

Metadata

Data lineage

Delta Lake

Delta Live Tables

Unity Catalog

Workflows

Structured Streaming

PostgreSQL

Aurora

DynamoDB

Feature engineering

Training data

Semantic embeddings

NLP/LLM techniques

Job description

Softek is seeking a senior data engineer with deep database design and migration experience to lead complex transformation projects. You will work with the Databricks stack, building scalable pipelines and ensuring data quality, lineage, and metadata governance.

Proficiency in SQL, Python, and PySpark is required, along with experience enabling AI-ready data through feature engineering and curated datasets.