Data Analyst /Data Integration/Quality & Governance

Softek International; Arlington (VA); On-site; USD 130,000 - 170,000; Full time

Job summary

Softek International is seeking a senior data engineer to design and manage databases, lead transformation and migration projects, and prepare AI-ready data in a cloud-based environment. The role involves implementing Databricks pipelines and AWS data services to enable scalable data processing and analytics.

The candidate should have strong SQL/Python skills, experience with PySpark, and familiarity with NLP/LLM techniques for metadata tagging and data quality frameworks.

Qualifications

• 7+ years of relevant experience.
• Experience in database design and management of database transformation and/or migration projects for software applications.

Responsibilities

• Prepare AI-ready data: feature engineering, labeled training datasets, and semantic embeddings.
• Apply NLP/LLM techniques for automated metadata tagging, classification, and entity/relationship extraction.

Skills

SQL

Python

PySpark

ETL/ELT

NLP/LLM

Tools

Databricks

Delta Lake

Delta Live Tables

Unity Catalog

Workflows

Structured Streaming

SQL Warehouses

AWS S3

AWS Glue

AWS Lambda

PostgreSQL

Aurora

DynamoDB

Job description

• Experience in database design and management of database transformation and/or migration projects for software applications

Description

• 7+ years of relevant experience
• Experience in database design and management of database transformation and/or migration projects for software applications

Databricks: Delta Lake, Delta Live Tables, Unity Catalog, Workflows, Structured Streaming, SQL Warehouses

• Languages: SQL, Python, PySpark/Spark
• AWS data services: S3, Glue, Lambda
• ETL/ELT, streaming ingestion, and API-based data exchange
• Data quality, metadata, and lineage tools (e.g., Unity Catalog lineage, data quality frameworks)
• Databases: PostgreSQL, Aurora, DynamoDB
• Preparing AI-ready data: feature engineering, labeled training datasets, and semantic embeddings

NLP/LLM techniques for automated metadata tagging, classification, and entity/relationship extraction