Data Engineer (WDP)

Bespoke Technologies, Inc.; Herndon (VA); On-site; USD 110,000 - 170,000; Full time

Job summary

Bespoke Technologies, Inc. in the United States is seeking a Mid-level Data Engineer to design, build, and maintain scalable data pipelines that power our platform.

You will work hands-on with Python, Spark, Databricks Delta Live Tables, Terraform, and data quality tooling in a security-conscious environment, with active TS/SCI clearance required. This on-site role in Chantilly/Herndon offers opportunity to implement modern data architectures and collaborate across engineering teams.

Qualifications

• 4+ years of data engineering experience.
• Experience building and maintaining ETL tools and pipelines.
• Proficiency in Python, SQL, and Spark/PySpark.

Responsibilities

• Design, develop, and maintain robust and scalable data pipelines for both new development and ongoing operations & maintenance.
• Build new pipelines using modern, modular patterns like Databricks Delta Live Tables, adhering to governance standards.
• Implement reusable pipeline templates and automated monitoring patterns to ensure consistency and scalability.
• Utilize IaC with Terraform to create consistent CI/CD deployments.
• Integrate automated data quality checks with Great Expectations to ensure data integrity and SLAs.
• Troubleshoot pipeline issues, optimize performance, and contribute to O&M improvements.
• Participate in code reviews and create documentation for ETL mappings, code, and deployment processes.

Skills

Python

SQL

Spark/PySpark

ETL pipelines

Agile/Scrum

Tools

Databricks Delta Live Tables

Terraform

Great Expectations

Airflow

Docker

Kubernetes

NiFi

ElasticSearch

Snowflake

Palantir

Job description

BT-152 – Data Engineer (WDP)

Skill Level: Mid

Location: Chantilly/Herndon

MUST HAVE AN ACTIVE TS OR TS/SCI CLEARANCE TO APPLY. Those without an active security clearance will not be considered.

Role Description: As a Data Engineer, you will be a hands‑on builder and a key member of the team creating and sustaining the data lifeblood of the platform. You will apply your technical skills to develop, deploy, and maintain resilient and efficient data pipelines. This role is perfect for a practitioner who is passionate about leveraging modern tools and automation to solve complex data challenges and deliver high-quality data solutions.

Responsibilities

• Design, develop, and maintain robust and scalable data pipelines for both new development and ongoing operations & maintenance (O&M).
• Build new pipelines using modern, modular patterns like Databricks Delta Live Tables, adhering to established governance standards.
• Implement reusable pipeline templates and automated monitoring patterns to ensure consistency and scalability across all data flows.
• Utilize Infrastructure-as-Code (IaC) with tools like Terraform to create consistent and repeatable CI/CD deployments.
• Integrate automated data quality checks and profiling using frameworks like Great Expectations to ensure data integrity and validate SLAs.
• Troubleshoot pipeline issues, optimize performance, and contribute to the continuous improvement of O&M processes.
• Participate in code reviews and create clear documentation for ETL mappings, code, and deployment processes.

Required Qualifications

• 4+ years of experience in data engineering.
• Experience with the development and maintenance of extract, transform, and load (ETL) tools and services.
• Proficiency in Python, SQL, and Spark/PySpark.
• Experience with cloud data platforms (e.g., AWS, Azure) and data engineering platforms like Databricks, Palantir, or Snowflake.
• Experience working in an Agile/Scrum environment.

Preferred Qualifications

• Direct experience with the data platform.
• Experience working in high-security environments (e.g., SIPR, JWICS).
• Hands‑on experience with Databricks Delta Live Tables and Terraform.
• Familiarity with data orchestration tools (e.g., Airflow) and containerization (Docker, Kubernetes).
• Knowledge of COTS and open-source data engineering tools such as NiFi or ElasticSearch.