Innovation and Automation Data Engineer (WDP)

Bespoke Technologies, Inc.; Herndon (VA); On-site; USD 140,000 - 210,000; Full time

Job summary

Bespoke Technologies, Inc. in Chantilly/Herndon seeks an Innovation and Automation Data Engineer Lead to steer a hands-on team focused on building reusable data pipeline templates, automated deployment patterns, and data quality frameworks.

You will set the technical vision for data engineering automation while mentoring the team and tackling complex challenges. You will lead the design of standardized ETL/ELT templates, implement DataOps CI/CD, and integrate data quality tooling into pipelines,

Qualifications

• 5+ years of experience in data engineering with a focus on automation and platform development.
• At least 2 years in technical leadership or mentorship.
• Expert-level Python and SQL.
• Proven experience building and optimizing data pipelines on Databricks or Snowflake.
• Hands-on IaC and CI/CD experience (Terraform, GitLab CI, Jenkins).
• Deep understanding of data architecture, data modeling, and modern data warehousing concepts.

Responsibilities

• Lead and mentor a team of automation-focused data engineers responsible for building and maintaining core data infrastructure frameworks.
• Design and develop standardized, reusable ETL/ELT data pipeline templates for ingestion, transformation, and data product creation.
• Architect and implement the CI/CD for Data (DataOps) strategy, enabling automated testing, validation, and deployment of data pipelines.
• Integrate and manage automated data quality, testing, and monitoring tools (e.g., Great Expectations) into core pipeline templates.
• Develop and maintain a library of IaC modules (Terraform) for provisioning common data engineering resources.
• Collaborate with platform and data engineering teams to identify complex problems, troubleshoot bottlenecks, and engineer automated solutions.
• Drive innovation by researching, prototyping, and implementing new technologies and patterns to improve efficiency and reliability of data engineering processes.
• Document data workflows and automation processes to ensure they are understood and adopted across the organization.

Skills

Python

SQL

Data engineering

Databricks

Snowflake

Terraform

CI/CD

Great Expectations

Airflow

Docker

Kubernetes

Tools

Terraform CI/CD tools (GitLab CI/Jenkins)

GitLab CI

Jenkins

Airflow

Docker

Kubernetes

Databricks

Snowflake

Job description

BT-161 - Innovation and Automation Data Engineer Lead (WDP)

Location: Chantilly/Herndon

• MUST HAVE AN ACTIVE TS OR TS/SCI CLEARANCE TO APPLY. Those without an active security clearance will not be considered.**

Role Description

As the Innovation and Automation Data Engineer Lead, you will be a hands-on leader of a specialized team that serves as the engineering backbone for our entire data organization. You will combine deep expertise in data engineering with a passion for automation to build the foundational frameworks that our Data Engineering and Data Product teams use every day. Your primary mission is to design, create, and manage reusable data pipeline templates, automated deployment patterns, and data quality frameworks. This is a "player-coach" role where you will set the technical vision for data engineering automation while mentoring a team and contributing directly to its most complex challenges.

Responsibilities

• Lead and mentor a team of automation-focused data engineers responsible for building and maintaining core data infrastructure frameworks.


• Design and develop standardized, reusable ETL/ELT data pipeline templates for ingestion, transformation, and data product creation.


• Architect and implement the CI/CD for Data (DataOps) strategy, enabling automated testing, validation, and deployment of data pipelines.


• Integrate and manage automated data quality, testing, and monitoring tools (e.g., Great Expectations) directly into the core pipeline templates.


• Develop and maintain a library of Infrastructure-as-Code (IaC) modules (e.g., Terraform) for provisioning common data engineering resources.


• Collaborate with platform and data engineering teams to identify complex problems, troubleshoot bottlenecks, and engineer automated solutions.


• Drive innovation by researching, prototyping, and implementing new technologies and patterns to improve the efficiency and reliability of our data engineering processes.


• Document data workflows and automation processes to ensure they are understood and adopted across the organization.

Required Qualifications

• 5+ years of experience in data engineering, with a strong focus on automation and platform development.


• At least 2 years of experience in a technical leadership or mentorship role, supervising a team or guiding junior engineers.


• Expert-level skills in Python and SQL.


• Proven experience building and optimizing data pipelines on platforms like Databricks or Snowflake.


• Hands-on expertise with Infrastructure-as-Code (Terraform) and CI/CD tools (e.g., GitLab CI, Jenkins).


• Deep understanding of data architecture, data modeling, and modern data warehousing concepts.

Preferred Qualifications

• Direct experience with the data platform.


• Experience with data orchestration tools like Airflow.


• Familiarity with containerization technologies (Docker, Kubernetes).


• Strong project management skills with experience in cross-functional initiatives.