Data & Software Engineer

Bespoketechinc; McLean (VA); On-site; USD 90,000 - 120,000; Full time

Job summary

Bespoketechinc in McLean, VA is seeking a Data & Software Engineer to build complex data flows for a custom application. The ideal candidate must have at least 5 years of experience, with strong skills in Python and expertise in data security and compliance principles.

The position requires working fully on-site and active Poly clearance is essential. Responsibilities include designing solutions for data requirements and enhancing pipeline performance.

Qualifications

• Minimum 5 years of experience in data engineering.
• Strong problem-solving skills in data pipeline issues.
• Experience with large-scale data migration or modernization.

Responsibilities

• Work with stakeholders to understand data requirements.
• Debug data quality issues and pipeline failures.
• Contribute to data engineering documentation and best practices.

Skills

Advanced Python programming

Building end-to-end data pipelines

Using orchestration tools for data pipelines

Containerizing applications in cloud environments

MySQL and PostgreSQL

Bash scripting for automation

Integrating AI/ML services

Tools

Apache Spark

Docker

AWS S3

Apache Airflow

Terraform

Job description

BT-338 – Data & Software Engineer

Skill Level: Mid-SME
Location: McLean, VA (fully on-site, no remote option)

Please do NOT apply if you do not have an active Poly clearance. Those without a Poly will not be considered.

The Data & Software Engineer works with a small team to build complex data flows for a custom application. Successful candidate will have advanced Python programming skills, familiarity with Java, an understanding of data security, privacy, governance and compliance principles and a demonstrated history of building production data pipelines and ETL workflows at scale.

Candidate Must Have Experience

• Building end-to-end data pipelines leveraging Python
• Using orchestration tools to deploy data pipelines, including configuring and updating Spark Jobs
• Containerizing and deploying applications in cloud environments like AWS.
• Working with MySQL and PostgreSQL including performance tuning, schema design, and query optimization for complex, analytical workloads.
• Leveraging industry standard tools for code control (Git, IaaC control, etc.)
• Working with data catalogs, tracking data lineage and handling a variety of data formats, including Geospatial.
• Using Bash scripting for automation and data processing tasks
• Integrating Al/ML services and models

Responsibilities

• Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
• Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
• Leverage a background in large-scale data migration or platform modernization efforts
• Contribute to data engineering documentation, best practices, and design patterns.

Required Skills

Minimum of 5 years' experience with:

• Apache Spark & PySpark
• Advanced Python skills (including Pandas & NumPy)
• Docker, Podman
• AWS S3, Lambda & Step functions
• Apache Iceberg, Airflow, etc.
• SQL (with Trino)
• NoSQL, DynamoDB
• Unity Catalog OSS, Apache Polaris
• Apache Superset
• Terraform or CloudFormation
• OpenLineage
• H3, PostGIS