Data & Software Engineer with Security Clearance

Hoplite Solutions LLC; McLean (VA); On-site; USD 150,000 - 175,000; Full time

Job summary

Hoplite Solutions LLC in McLean, VA seeks a Data and Software Engineer to build complex data flows for a custom application. You will leverage advanced Python and PySpark to create end-to-end pipelines, containerize apps, and deploy in AWS.

Knowledge of data security, governance, and lineage is essential. The role requires 5+ years of experience with Spark, Python, SQL (Trino), and NoSQL databases like DynamoDB, plus experience with Airflow, Iceberg, and OpenLineage.

Qualifications

• Minimum 5 years of experience in data engineering with Spark and Python.
• Strong experience with SQL engines (Trino) and NoSQL databases such as DynamoDB.
• Proven track record building production data pipelines and ETL workflows at scale.
• Familiarity with cloud environments (AWS) and containerization (Docker/Podman).

Responsibilities

• Design and implement data pipelines to support the application.
• Collaborate with stakeholders to gather data requirements and ensure data quality.
• Migrate and modernize platforms for scalability and performance.
• Document data engineering standards and best practices.

Skills

Apache Spark

PySpark

Python

Pandas

NumPy

Docker

Podman

AWS S3

Lambda

Step Functions

Apache Iceberg

Airflow

SQL (Trino)

NoSQL

DynamoDB

Unity Catalog OSS

Apache Polaris

Apache Superset

Terraform

CloudFormation

OpenLineage

H3

PostGIS

Bash scripting

ETL pipelines

Data governance

Data security

Tools

Git

Airflow

Terraform

CloudFormation

OpenLineage

Job description

Job Description Hoplite Solutions is seeking a Data and Software Engineer to join our team at McLean, VA. The Data & Software Engineer works with a small team to build complex data flows for a custom application. Successful candidate will have advanced Python programming skills, familiarity with Java, an understanding of data security, privacy, governance and compliance principles and a demonstrated history of building production data pipelines and ETL workflows at scale. Candidate must have experience:

• Building end-to-end data pipelines leveraging Python Using orchestration tools to deploy data pipelines, including configuring and updating Spark Jobs
• Containerizing and deploying applications in cloud environments like AWS.
• Working with MySQL and PostgreSQL including performance tuning, schema design, and query optimization for complex, analytical workloads.
• Leveraging industry standard tools for code control (Git, IaaC control, etc.)
• Working with data catalogs, tracking data lineage and handling a variety of data formats, including Geospatial.
• Using Bash scripting for automation and data processing tasks
• Integrating Al/ML services and models Responsibilities:
• Work with stakeholders to understand data requirements, assess feasibility, and design appropriate solutions with minimal oversight
• Leverage strong problem-solving and debugging skills for data quality issues, pipeline failures, and performance bottlenecks
• Leverage a background in large-scale data migration or platform modernization efforts
• Contribute to data engineering documentation, best practices, and design patterns. Required Skills: Minimum of 5 years' experience with:
• Apache Spark & PySpark
• Advanced Python skills (including Pandas & NumPy)
• Docker, Podman
• AWS S3, Lambda & Step functions
• Apache Iceberg, Airflow, etc.
• SQL (with Trino)
• NoSQL, DynamoDB
• Unity Catalog OSS, Apache Polaris
• Apache Superset
• Terraform or CloudFormation
• OpenLineage
• H3, PostGIS