Data & Software Engineer: Build Scalable Pipelines in AWS

Hoplite Solutions LLC; McLean (VA); On-site; USD 150,000 - 175,000; Full time

Job summary

Hoplite Solutions LLC in McLean, VA seeks a Data and Software Engineer to build complex data flows for a custom application. You will leverage advanced Python and PySpark to create end-to-end pipelines, containerize apps, and deploy in AWS.

Knowledge of data security, governance, and lineage is essential. The role requires 5+ years of experience with Spark, Python, SQL (Trino), and NoSQL databases like DynamoDB, plus experience with Airflow, Iceberg, and OpenLineage.

Qualifications

• Minimum 5 years of experience in data engineering with Spark and Python.
• Strong experience with SQL engines (Trino) and NoSQL databases such as DynamoDB.
• Proven track record building production data pipelines and ETL workflows at scale.
• Familiarity with cloud environments (AWS) and containerization (Docker/Podman).

Responsibilities

• Design and implement data pipelines to support the application.
• Collaborate with stakeholders to gather data requirements and ensure data quality.
• Migrate and modernize platforms for scalability and performance.
• Document data engineering standards and best practices.

Skills

Apache Spark

PySpark

Python

Pandas

NumPy

Docker

Podman

AWS S3

Lambda

Step Functions

Apache Iceberg

Airflow

SQL (Trino)

NoSQL

DynamoDB

Unity Catalog OSS

Apache Polaris

Apache Superset

Terraform

CloudFormation

OpenLineage

H3

PostGIS

Bash scripting

ETL pipelines

Data governance

Data security

Tools

Git

Airflow

Terraform

CloudFormation

OpenLineage

Job description

Hoplite Solutions LLC in McLean, VA seeks a Data and Software Engineer to build complex data flows for a custom application. You will leverage advanced Python and PySpark to create end-to-end pipelines, containerize apps, and deploy in AWS.

Knowledge of data security, governance, and lineage is essential. The role requires 5+ years of experience with Spark, Python, SQL (Trino), and NoSQL databases like DynamoDB, plus experience with Airflow, Iceberg, and OpenLineage.