Data Architect & Engineer: Scalable Pipelines on AWS

Bridge Defense

Washington (District of Columbia)

On-site

USD 120,000 - 180,000

Full time

14 days+

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Bridge Defense is seeking a Data Engineer to design and optimize data pipelines, implement data governance, and partner with a government product team. You will integrate pipelines with AWS services, develop schemas and transforms for ETL, and provide Tier 3 support for data flows.

You will work with stakeholders to address data infrastructure needs and ensure scalable, robust solutions while coordinating with a dedicated data engineering group. This is a high-colocation, mostly onsite role.

Qualifications

Bachelor's degree or equivalent practical experience.
Expertise in distributed computing frameworks to handle large-scale data processing.
Familiarity working with AWS or any other cloud environments.
Working experience with datastores like PostgreSQL, S3, Redshift, MongoDB/DynamoDB, Redis, Elasticsearch/OpenSearch and SQL.
Proficient utilizing Python with key libraries like pandas and PySpark, NiFi, Airflow, AWS Lambda or similar technologies.
Working knowledge with software platforms and services, such as Docker, Kubernetes, JMS/SQS, SNS and Kafka.
Familiar with Linux/Unix server environments.
Experience with Agile development methodology.
Publishing and/or presenting design reports.
Coordinating with other team members to reach project milestones and deadlines.
Working knowledge with Collaboration tools, such as Jira and Confluence

Responsibilities

Identify, design and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
Develop and design data pipelines to support an end-to-end solution.
Develop and maintain artifacts (e.g. schemas, data dictionaries, and transforms related to ETL processes).
Integrate data pipelines with AWS cloud services to extract meaningful insights.
Work with stakeholders to support their data infrastructure needs and assist with data-related technical issues.
Design and develop robust and functional dataflows to support raw data and expected data.
Provide Tier 3 technical support for deployed applications and dataflows.
Define and communicate a clear product vision for our client’s software products, aligning with user needs and business objectives.
Create and manage product roadmaps that reflect both innovation and growth strategies.
Partner with a government product owner and a product team of 7-8 FTEs.
Collaborate with the rest of data engineering team to design and launch new features.
Coordinate and document dataflows, capabilities, etc.
Occasionally (as needed) support to off-hours deployment such as evening or weekends.
Co-location: High colocation, 70-80% onsite