Senior Data Engineer

NIRA, Inc.; Washington (District of Columbia); On-site; USD 150,000 - 190,000; Full time

Job summary

NIRA, Inc. is seeking a Senior Data Engineer to support the U.S. DOJ Digital Evidence Review Platform program in Washington, DC.

You will design, develop, test, operate, and optimize high-volume data ingestion and processing pipelines for digital evidence and metadata while preserving provenance, auditability, and chain-of-custody information. The role requires extensive experience in data engineering, cloud platforms, and CI/CD with a focus on secure, auditable data processing across

Qualifications

• Bachelors degree in engineering, mathematics or science as a baseline.
• 15+ years of related data engineering, integration, architecture, or large-scale data processing experience.
• Masters or PhD substitutions described for years of experience.
• Extensive experience building enterprise data ingestion, transformation and processing pipelines.
• Experience with structured and unstructured data from multiple sources and formats.
• Experience with data modeling, metadata, schema design, data lineage, and audit logging.
• Experience developing in Python, Java, Scala, SQL, or comparable data-engineering languages.
• Experience with cloud-based data platforms, distributed processing, object storage, databases, search platforms, or analytics environments.
• Experience supporting CI/CD, automated testing, configuration management, and production debugging.
• S. citizenship and ability to meet DOJ residency and security requirements.
• Ability to obtain and maintain required security credentials.

Responsibilities

• Design, develop, test, operate, and optimize high-volume ingestion and processing pipelines.
• Ingest, normalize, enrich, validate, store, search, export, archive, and restore data.
• Process forensic and metadata sources including documents, multimedia, geolocation, and logs.
• Implement validation, hashing, error handling, retry, reconciliation, and data-quality controls.
• Preserve source identifiers, lineage, provenance, and chain-of-custody events.
• Support OCR, transcription, translation, entity extraction, and metadata enrichment workflows.
• Develop schemas and canonical data models for open, portable exports.
• Support search, analytics, and AI/ML-enabled processing with human oversight and auditability.
• Automate deployment and testing via CI/CD and configuration-management practices.
• Tune pipeline performance and troubleshoot across dev, test, UAT, prod, and DR environments.
• Maintain technical documentation and collaborate with architects, cloud integrators, cybersecurity, and product specialists.

Skills

Python

Java

Scala

SQL

data-engineering languages

Education

Bachelors degree in engineering, mathematics, or science

Masters degree may substitute for two years of required experience

PhD may substitute for four years

Tools

Spark

Databricks

Kafka

Airflow

cloud-native data services

Job description

NIRA is seeking a highly experienced Senior Data Engineer to support the U.S. Department of Justice Digital Evidence Review Platform program. The Data Engineer will design, develop, test, operate, and optimize high-volume data ingestion and processing pipelines for digital evidence and associated metadata. The platform must process large and diverse datasets while preserving integrity, provenance, auditability, access controls, and chain-of-custody information throughout the evidence lifecycle.

Key Responsibilities

• Design, develop, test, operate, and optimize high-volume data ingestion, transformation, integration, and processing pipelines.
• Ingest, normalize, enrich, validate, store, search, export, archive, and restore structured and unstructured data.
• Process sources such as forensic extraction packages, documents, messages, multimedia, call records, geolocation data, cloud-provider returns, metadata, and outputs from Government-furnished parsers and external systems.
• Implement validation, hashing, error handling, retry, reconciliation, reprocessing, and data-quality controls.
• Preserve source identifiers, processing history, lineage, provenance, and chain-of-custody events.
• Support OCR, transcription, translation, deduplication, metadata extraction, entity extraction, and enrichment workflows.
• Develop schemas, canonical data models, metadata structures, and open, portable export formats.
• Support search, analytics, and AI/ML-enabled processing while maintaining human oversight and auditability.
• Automate deployment and testing through CI/CD and configuration-management practices.
• Tune pipeline performance and troubleshoot failures across development, test, UAT, production, and disaster-recovery environments.
• Maintain technical documentation and collaborate with architects, software engineers, cloud integrators, cybersecurity personnel, and product specialists.

Minimum Qualifications

• Bachelor\'s degree in engineering, mathematics, or science.
• At least 15 years of relevant data engineering, data integration, data architecture, software engineering, or large-scale data-processing experience.
• A master\'s degree may substitute for two years of required experience; a PhD may substitute for four years.
• Extensive experience designing, developing, and maintaining enterprise data ingestion, transformation, integration, and processing pipelines.
• Experience processing structured and unstructured data from multiple sources and formats.
• Experience with data modeling, metadata, schema design, data lineage, data quality, error handling, reconciliation, and audit logging.
• Experience developing in Python, Java, Scala, SQL, or comparable data-engineering languages.
• Experience with cloud-based data platforms, distributed processing, object storage, databases, search platforms, or analytics environments.
• Experience supporting CI/CD, automated testing, configuration management, performance tuning, and production troubleshooting.
• S. citizenship and ability to meet DOJ residency and personnel-security requirements.
• Ability to obtain and maintain a High-Risk Public Trust investigation and PIV credential.

Preferred Qualifications

• Experience with digital evidence, forensic data, eDiscovery, law-enforcement data, case-management data, or other high-integrity federal datasets.
• Experience with AWS GovCloud, Azure Government, FedRAMP-authorized SaaS, or hybrid federal cloud environments.
• Experience processing PDFs, text, messages, multimedia, call records, geolocation data, forensic extraction files, or large document collections.
• Experience with OCR, natural-language processing, entity extraction, transcription, translation, deduplication, AI/ML enrichment, or LLM/RAG pipelines.
• Experience with Python-based data frameworks, Spark, Databricks, Kafka, Airflow, cloud-native data services, or comparable technologies.
• Experience implementing hashing, immutable audit records, provenance, lineage, retention, archival, and data-restoration controls.
• Experience designing open-format exports, APIs, canonical schemas, or data-exchange interfaces that reduce vendor lock-in.