Data Engineer: AI-Ready Data Pipelines & Platforms

Black Canyon Consulting; Washington (District of Columbia); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, vision coverage

401k plan with employer contribution

Paid holidays and vacation

Tuition reimbursement

Job summary

Black Canyon Consulting seeks a Data Engineer to design, develop, and maintain data infrastructure that powers AI, machine learning, and analytical workloads. You will build scalable pipelines, ETL/ELT workflows, and APIs across databases, data lakes, and warehouses to ensure reliable data for analytical systems.

The role collaborates with software engineers, data scientists, and analysts to optimize data quality, availability, and performance.

Qualifications

• Experience designing, developing, and maintaining data pipelines.
• Experience building and optimizing ETL/ELT workflows.
• Experience with relational and/or non-relational databases.
• Experience integrating databases, data lakes, data warehouses, or similar data platforms.
• Experience supporting data for analytics, AI, or ML workloads.
• Experience developing or integrating APIs and service layers.
• Strong understanding of data reliability, accessibility, quality, and operational readiness.
• Experience monitoring, troubleshooting, and optimizing data pipelines.
• Understanding of data modeling and data architecture principles.
• Strong programming and problem-solving skills.
• Ability to work collaboratively with software engineers, data scientists, analysts, and other technical stakeholders.

Responsibilities

• Design, implement, and maintain scalable data pipelines for analytical and AI/ML workloads.
• Build, monitor, troubleshoot, and optimize ETL and ELT data pipelines.
• Develop workflows for ingesting, transforming, processing, and delivering data that supports downstream systems.
• Configure and integrate databases, data lakes, data warehouses, feature stores, and other data platforms.
• Develop and maintain data transformation and feature engineering processes.
• Ensure data reliability, accessibility, quality, and operational readiness.
• Develop APIs and service layers to expose data or model endpoints to downstream applications and systems.
• Collaborate with data scientists, software engineers, analysts, and other technical teams to support data requirements.
• Monitor pipeline performance and resolve data processing or integration issues.
• Implement processes to validate data accuracy, completeness, and consistency.
• Support deployment and operation of data infrastructure throughout the system lifecycle.
• Identify opportunities to improve pipeline scalability, performance, maintainability, and efficiency.
• Document data pipelines, integrations, transformations, and technical processes.

Skills

Data pipelines

ETL/ELT workflows

Databases

Data platforms

AI/ML data usage

APIs/service layers

Data reliability/quality

Pipeline monitoring

Data modeling/architecture

Programming

Cross-functional collaboration

Job description

Black Canyon Consulting seeks a Data Engineer to design, develop, and maintain data infrastructure that powers AI, machine learning, and analytical workloads. You will build scalable pipelines, ETL/ELT workflows, and APIs across databases, data lakes, and warehouses to ensure reliable data for analytical systems.

The role collaborates with software engineers, data scientists, and analysts to optimize data quality, availability, and performance.