Data Engineer: Scalable AI/ML Pipelines

Black Canyon Consulting; Bethesda (AR); On-site; USD 110,000 - 150,000; Full time

Job summary

Black Canyon Consulting is seeking a Data Engineer to design, build, and maintain scalable data pipelines and infrastructure supporting analytics, AI/ML workloads, and feature engineering workflows.

You will work with databases, data lakes, data warehouses, and service layers to ensure reliable, accessible data throughout ingestion to deployment, collaborating with data scientists and software engineers on data requirements.

Qualifications

• Experience designing, developing, and maintaining data pipelines.
• Experience building and optimizing ETL and/or ELT workflows.
• Experience working with relational and/or non-relational databases.
• Experience integrating databases, data lakes, data warehouses, or similar data platforms.
• Experience developing data transformation workflows.
• Experience supporting data used for analytical, AI, or machine learning workloads.
• Experience developing or integrating APIs and service layers.
• Strong understanding of data reliability, accessibility, quality, and operational readiness.
• Experience monitoring, troubleshooting, and optimizing data pipelines.
• Understanding of data modeling and data architecture principles.
• Strong programming and problem-solving skills.
• Ability to work collaboratively with software engineers, data scientists, analysts, and other technical stakeholders.

Responsibilities

• Design, implement, and maintain scalable data pipelines supporting analytical and AI/ML workloads.
• Build, monitor, troubleshoot, and optimize ETL and ELT data pipelines.
• Develop workflows for ingesting, transforming, processing, and delivering data.
• Configure and integrate databases, data lakes, data warehouses, feature stores, and other data platforms.
• Develop and maintain data transformation and feature engineering processes.
• Ensure data reliability, accessibility, quality, and operational readiness.
• Develop APIs and service layers to expose data or model endpoints to downstream applications and systems.
• Collaborate with data scientists, software engineers, analysts, and other technical teams to support data requirements.
• Monitor pipeline performance and resolve data processing or integration issues.
• Implement processes to validate data accuracy, completeness, and consistency.
• Support deployment and operation of data infrastructure throughout the system lifecycle.
• Identify …
• Document data pipelines, integrations, transformations, and technical processes

Skills

Data pipelines

ETL/ELT

Databases

Data platforms

APIs

Data modeling

Programming

Collaboration

Monitoring

Reliability

Distributed systems

Job description

Black Canyon Consulting is seeking a Data Engineer to design, build, and maintain scalable data pipelines and infrastructure supporting analytics, AI/ML workloads, and feature engineering workflows.

You will work with databases, data lakes, data warehouses, and service layers to ensure reliable, accessible data throughout ingestion to deployment, collaborating with data scientists and software engineers on data requirements.