Remote Data Engineer for AI-Driven Platform

Irth Solutions; United States; On-site; USD 110,000 - 160,000; Full time

Job summary

Irth Solutions is seeking a Data Engineer for the Insights (AI/ML) team. You will build ingestion and transformation pipelines across GIS, inline inspection data, and enterprise systems, enabling model-ready data for our AI platform.

You will implement Delta Lake storage, Unity Catalog, and data quality, work with data scientists, and help operationalize pipelines. This mid-level role offers remote work in the US and opportunities to deepen Databricks, Spark, and lakehouse engineering skills.

Qualifications

• 3-5 years of data engineering, ETL, or cloud data platform experience.
• Hands-on with Databricks, Spark, PySpark or similar.
• Strong SQL and structured data transformation experience.
• Experience with at least one major cloud platform; Azure preferred.
• Familiarity with data modeling, data quality practices, schema evolution, and pipeline troubleshooting.
• Experience with workflow orchestration and scheduling frameworks.
• Understanding of data-security practices and access control.

Responsibilities

• Pipeline Development: build and maintain ingestion pipelines for GIS, inline inspection data, SCADA, and enterprise systems; implement batch and streaming ingestion using Databricks Workflows, Spark, PySpark, SQL; apply medallion architecture for transformation and enrichment.
• AI-Assisted Ingestion: automate data normalization and quality repair workflows; productionize document-extraction pipelines with data scientists; build human-in-the-loop review processes.
• Platform & Storage: configure Delta Lake tables, partitioning, and optimization; use Unity Catalog; maintain connectors to customer systems; support geospatial data processing.
• Governance, Quality & Compliance Enablement: implement data-quality tests, profiling, and drift monitoring; enforce access-control policies; capture lineage for regulatory traceability.
• Orchestration, Automation & Operational Support: build and monitor data workflows; contribute to CI/CD for pipelines; troubleshoot incidents and optimize performance.
• Collaboration & Documentation: work with Data Architect to translate designs; participate in reviews; document pipelines, data dictionaries, schedules, and runbooks.

Skills

SQL

Data modeling

Data quality

Pipeline troubleshooting

Security/compliance

CI/CD for pipelines

Cross-functional collaboration

Tools

Databricks

Spark

PySpark

Azure

Unity Catalog

Delta Lake

Job description

Irth Solutions is seeking a Data Engineer for the Insights (AI/ML) team. You will build ingestion and transformation pipelines across GIS, inline inspection data, and enterprise systems, enabling model-ready data for our AI platform.

You will implement Delta Lake storage, Unity Catalog, and data quality, work with data scientists, and help operationalize pipelines. This mid-level role offers remote work in the US and opportunities to deepen Databricks, Spark, and lakehouse engineering skills.