Data Engineer: Build Scalable Pipelines & Data Platforms

Devi Technologies; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Devi Technologies is seeking a Data Engineer to design, build, and maintain scalable data systems and pipelines across multiple data sources. You will translate business and technical requirements into robust architectures, ensure data quality, and automate ingestion and transformation processes.

The role involves collaborating with analysts and developers to improve data platforms and performance. You will implement data validation, maintain documentation, and contribute to the adoption of new

Responsibilities

• Design, develop and maintain scalable data systems and data engineering solutions to meet business and technical requirements.
• Build and maintain data pipelines to collect, process, transform and integrate data from multiple internal and external sources.
• Analyse business and technical requirements and translate them into appropriate data architecture and engineering solutions.
• Design and implement processes for extracting, transforming and loading data into databases, data warehouses and other data platforms.
• Develop and maintain data models, database structures and data integration processes to support organisational requirements.
• Integrate structured and unstructured data from multiple systems and ensure data is available in appropriate formats for analysis and reporting.
• Develop automated processes for data ingestion, transformation, validation and processing.
• Monitor data pipelines and data platforms to identify performance, reliability and data-quality issues.
• Investigate and resolve technical problems relating to data processing, integration, storage and availability.
• Implement appropriate data validation and quality controls to maintain the accuracy, consistency and reliability of organisational data.
• Optimise data pipelines, database queries and processing workflows to improve system performance and efficiency.
• Work with business analysts, data analysts, software developers and other technical professionals to understand data requirements and deliver appropriate solutions.
• Assess existing data systems and recommend improvements to data architecture, integration methods and processing capabilities.
• Support the implementation and integration of new data technologies and platforms in accordance with organisational requirements.
• Develop and maintain technical documentation covering data architecture, data models, data pipelines, system configurations and integration processes.
• Apply appropriate data security, access-control and data-management requirements when designing and maintaining data solutions.
• Test data engineering solutions and validate data outputs to ensure they meet defined technical and business requirements.

Job description

Devi Technologies is seeking a Data Engineer to design, build, and maintain scalable data systems and pipelines across multiple data sources. You will translate business and technical requirements into robust architectures, ensure data quality, and automate ingestion and transformation processes.

The role involves collaborating with analysts and developers to improve data platforms and performance. You will implement data validation, maintain documentation, and contribute to the adoption of new