Vylor is seeking a full-time Data Engineer II based in Des Moines, IA (onsite). The role focuses on designing and delivering scalable lakehouse and analytics data solutions for Seeds and Biotech domains, alongside support for an enterprise data platform.
What You’ll Do
- Design, develop, and maintain lakehouse data pipelines, APIs, and data products for seed models, biotech research assets, enterprise applications, dashboards, and analytics.
- Build scalable ETL/ELT pipelines using Python, SQL, dbt, Trino, and shell scripting to process and integrate large-scale structured and unstructured agricultural data from enterprise systems, laboratory platforms, geographic datasets, drones, satellites, and IoT devices.
- Create and maintain Bronze, Silver, and Gold Layer data assets by integrating sources such as Exasol, ClickHouse, and MSSQL Server, applying transformation, validation, reconciliation, and quality-control processes.
- Design lakehouse datasets and transformation pipelines for seed, biotech, pathology, and disease analytics, including integration of laboratory, geographic, and historical research data.
- Develop and optimize incremental and full-refresh dbt models and complex Trino SQL queries to improve performance, scalability, and maintainability of Seeds and Biotech data products.
- Modernize legacy data applications by documenting business rules and source-to-target mappings, migrating ETL logic from legacy and Exasol-based environments to Trino and lakehouse architectures, and deploying curated datasets to ClickHouse and Gold Layer data products.
- Develop and enhance PRISM and FRIP data integrations by implementing Trino connectors and end-to-end comparison pipelines with reusable validation and reconciliation logic across PRISM, FRIP, lakehouse, Exasol, and downstream application datasets.
- Build reusable frameworks for logging, error handling, scheduling, automated testing, monitoring, and data-quality validation; troubleshoot integration issues; and create analytics-ready datasets for statistical analysis, predictive modeling, regression, classification, clustering, dashboards, and business-intelligence reporting.
Required Qualifications
- Master’s degree (or equivalent) in Computer Science, Statistics, or a related field, plus 2 years of related experience.
- 24 months of experience designing and developing scalable ETL/ELT pipelines using Python, SQL, and shell scripting to extract, transform, validate, reconcile, and integrate large-volume structured, semi-structured, and unstructured data used by enterprise R&D applications, analytical models, dashboards, and reporting systems.
- 24 months of experience developing ETL design mappings, source-to-target transformation rules, data reconciliation processes, data-quality controls, and complex SQL transformations to ensure accuracy and consistency across platforms.
- 24 months of experience developing and optimizing complex Trino SQL queries and integrations among Lakehouse, Exasol, ClickHouse, MSSQL Server, and other database platforms to deliver analytics-ready curated datasets for R&D models, scientific analysis, and enterprise applications such as phenomic pillar, dashboards, and reporting.
- 24 months of experience building reusable Python- and Linux shell-based pipeline reliability and observability frameworks for job scheduling, structured logging, exception handling, automated testing, monitoring, data-quality validation, cross-platform reconciliation, and application integration; using version-control, containerization, and automated-deployment tools to deliver production data products.
- 24 months of experience analyzing and modernizing legacy data applications by documenting business rules and data mappings, migrating ETL logic to cloud-based or lakehouse architecture, deploying curated data products, and using Git and Docker for source control, automated deployment, and containerization.
Technologies
- Lakehouse, Python, SQL, dbt, Trino, shell scripting
- Exasol, ClickHouse, MSSQL Server
- Kubernetes, Docker, Git, Linux
Benefits
- Health benefits for you and your family on your first day of employment
- Four weeks of paid time off and two weeks of well-being pay per year, plus paid holidays
- Excellent parental leave which includes a minimum of 16 weeks for mother and father
- Future planning with a competitive retirement savings plan and tuition reimbursement program
- Numerous development opportunities to build skills
- Opportunity to contribute to making the world a better place
- Learn more about the total rewards package: Vylor Benefits
Telecommute Benefit
The position includes a telecommute benefit within commuting distance to the Johnston, IA Pioneer Hi-Bred/Vylor office, as directed.