Sr. Data Engineer - AI

Dairy Farmers of America, Inc.; Kansas City (KS); On-site; USD 140,000 - 180,000; Full time

Job summary

Dairy Farmers of America, Inc. is seeking a senior data engineer to design, build, and support secure data pipelines that feed AI/ML models.

You will also provide practical front-end support using TypeScript, JavaScript, Vue, and Quasar to deliver intuitive AI-enabled applications. The role requires 8+ years of data engineering experience, strong cloud and orchestration skills, and the ability to lead cross-team initiatives, while upholding governance and security standards.

Qualifications

• Undergraduate degree in Computer Science, Data Engineering, Information Systems, or related field.
• 8+ years data engineering or related experience, including AI/ML pipelines.
• Proficiency with Python, SQL, PySpark, Snowflake, Databricks, and Apache Airflow.
• Experience developing SPA using TypeScript/JavaScript with Vue and Quasar.
• Strong knowledge of relational and NoSQL databases, data warehouses/lakes, Azure Data Factory, and Kafka.
• Experience with at least one major cloud data ecosystem, preferably Azure, and IaC/automation.
• Certifications in cloud data engineering or data privacy/security and MLOps experience preferred.

Responsibilities

• Design, build, and optimize scalable ETL/ELT pipelines for AI/ML training and inference.
• Ensure data is accurate, current, reliable, and secure via cleansing, validation, and monitoring.
• Troubleshoot integration and performance issues; apply caching and distributed processing.
• Own AI data infrastructure components with monitoring, logging, data lineage, and quality metrics.
• Apply enterprise architecture, AI governance, privacy, security, CI/CD standards and communicate risks.
• Partner with software engineers, data architects, domain experts, product and platform teams.
• Develop and support front-end components using TS/JS, Vue, and Quasar for AI-enabled apps.
• Independently deliver work, influence cross-team decisions, promote data engineering best practices.

Skills

8+ years data engineering

Python

SQL

PySpark

Snowflake

Databricks

Apache Airflow

TypeScript/JavaScript SPA

Vue & Quasar

Azure cloud

Education

Undergraduate degree in Computer Science, Data Engineering, Information Systems, or related field

Tools

Azure Data Factory

Kafka

Snowflake

Databricks

Apache Airflow

Job description

Serve in a senior-level technical role designing, building, and supporting secure, scalable data pipelines that connect enterprise data sources to AI/ML models. Ensure data is accurate, current, compliant, and aligned with enterprise AI governance standards while independently solving complex integration challenges and advancing data engineering best practices. Provide practical front-end development support using TypeScript, JavaScript, Vue, and Quasar to help deliver intuitive AI-enabled applications.

Job Duties and Responsibilities:

• Design, build, and optimize scalable ETL/ELT pipelines that ingest, transform, and integrate structured and unstructured enterprise data for AI/ML model training and inference
• Ensure data is accurate, current, reliable, and secure through cleansing, normalization, validation, automation, continuous updates, monitoring, and error handling
• Troubleshoot complex integration and performance issues, remove bottlenecks, and apply appropriate techniques such as caching, distributed processing, and specialized ETL patterns
• Own key AI data infrastructure components and ensure they are scalable, maintainable, and supported by effective monitoring, logging, documentation, data lineage, and quality metrics
• Apply enterprise architecture, AI governance, privacy, security, coding, and CI/CD standards to data solutions and communicate risks or limitations to leadership
• Partner with Software engineers, data architects, domain experts, product teams, and platform teams to align data solutions with model requirements, business goals, and enterprise standards
• Develop and support front-end components for AI-enabled applications using TypeScript, JavaScript, Vue, and Quasar to deliver intuitive, maintainable user experiences
• Independently deliver assigned work, influence cross-team decisions, promote data engineering best practices, and evaluate technologies that improve AI data delivery
• The requirements herein describe the general nature and level of work performed by the employee but are not a complete list of responsibilities, duties, and skills required. Other duties may be assigned as needed

Minimum Requirements:

Education and Experience

• Undergraduate degree in Computer Science, Data Engineering, Information Systems, or related field
• 8+ years data engineering, software development, or related experience, including work on data pipelines for AI/ML systems
• Proficiency with data engineering languages, platforms, and orchestration tools such as Python, SQL, PySpark, Snowflake, Databricks, and Apache Airflow
• Experience developing or supporting modern single-page applications using TypeScript or JavaScript, preferably with Vue and Quasar, including component-based design and API integration
• Strong knowledge of relational and NoSQL databases, data warehouses or data lakes, integration platforms such as Azure Data Factory, and streaming technologies such as Kafka
• Experience with at least one major cloud data ecosystem, preferably Azure, and familiarity with infrastructure-as-code and automated deployment practices
• Proven ability to design and implement ETL/ELT pipelines, manage databases or data lakes, and integrate large-scale data systems
• Certifications in cloud data engineering or in data privacy/security and experience with MLOps or AI/ML lifecycle management preferred

Knowledge, Skills and Abilities

• Must be willing to travel 15-25% (1-2 times per quarter)
• Deep knowledge of data engineering and AI/ML pipeline practices, including the ability to design and optimize scalable, reliable, and efficient data solutions
• Strong understanding of data quality, governance, privacy, security, and regulatory requirements
• Able to independently analyze and troubleshoot complex data and integration issues and apply creative, data-driven solutions when standard approaches are insufficient
• Able to align technical solutions with business priorities, including speed, cost, risk, reliability, and desired outcomes, and exercise sound judgment in situations with limited precedent
• Able to communicate complex technical concepts clearly, collaborate across teams, and build stakeholder trust through transparency and reliable delivery
• Able to lead technical initiatives, document approaches clearly, influence peers, and promote adoption of standards and best practices
• Practical ability to develop and support accessible, responsive single-page applications using TypeScript or JavaScript, preferably Vue and Quasar, with reusable components and API integration
• Must be able to read, write, and speak English

An Equal Opportunity Employer including Disabled/Veterans