US - IT Data Engineer

ol-kit; United States; Remote; USD 120,000 - 150,000; Full time

Job summary

A global agricultural technology firm in the United States is looking for a Senior Software Engineer to drive innovative solutions in data engineering. You will be instrumental in developing distributed analysis capabilities that enhance agricultural solutions through complex data management. Candidates should have significant experience in software development, particularly with the Go programming language and RESTful APIs, and must be adept at using advanced technologies like Apache Kafka and Docker. This role presents an opportunity to make a direct impact on global food production.

Qualifications

• Minimum 7 years of professional software engineering experience.
• At least 2 years of hands-on experience with Go programming language.
• Demonstrated experience implementing stream processing with Apache Kafka.

Responsibilities

• Design and implement distributed analysis capabilities for diverse datasets.
• Build and maintain data-intensive APIs using RESTful architectural principles.
• Collaborate with cross-functional teams to translate scientific requirements.

Skills

Go programming language

RESTful API design and development

Apache Kafka

Unit Testing

Test Driven Development

Containerization technologies

Cloud infrastructure

Problem-solving abilities

Job description

Join a transformative mission to revolutionize agriculture through cutting-edge data engineering and software innovation. This senior-level position offers the unique opportunity to work with terabytes of complex scientific data spanning genome sequencing, crop field trials, manufacturing, and supply chain operations. You will be instrumental in building distributed analysis capabilities that accelerate agricultural solutions development, directly contributing to sustainable food production for a growing global population. This role combines technical excellence with meaningful impact, allowing you to apply advanced software craftsmanship to solve unprecedented challenges in agricultural science. Working alongside top-tier talent, you will have the autonomy to explore innovative technology stacks and shape solutions that transform raw data into actionable insights across the entire crop development pipeline.

Responsibilities

• Serve as a critical senior member of the data engineering team, designing and implementing distributed analysis capabilities for diverse and complex datasets
• Apply deep knowledge of algorithms and data structures to continuously improve software quality and drive innovation
• Build and maintain data-intensive APIs using RESTful architectural principles
• Implement stream processing solutions using Apache Kafka to handle real-time data flows
• Design and maintain containerized application deployments using platforms such as Docker
• Develop and manage cloud-based infrastructure on major cloud providers
• Create and optimize data models for large-scale databases, both relational and NoSQL
• Evaluate and recommend appropriate technology stacks tailored to specific dataset requirements
• Collaborate with cross-functional teams to translate scientific requirements into scalable software solutions
• Present technical work and innovations at industry conferences and professional forums

Skills

• Advanced proficiency in Go programming language
• Expertise in RESTful API design and development
• Strong experience with Apache Kafka for stream processing
• Proficiency in Unit Testing and Test Driven Development methodologies
• Deep understanding of algorithms and data structures
• Expertise in containerization technologies and deployment strategies
• Cloud infrastructure development and maintenance capabilities
• Strong software engineering principles and best practices
• Problem-solving abilities for complex, unique technical challenges

Experience

• Minimum 7 years of professional software engineering experience
• At least 2 years of hands-on experience with Go programming language
• Minimum 2 years building and maintaining data-intensive RESTful APIs
• Demonstrated experience implementing stream processing with Apache Kafka
• Proven track record working with containerized applications using Docker or similar platforms
• Experience building and maintaining cloud infrastructure on AWS, Azure, or Google Cloud Platform
• Experience with large-scale database data modeling in relational or NoSQL environments

Qualifications

• Protocol buffers and gRPC experience preferred
• Google Cloud Platform, Apache Beam, Google Cloud Dataflow, or Kubernetes experience highly desirable
• Background in scientific datasets or quantitative science applications to business problems is a plus
• Bioinformatics experience, particularly in large-scale variant data storage, data mining, variant annotation, and genotype-to-phenotype correlation is advantageous

This is an exceptional opportunity to make a tangible difference in global food security while working at the forefront of agricultural technology innovation. You will have the freedom to project your talent into meaningful projects where the strength of your ideas matters more than hierarchy, and you will be encouraged to share your expertise through conference presentations and technical publications. Join a team that values software craftsmanship, embraces cutting-edge technology, and is committed to solving some of the most important challenges facing our planet's future.