The Challenge of Missing Links

Ever since I can remember, I've been captivated by the intricate workings of nature—the "hows" and "whys" behind the connections between organisms. This curiosity guided me into the fascinating realm of network science, a framework ecologists now frequently use to unravel complex relationships within ecosystems. One significant obstacle to this framework is the uncertainty surrounding missing links. Mapping all the interactions in a network is an immense task due to the sheer number of species and often hidden interactions. A fully detailed ecological network would offer profound insights, but observing each interaction directly is nearly impossible. So the answer to the question - why isn't this flower pollinated by this bee? - is not so straightforward. Is it due to a mismatch between these two species, or did we just miss this interaction in our field sampling?

To simplify, imagine predicting movies someone might enjoy on Netflix. Netflix doesn't know every movie you'd like—these are the missing links in your viewing network. Similarly, ecologists don't know all species interactions—these are the missing links in ecological networks. The challenge they have in common is accurately predicting these hidden connections.

Together with my PhD student, Barry Biton, and Professor Rami Puzis from the Department of Software and Information Systems Engineering, we explored this challenge. We found that although researchers have been applying link prediction methods to ecological networks, two major hurdles still limit our ability to predict interactions. First, predictions usually rely on data within a single network, leaving limited information for model training. Second, models often lack broader applicability beyond their original datasets. We sought a novel approach to overcome these constraints and enable broader predictions.

Inductive Prediction: Generalizing Ecological Patterns

Our innovation lies in our method's inductive nature. Traditional "transductive" methods train and test within the same network, limiting generalizability. Our inductive approach learns broader features applicable across multiple ecological networks and is trained on multiple networks. Returning to the Netflix analogy, traditional link prediction might use a simple rule: "People who watched Movie A also watched Movie B." In ecology, the equivalent would be assuming species that are interacting with similar partners might also interact. However, this approach falls short if interactions are sparse or if we are trying to predict links in a network the model has not been trained on.

So how did we train an inductive model? Ecological networks have similar properties regardless of the interaction type. Indeed, distinguishing between network types (e.g., parasitism vs. pollination networks) based solely on structural properties is surprisingly challenging. We leveraged this fact to train our model by pooling data across 538 networks of four types: plant–seed disperser, plant–pollinator, host–parasite and plant–herbivore, instead of a single network.

In addition, we trained a general, inductive model. Imagine Netflix learning patterns across millions of users to predict your preferences—even if you're new or the movie just arrived. Similarly, our model learns the "grammar" of ecological networks, predicting interactions in entirely new communities (assuming you have at least some knowledge of the new community).

Unexpectedly, we found that different ecological networks varied significantly in predictability—host-parasite networks were most predictable, while plant-pollinator networks were the most challenging. Interestingly, the distinction between mutualistic (both species benefit) and antagonistic interactions (one species benefits at the other's expense) wasn't the main factor influencing predictability.

Transferring knowledge between communities

A thrilling aspect of our approach is its cross-community inference ability. Can detailed interaction knowledge from one community type (e.g., host-parasite networks) help predict interactions in another (like plant-seed dispersers)? Surprisingly, yes! 

It would be as if Netflix insights help Disney+ recommend shows without knowing your viewing habits directly. This "transfer learning" leverages common patterns across different contexts.

It turns out that training our model on one kind of ecological network sometimes improved predictions in entirely different types. This suggests fundamental similarities in network structures across different ecological interactions—akin to discovering Netflix viewing patterns could enhance Spotify music recommendations. 

Another unexpected finding was that combining data from all network types didn’t always improve predictions. Removing plant-pollinator networks actually enhanced the model's overall performance, likely because extreme sparsity affected the model training.

Looking Ahead

Our work offers ecologists a powerful tool for understanding life's intricate web. By harnessing inductive link prediction, we shift from merely describing networks to proactively predicting unseen ecological interactions.

To help empirical ecologists apply our method, we've developed an accessible online platform for link prediction: https://ecomplab-eco-ilp.hf.space. Ecologists simply upload their data and immediately receive predictions of likely missing interactions, enabling more targeted research efforts. In the spirit of open science, we've made our predictive model and associated code open source, inviting the research community to build upon and refine our methods. We are incredibly excited to see where this integration of modeling and empirical discovery takes us, deepening our understanding of nature’s complexities.

Our paper "Inductive link prediction facilitates the discovery of missing links and enables cross-community inference in ecological networks" is now available in Nature Ecology & Evolution.