Temporal point processes for forecasting events in higher-order networks

includes bibliographical references and index

PhD; 2023; Computer Science and Automation

Real-world systems consisting of interacting entities can be effectively represented as time-evolving networks or graphs, where the entities are depicted as nodes, and the interactions between them are represented as instantaneous edges. Modeling the evolution of these systems and forecasting interaction events are important for many fields, such as e-commerce, financial markets, neuroscience, etc. This is achieved using the Temporal Point Process (TPP) framework, a stochastic process that models these interactions as discrete events occurring in continuous time. The existing works on interaction forecasting are applicable only to pair-wise edges. However, real-world interactions are much more complex than pair-wise interactions. It involves a group of entities interacting in a complex way rather than just two entities. This leads to the formation of time-evolving higher-order networks. There has not been much research to develop machine learning algorithms for event prediction in these types of networks. This thesis addresses this by providing solutions to the following problems: (i) How can we train models on the events observed in a higher-order network? (ii) Considering the number of possible events grows exponentially when problem setting changes from pair-wise to higher-order, can we forecast the next event in a scalable way? (iii) How to model the influence of interactions on nodes in a higher-order network, and how to improve its scalability? (iv) How can we incorporate relations and group structure within an interaction to an event forecasting model? The first contribution of this thesis is a model for forecasting interactions among a group of entities as instantaneous hyperedge events in a network. In this model, we introduce a TPP on each hyperedge, with the conditional intensity parameterized by a hyperedge link predictor that uses node embeddings. We employ a dynamic node embedding strategy to account for the temporal evolution of entities with each interaction. Here, all the model parameters are learned by a mini-batch based negative log-likelihood calculation with negative sampling for incorporating unseen hyperedges. Furthermore, our model has been extended to accommodate bipartite interactions, where interactions occur between two distinct groups of entities of different types. To achieve this, we introduce a bipartite hyperedge link predictor and use separate node embedding modules for each node type. Secondly, we propose a model to forecast directed higher-order interactions occurring between two distinct groups of entities. Unlike the previous approach that focuses on representation learning from higher-order network events, here we also introduce a strategy to forecast hyperedges in a scalable way. For that, we employ a multi-task framework for forecasting candidate hyperedges. This involves a TPP-based model to predict the time of events on each node, followed by pairwise neighborhood and hyperedge size prediction modules for generating candidate hyperedges. This will reduce the exponential search in forecasting future hyperedges in the previous models. Then a directed hyperedge link predictor is used to identify the true hyperedge from false ones. Further, we devise a dynamic node embedding architecture that processes samples in a batch using memory network and temporal graph attention modules. This improves the scalability of the model when dealing with datasets containing a large number of events. In our final contribution, we extend the existing interaction forecasting approaches based on TPP to accommodate real-world interactions that involve internal group structures of different types. Here, each group is associated with a specific relation type, and to address this complexity, we introduce the concept of multi-relational recursive hyperedge formation events. In this framework, hyperedges can serve as nodes within other hyperedges, creating a hierarchical structure. Further, we also introduce a contrastive learning strategy to learn model parameters without using the intractable likelihood of TPP. In conclusion, this thesis emphasizes and demonstrates the importance of employing higher-order temporal network models to forecast interactions in real-world systems accurately. To achieve this, we created deep neural network based approaches for dynamic node embedding to extract information from historical data and link predictors to model the occurrence of the edge formation event. Then devised different strategies to train these models and forecast events from them.