Fine-Tuning Foundation Models for Temporal Knowledge Graph Reasoning

Foundation models have recently demonstrated strong performance in various knowledge graph reasoning tasks. However, their applicability to temporal knowledge graphs (TKGs), where facts evolve over time, remains underexplored. In this work, we investigate whether a foundation model designed for knowledge graph reasoning can be adapted to temporal reasoning through fine-tuning. Specifically, we extend and fine-tune ULTRA [5] for temporal knowledge graph forecasting tasks. To this end, we adapt the training and evaluation setting of the model, originally designed to perform KG completion tasks, to KG forecasting tasks. Furthermore, we allow ULTRA to incorporate temporal information of facts and queries, in the form of quadruples, via positional encoding of timestamps. Experimental results on standard TKG benchmarks reveal that fine-tuned ULTRA achieves competitive performance with state-of-the-art (SOTA) supervised competitors, particularly on the ICEWS datasets. These datasets emphasize entity-driven prediction over time, where relational patterns are sparse and events such as diplomatic visits or negotiations often occur once without strong temporal regularities. However, on more structurally and temporally rich datasets like YAGO, GDELT, and WIKI, ULTRA falls short of SOTA supervised models, which leverage relational temporal dynamics and evolving patterns more effectively. These findings suggest that while static foundation models can be effectively fine-tuned for certain types of temporal reasoning, they lack the inductive biases necessary to fully capture evolving relational structures. This underscores the development of foundation models explicitly tailored for temporal knowledge graphs as a promising research direction for mining and learning complex patterns from these systems.