Graph Transformers (2026): Papers, Scalable Models, Benchmarks & Code

What Is a Graph Transformer?

A graph transformer applies transformer-style self-attention to graph data, letting every node attend to many or all other nodes instead of only its immediate neighbours. Because attention alone ignores graph structure, graph transformers add it back through positional and structural encodings, such as Laplacian eigenvectors, node centrality and shortest-path distances in Graphormer, or by combining attention with local message passing as in the GPS recipe. Global attention helps with long-range dependencies and the over-squashing limit of message passing, but its quadratic cost on large graphs has driven sparse and linear variants such as Exphormer, NodeFormer and SGFormer. Graph transformers are also a building block for graph foundation models and for connecting graphs to large language models.

Foundational Models

5 entries

The architectures that established graph transformers.

01A Generalization of Transformer Networks to GraphsAdapts the transformer to graphs with neighbour-based attention and Laplacian positional encodings. 02Do Transformers Really Perform Bad for Graph Representation? (Graphormer)Adds centrality, spatial and edge encodings so a standard transformer can learn graph structure. 03Rethinking Graph Transformers with Spectral Attention (SAN)Uses learned positional encodings from the graph Laplacian spectrum. 04Pure Transformers are Powerful Graph Learners (TokenGT)Treats nodes and edges as tokens in an unmodified transformer. 05Recipe for a General, Powerful, Scalable Graph Transformer (GPS)Combines local message passing with global attention and flexible positional encodings.

Scalable Graph Transformers

3 entries

Ways to cut the quadratic cost of global attention.

06Exphormer: Sparse Transformers for GraphsSparse attention built from expander graphs and virtual nodes. 07NodeFormerScalable transformer for node classification with kernelised, linear-cost attention. 08SGFormerSimplified, single-layer global attention for large-graph representations.

Graph Foundation Models & LLMs

2 entries

Where graph transformers meet pretraining and language models.

09Position: Graph Foundation Models are Already HerePosition paper arguing for graph foundation models and outlining a path to them. 10GraphGPT: Graph Instruction Tuning for Large Language ModelsAligns large language models with graph structure through instruction tuning.

Surveys & Courses

3 entries

Overviews and a course for background.

11Attending to Graph TransformersSurvey of graph transformer architectures, theory and open problems. 12Transformer for Graphs: An Overview from Architecture PerspectiveOverview organising graph transformers by architecture. 13Stanford CS224W: Machine Learning with GraphsStanford course covering GNNs and graph transformers, with lectures and notes.

Benchmarks & Code

7 entries

Benchmarks and reference implementations.

14Long Range Graph BenchmarkBenchmark datasets that test whether models capture long-range dependencies. 15LRGB codeDatasets and baselines for the Long Range Graph Benchmark. 16GraphGPS codeReference implementation of the GPS graph transformer recipe. 17Graphormer codeMicrosoft's implementation of Graphormer. 18Exphormer codeImplementation of Exphormer sparse graph transformers. 19PyTorch Geometric GPSConvDocumentation for the GPS layer in PyTorch Geometric. 20Open Graph BenchmarkBenchmark datasets and leaderboards for machine learning on graphs.

5 entries

Related research directories on this site.

21Graph Neural NetworksHow Graph Neural Networks learn from nodes and edges via message passing, the four core GNN architectures, real applications, graph transformers and relational… 22Graph Attention NetworksGuide to graph attention networks (GATs) for 2026: the original GAT, GATv2, heterogeneous and gated variants, theory, graph transformers, tutorials, reference… 23GraphSAGEGuide to GraphSAGE for 2026: the inductive graph neural network that samples and aggregates neighbourhoods, scaling methods, PinSage, GraphStorm, tutorials… 24Vision TransformersHow Vision Transformers (ViT) work in 2026: patches, embeddings, position and CLS tokens, attention layers and classification, with papers, models and tools. 25GNN LibrariesDirectory of open-source graph neural network libraries for 2026, by ecosystem: PyTorch Geometric, Deep Graph Library, TorchDrug and ptgnn for PyTorch.

Frequently Asked Questions

What is a graph transformer?

A graph transformer applies transformer self-attention to graph data so nodes can attend to many or all other nodes. Positional and structural encodings, or a hybrid with message passing, give the model the graph's structure.

How do graph transformers differ from message-passing GNNs?

Message-passing GNNs update each node from its neighbours over several layers, which limits how far information travels. Graph transformers let nodes attend globally in one layer, which helps with long-range dependencies but costs more computation.

Why do graph transformers need positional encodings?

Self-attention treats its input as a set and ignores edges. Encodings such as Laplacian eigenvectors, shortest-path distances or node centrality inject structure so attention can use it.

How do graph transformers scale to large graphs?

Sparse and linear-attention designs such as Exphormer, NodeFormer and SGFormer reduce the quadratic cost of global attention, and hybrids like GPS combine cheap local message passing with limited global attention.

What are graph foundation models?

Pretrained models intended to transfer across graphs and tasks. Many build on transformer architectures, and recent work also links graph encoders to large language models, as in GraphGPT.