Hongyi ZHANG 89fd3285b7 | 1 year ago | |
---|---|---|
.. | ||
README.md | 1 year ago |
Model | Paper |
---|---|
HGT(WWW 2019) | Heterogeneous Graph Transformer |
SimpleHGN(KDD 2021) | Are we really making much progress? Revisiting, benchmarking,and refining heterogeneous graph neural networks |
HetSANN(AAAI 2020) | An Attention-Based Graph Neural Network for Heterogeneous Structural Learning |
ieHGCN(TKDE 2021) | Interpretable and Efficient Heterogeneous Graph Convolutional Network |
This part, we will give the definition of attention methanism based on GAT and Transformer.
$$
e_{ij} = a(Wh_i, Wh_j)
$$
$$
\alpha_{ij} = softmax_j(e_{ij}) = \frac{\text{exp}(e_{ij})}{\sum_{k\in \mathcal{N}i} \text{exp}(e{ik})}
$$
$$
Attention(Q, K, V) = softmax(\frac{QK^T}{\sqrt{d_k}})V
$$
This part, we will give DGL API we used. As DGL released 0.8.0 version, more API can support heterogeneous graph such as TypedLinear, HeteroLinear. So we will give some details of these APIs.
class TypedLinear(in_size, out_size, num_types, regularizer=None, num_bases=None)
Apply linear transformation according to types.
Parameters:
in_size(int): Input feature size.
out_size(int): Output feature size.
num_types(int): Number of types(node or edge).
regularizer(str, optional): Which weight regularizer to use “basis” or “bdd”, default is None:
num_bases(int, optional): Number of bases. Needed when regularizer is specified. Typically smaller than num_types. Default: None.
forward(x, x_type, sorted_by_type=False)
Parameters:
So this API can be used when we use to_homogeneous to convert a heterogeneous graph to a homogeneous graph.
class HeteroLinear(in_size, out_size, bias=True)
Apply linear transformations on heterogeneous inputs.
Parameters:
forward(feat)
Parameters:
So this API can be used if we want to apply different linear transformations to different types.
class HeteroGraphConv(mods, aggregate='sum')
The heterograph convolution applies sub-modules on their associating relation graphs, which reads the features from source nodes and writes the updated ones to destination nodes. If multiple relations have the same destination node types, their results are aggregated by the specified method. If the relation graph has no edge, the corresponding module will not be called.
Parameters:
forward(g, inputs, mod_args=None, mod_kwargs=None)
Parameters:
So this API can be used when we need to get relation subgraphs and apply nn.Module to each subgraph.
Based on HeteroGraphConv, we divide the attention model into two categories: Direct-Aggregation models and Dual-Aggregation models.
Model | Attention coefficient |
---|---|
HGT | $W_{Q_{\phi{(s)}}}h_s W^{ATT}{\psi{(r)}}(W{K_{\phi{(t)}}}h_t)^T$ |
SimpleHGN | $LeakyReLU(a^T[Wh_s \parallel Wh_t \parallel W_r r_{\psi(<s,t>)}])$ |
HetSANN | $LeakyReLU([W_{\phi(t),\phi(s)} h_s\parallel W_{\phi(t),\phi(s)} h_t]a_r)$ |
These models only have one aggregation process and do not distinguish between types of edges when aggregating, so they are not suitable for HeteroGraphConv.
Model | Attention coefficient |
---|---|
ieHGCN | $ELU(a^T_{\phi(s)}[W_{Q_{\phi(s)}}h_s\parallel W_{K_{\phi(t)}}h_t])$ |
This model has two aggregation process and distinguish between types of edges when aggregating, so this is suitable for HeteroGraphConv.
We first implement the convolution layer of the model SimpleHGN, and HetSANN. The convolutional layer of HGT we use is hgtconv. The __init__ parameters can be different as the models need different parameters.
The parameters of the forward part are the same: g
is the homogeneous graph, h
is the features, ntype
denotes the type of each node, etype
denotes the type of each edge, presorted
tells if the ntype
or etype
is presorted to use TypedLinear in dgl.nn conveniently. If we use dgl.to_homogeneous to get the features, the features are presorted.
Then, we use the convolution layers to implement corresponding models. We need dgl.to_homogeneous to get a homogeneous graph as when we use edge_softmax, we put all the edges together to calculate the attention coefficient instead of distinguishing the type of edges.
After passing the convolution layers, we need to convert the output features to a feature dictionary in a heterogeneous graph. We designed a tool in openhgnn.utils.utils.py named to_hetero_feat. This is because we do not have a better solution to get a feature dictionay using dgl. We can only use dgl.to_heterogeneous, but it has many additional operations to make the programs slowly. After we get the feature dictionary, the model is complete.
Clone the Openhgnn-DGL
# For node classification task
# You may select model HGT, SimpleHGN, HetSANN
python main.py -m HGT -t node_classification -d imdb4MAGNN -g 0 --use_best_config
If you do not have gpu, set -gpu -1.
Evaluation metric: Micro/Macro-F1
HGBn-ACM | acm4GTN | imdb4MAGNN | dblp4MAGNN | |||||
Model | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 |
HGT | 88.95 | 89.18 | 90.21 | 90.24 | 49.37 | 49.18 | 87.23 | 86.46 |
SimpleHGN | 92.27 | 92.36 | 89.27 | 89.28 | 52.25 | 48.78 | 87.72 | 87.08 |
HetSANN | 88.4 | 88.7 | 92.24 | 92.31 | 52.88 | 47.44 | 89.54 | 90.24 |
ie-HGCN | 91.71 | 91.99 | 92.47 | 92.56 | 55.03 | 52.18 | 88.36 | 87.37 |
You can modify the parameters [HGT], [SimpleHGN], [HetSANN], [ieHGCN] in openhgnn/config.ini.
Yaoqi Liu[GAMMA LAB]
Submit an issue or email to YaoqiLiu@bupt.edu.cn.
OpenHGNN是由北邮GAMMA Lab开发的基于PyTorch和DGL的开源异质图神经网络工具包。
Python Markdown Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》