Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solve issues #229 , #230 , #231 and #233 #263

Open
wants to merge 13 commits into
base: v2.0
Choose a base branch
from
Open

Solve issues #229 , #230 , #231 and #233 #263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
5b643af
Update WebQuestionsSP.md
junbohuang Jun 12, 2024
c73bff6
Update ComplexWebQuestions.md
junbohuang Jun 12, 2024
2f3fbf9
Update WebQuestionsSP.md
junbohuang Jun 12, 2024
dd3bf18
Update QALD-7.md
junbohuang Jun 12, 2024
49033b3
Update CronQuestions.md
junbohuang Jun 12, 2024
8aa9a5e
Update TimeQuestions - Explicit.md
junbohuang Jun 12, 2024
4aea2db
Update TimeQuestions - Explicit.md
junbohuang Jun 12, 2024
aae8f87
Update TimeQuestions - Implicit.md
junbohuang Jun 12, 2024
073b2f8
Update TimeQuestions - Ordinal.md
junbohuang Jun 12, 2024
ce60cd4
Update TimeQuestions - Temporal Answer.md
junbohuang Jun 12, 2024
630c68e
Update TimeQuestions - Overall.md
junbohuang Jun 12, 2024
998ada2
#229
junbohuang Jun 13, 2024
fbdcaaf
:bug: added to systems updated and added dataset to frontend
junbohuang Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions dbpedia/QALD-7.md
View file
Open in desktop
junbohuang marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,4 @@
| Qanary(TM+DP+QB) | 2021 | - | - | 6 | - | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| Entity Type Tags Modified |2022| - | - | - | 76.69 | EN | [Lin and Lu](https://journals.riverpublishers.com/index.php/JWE/article/view/11463/14671) |
| SPARQL Generator |2022| - | - | - | 60.74 | EN | [Lin and Lu](https://journals.riverpublishers.com/index.php/JWE/article/view/11463/14671) |
| KGQA^{PT} | 2024 | 79.4 | 28.5 | 41.9 | - | PT | [Miranda et al.](https://aclanthology.org/2024.propor-1.6.pdf) |
1 change: 1 addition & 0 deletions freebase/ComplexWebQuestions.md
View file
Open in desktop
junbohuang marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
| TransferNet | 2022 | - | 48.6 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| PullNet | 2022 | - | 45.9 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| MRP-QA-marginal_prob | 2022 | 49.9 | - | - | EN | [Wang et al.](https://aclanthology.org/2022.naacl-main.294.pdf) |
| QRGAT | 2024 | 49.7 | 52.4 | - | EN | [Zhang et al.](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10472080) |
| ProgramTransfer-f | 2022 | 45.9 | 45.2 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| KBIGER | 2022 | 45.5 | 50.2 | - | EN | [Du et al.](https://arxiv.org/pdf/2209.03005.pdf) |
| TERP | 2022 | - | 49.2 | - | EN | [Qiao et al.](https://aclanthology.org/2022.coling-1.156.pdf) |
Expand Down
3 changes: 2 additions & 1 deletion freebase/WebQuestionsSP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
---

| Model / System | Year | F1 | Hits@1 | Accuracy | Language | Reported by |
| :---------------------------------: | :--: | :--------: | :--------: | :------: | :------: | :-----------------------------------------------------------------------------------: |
|:-----------------------------------:| :--: | :--------: | :--------: | :------: | :------: | :-----------------------------------------------------------------------------------: |
| chatGPT | 2023 | - | - | 83.70 | EN | [Tan et al.](https://arxiv.org/pdf/2303.07992.pdf) |
| TIARA | 2022 | 78.9 | 75.2 | - | EN | [Shu et. al.](https://aclanthology.org/2022.emnlp-main.555.pdf) |
| DECAF (DPR + FiD-3B) | 2022 | 78.8 | 82.1 | - | EN | [Yu et al.](https://arxiv.org/pdf/2210.00063.pdf) |
Expand Down Expand Up @@ -68,6 +68,7 @@
| SQALER | 2022 | - | 70.6 | - | EN | [Costas Mavromatis and George Karypis](https://arxiv.org/pdf/2210.13650.pdf) |
| STAGG | 2015 | 69.00 | - | - | EN | [Ma L et al.](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9747229) |
| UHop | 2019 | 68.5 | - | - | EN | [Ma L et al.](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9747229) |
| QRGAT | 2024 | 68.5 | 74.4 | - | EN | [Zhang et al.](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10472080) |
| KBIGER | 2022 | 68.4 | 75.3 | - | EN | [Du et al.](https://arxiv.org/pdf/2209.03005.pdf) |
| NSM | 2022 | - | 69.0 | - | EN | [Cao et al.](https://aclanthology.org/2022.acl-long.559.pdf) |
| GraftNet-EF+LF | 2018 | - | 68.7 | - | EN | [Sun et al.](https://aclanthology.org/D18-1455.pdf) |
Expand Down
1 change: 1 addition & 0 deletions frontend/static/entries.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,5 @@
/datasets/other/WC2014QA - 2 Hop
/datasets/other/WC2014QA - Total
/datasets/other/Spider4SPARQL
/datasets/other/STQAD
/systems/
16 changes: 16 additions & 0 deletions other/STQAD.md
View file
Open in desktop
junbohuang marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: STQAD
datasetUrl: NA
---

| Model / System | Year | Hits@1 | Hits@3 |Hits@10 | Language | Reported by |
|:--------------------------:|:----:|:------:|:------:|:------:|:--------:|:----------------------------------------------------------------------------:|
| BERT | 2024 | 39.10 | 53.70 |66.87 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| RoBERTa | 2024 |32.37| 46.77 | 60.83 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| EmbedKGQA | 2024 |38.00 | 53.03 |56.00 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| CronKGQA | 2024 |34.60| 49.33 |62.17 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| TempoQR | 2024 | 54.97 | 66.03 |76.23 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| SubGTR | 2024 | 58.23 | 68.73 |78.50 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |
| STCQA | 2024 |61.63 |76.67 |84.17 | EN | [Dai et al.](https://arxiv.org/pdf/2402.11542) |


13 changes: 13 additions & 0 deletions other/^STQAD.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# STQAD

**STQAD**<sup>[[1]](#myfootnote1)</sup> is a dataset comprising 10,000 natural language questions for spatio-temporal
knowledge graph question answering (STKGQA), utilizing YAGO2<sup>[[2]](#myfootnote2)</sup> as the underlying STKG.
STQAD consists of 10,000 questions specifically designed for the STKGQA task. This is the first comprehensive STKGQA
dataset that encompasses spatial and temporal constraints in each question.

## References
<a name="myfootnote1">[1]</a> Xinbang Dai, Huiying Li, Guilin Qi:
[Question Answering Over Spatio-Temporal Knowledge Graph](https://arxiv.org/pdf/2402.11542). CoRR abs/2402.11542 (2024)

<a name="myfootnote2">[2]</a> Johannes Hoffart, Fabian M. Suchanek, Klaus Berberich, Gerhard Weikum:
[YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia](https://resources.mpi-inf.mpg.de/yago-naga/yago/publications/aij.pdf). Artif. Intell. 194: 28-61 (2013)
286 changes: 145 additions & 141 deletions systems_updated.md
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions wikidata/CronQuestions.md
View file
Open in desktop
junbohuang marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

| Model / System | Year | Hits@1 | Hits@10 | Reported by |
| :------------: | :--: | :----: | :-----: | :---------------------------------------------------------------------------: |
| QC-MHM | 2024 | 0.971 | 0.992 | [Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| TempoQR-Hard | 2021 | 0.918 | 0.978 | [Mavromatis et. al.](https://arxiv.org/pdf/2112.05785.pdf) |
| TempoQR-Soft | 2021 | 0.799 | 0.957 | [Mavromatis et. al.](https://arxiv.org/pdf/2112.05785.pdf) |
| TMA | 2021 | 0.784 | 0.943 | [Liu et. al.](https://arxiv.org/pdf/2302.12529.pdf) |
Expand Down
3 changes: 2 additions & 1 deletion wikidata/TimeQuestions - Explicit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
|:--------------:|:----:|:-----------:|:------:|:-------:|:-------:|:------------------------------------------------------:|
| TwiRGCN (average) | 2022 | - | - | - | 0.602 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| EXAQT | 2022 | - | - | - | 0.568 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| QC-MHM | 2024 | - | - | - | 0.533 |[Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| CronKGQA | 2022 | - | - | - | 0.466 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| TempoQR | 2022 | - | - | - | 0.465 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| Graft-Net | 2022 | - | - | - | 0.445 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
Expand All @@ -17,4 +18,4 @@
| EXAQT | 2021 | 0.568 | 0.594 | 0.636 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| Graft-Net | 2021 | 0.445 | 0.478 | 0.531 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| UNIQORN | 2021 | 0.318 | 0.406 | 0.536 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.105 | 0.136 | 0.186 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.105 | 0.136 | 0.186 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
3 changes: 2 additions & 1 deletion wikidata/TimeQuestions - Implicit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
|:--------------:|:----:|:-----------:|:------:|:-------:|:-------:|:------------------------------------------------------:|
| TwiRGCN (interval) | 2022 | - | - | - | 0.603 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| EXAQT | 2022 | - | - | - | 0.512 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| QC-MHM | 2024 | - | - | - | 0.508 |[Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| CronKGQA | 2022 | - | - | - | 0.445 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| Graft-Net | 2022 | - | - | - | 0.428 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| TempoQR | 2022 | - | - | - | 0.36 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
Expand All @@ -16,4 +17,4 @@
| EXAQT | 2021 | 0.508 | 0.567 | 0.633 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| Graft-Net | 2021 | 0.428 | 0.465 | 0.525 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| UNIQORN | 2021 | 0.316 | 0.415 | 0.545 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.081 | 0.123 | 0.192 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.081 | 0.123 | 0.192 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
3 changes: 2 additions & 1 deletion wikidata/TimeQuestions - Ordinal.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
|:--------------:|:----:|:-----------:|:------:|:-------:|:-------:|:------------------------------------------------------:|
| TwiRGCN (average) | 2022 | - | - | - | 0.518 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| EXAQT | 2022 | - | - | - | 0.42 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| QC-MHM | 2024 | - | - | - | 0.401 |[Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| CronKGQA | 2022 | - | - | - | 0.369 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| TempoQR | 2022 | - | - | - |0.349 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| Graft-Net | 2022 | - | - | - | 0.322 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
Expand All @@ -16,4 +17,4 @@
| EXAQT | 2021 | 0.420 | 0.432 | 0.508 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| Graft-Net | 2021 | 0.322 | 0.313 | 0.371 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| UNIQORN | 2021 | 0.202 | 0.236 | 0.356 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.029 | 0.049 | 0.083 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.029 | 0.049 | 0.083 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
3 changes: 2 additions & 1 deletion wikidata/TimeQuestions - Overall.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
|:--------------:|:----:|:-----------:|:------:|:-------:|:-------:|:------------------------------------------------------:|
| TwiRGCN | 2022 | - | - | - | 0.605 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| EXAQT | 2022 | - | - | - | 0.572 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| QC-MHM | 2024 | - | - | - | 0.531 |[Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| CronKGQA | 2022 | - | - | - | 0.462 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| Graft-Net | 2022 | - | - | - | 0.452 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| TempoQR | 2022 | - | - | - | 0.416 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
Expand All @@ -16,4 +17,4 @@
| EXAQT | 2021 | 0.565 | 0.599 | 0.664 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| Graft-Net | 2021 | 0.452 | 0.485 | 0.554 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| UNIQORN | 2021 | 0.331 | 0.409 | 0.538 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.105 | 0.136 | 0.186 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.105 | 0.136 | 0.186 | - | [Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
3 changes: 2 additions & 1 deletion wikidata/TimeQuestions - Temporal Answer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
|:--------------:|:----:|:-----------:|:------:|:-------:|:-------:|:------------------------------------------------------:|
| TwiRGCN (interval) | 2022 | - | - | - | 0.646 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| EXAQT | 2022 | - | - | - | 0.642 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| QC-MHM | 2024 | - | - | - | 0.607 |[Xue et. al.](https://arxiv.org/html/2402.13188v1#Sx5) |
| Graft-Net | 2022 | - | - | - | 0.515 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| CronKGQA | 2022 | - | - | - | 0.511 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
| TempoQR | 2022 | - | - | - | 0.4 | [Sharma et. al.](https://arxiv.org/pdf/2210.06281.pdf) |
Expand All @@ -16,4 +17,4 @@
| EXAQT | 2021 | 0.623 | 0.672 | 0.756 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| Graft-Net | 2021 | 0.515 | 0.568 | 0.660 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| UNIQORN | 2021 | 0.392 | 0.472 | 0.597 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.234 | 0.277 | 0.349 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |
| PullNet | 2021 | 0.234 | 0.277 | 0.349 | - |[Jia et. al.](https://dl.acm.org/doi/abs/10.1145/3459637.3482416) |