-
Notifications
You must be signed in to change notification settings - Fork 0
/
evaluation.txt
64 lines (49 loc) · 2.62 KB
/
evaluation.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
LSTM - hidden (100) - margin (0.2) - params (120800)
dev - MAP: 56.6011305954, MRR: 70.585013844, P@1: 58.7301587302, P@5: 43.9153439153
test - MAP: 56.9216908178, MRR: 70.6946045528, P@1: 56.9892473118, P@5: 44.0860215054
LSTM - hidden (200) - margin (0.2) - params (321600)
dev - MAP: 57.4908149465, MRR: 71.6580621716, P@1: 59.7883597884, P@5: 45.5026455026
test - MAP: 57.2535903907, MRR: 69.461805533, P@1: 54.3010752688, P@5: 43.6559139785
LSTM - hidden (235) - margin (0.2) - params (410780)
dev - MAP: 57.4421332551, MRR: 72.459226492, P@1: 61.3756613757, P@5: 45.0793650794
test - MAP: 56.3709411235, MRR: 68.4387043107, P@1: 53.7634408602, P@5: 42.9032258065
LSTM - hidden (240) - margin (0.2) - params (424320)
dev - MAP: 57.7400310459, MRR: 71.1112172892, P@1: 58.2010582011, P@5: 46.1375661376
test - MAP: 56.6448349887, MRR: 70.0646916044, P@1: 56.4516129032, P@5: 43.4408602151
LSTM - hidden (200) - margin (0.25) - params (321600)
dev - MAP: 58.5454827251, MRR: 72.1531449064, P@1: 59.7883597884, P@5: 46.2433862434
test - MAP: 55.7737256626, MRR: 67.6543577626, P@1: 53.7634408602, P@5: 43.8709677419
------
LSTM - hidden (235) - margin (0.25) - params (410780)
dev - 56.1430262461, MRR: 70.8425517205, P@1: 59.2592592593, P@5: 45.1851851852
test - 58.0309710102, MRR: 72.2087067097, P@1: 58.6021505376, P@5: 44.4086021505
-----
CNN - hidden (600) - margin (0.2) - params (360600)
dev - MAP: 55.5937497635, MRR: 68.5064322489, P@1: 56.6137566138, P@5: 44.6560846561
test - MAP: 58.2964082527, MRR: 72.7873753298, P@1: 59.6774193548, P@5: 42.9032258065
CNN - hidden (630) - margin (0.2) - params (378630)
dev - MAP: 55.9878176024, MRR: 68.8328888924, P@1: 57.1428571429, P@5: 45.291005291
test - MAP: 57.6971032469, MRR: 72.4588538147, P@1: 59.1397849462, P@5: 43.2258064516
-----
CNN - hidden (600) - margin (0.25) - params (360600)
dev - MAP: 56.2967027869, MRR: 69.1019762595, P@1: 57.1428571429, P@5: 45.6084656085
test - MAP: 58.6624320169, MRR: 72.9278686006, P@1: 60.2150537634, P@5: 43.1182795699
-----
CNN - hidden (630) - margin (0.25) - params (378630)
dev - MAP: 55.4258670699, MRR: 68.5898915585, P@1: 56.6137566138, P@5: 45.0793650794
test - MAP: 59.0191667916, MRR: 72.6776394391, P@1: 59.1397849462, P@5: 44.3010752688
-----
CNN - hidden (500) - margin (0.1)
dev - AUC(0.05): 0.658249845727
test - AUC(0.05): 0.638314942406
LSTM - hidden (100) - margin (0.1)
dev - AUC(0.05): 0.613165685101
-----
-----
LSTM - hidden (100) - margin (0.15) - lmbda (1e-3)
dev - AUC(0.05): 0.730344036241
test - AUC(0.05): 0.700761772508
CNN - hidden (600) - margin (0.15) - lmbda (1e-3)
dev - AUC(0.05): 0.727267957537
test - AUC(0.05): 0.720437251865
-----