MovieLens 100k benchmark results - Revision history

Zeno Gantner at 11:10, 2 February 2013

2013-02-02T11:10:13Z

Zeno Gantner at 23:46, 31 January 2013

2013-01-31T23:46:51Z

Zeno Gantner: /* Baseline Methods */

2013-01-31T23:01:54Z

Baseline Methods

Zeno Gantner at 10:08, 28 January 2013

2013-01-28T10:08:32Z

Zeno Gantner: first sketch of the first benchmark page!

2013-01-28T08:06:46Z

first sketch of the first benchmark page!

New page

''This page is a first example of how a benchmark page in RecSysWiki could look like. It is work in progress. Please contribute and comment.''

''Rationale: This is primarily meant to be a comparison between methods, not between tools. This is why we sort by method. At the same time, we state the version number and all input arguments for maximum reproducibility.''

''If there are two lines for one method, then the first line are results with the random seed set to 1; the second line (or otherwise the only line) contains the average results for 5 runs with random initialization.''

== Baseline Methods ==
{|
! Software
! Method
! 5-fold CV
! all-but-10
! References
|-
| MyMediaLite 3.07
| global average
|
|
|
|-
| MyMediaLite 3.07
| user average
|
|
|
|-
| MyMediaLite 3.07
| item average
|
|
|
|-
| MyMediaLite 3.07
| user and item bias
|
|
|
|}

== kNN-based Collaborative Filtering ==

== Matrix Factorization ==

== Attribute-Aware Methods ==

== Miscellaneous Methods ==

== Disclaimers ==
* The results presented here come with no warranty whatsoever. Use at your own risk.
* Most if not all results are self-reported by the implementations, which may contain bugs in their evaluation routines.
* The results are not necessarily fair towards the compared methods and implementations. There could be [[hyper-parameter]] [[overfitting]], or you could achieve a lot better results by better tuning.
* MovieLens 100k is one of the oldest existing collaborative filtering datasets, and it was dominating the literature for years, because it was one of the few available datasets. It could be that methods developed in that period have a certain bias towards this dataset. The dataset is also quite small by today's standards.

[[Category: Benchmark]]
[[Category: Dataset]]

@@ Line 23: / Line 23: @@
 == kNN-based Collaborative Filtering ==
 {|
@@ Line 38: / Line 52: @@
 |}
-== Matrix Factorization ==
 == Attribute-Aware Methods ==
-== Miscellaneous Methods ==
+== Other Methods ==

@@ Line 13: / Line 13: @@
 ! References
 |-
-| MyMediaLite 3.07 || GlobalAverage || 1.1256 || 1.1238
+| [https://github.com/zenogantner/recsys-benchmark/ml-100k/mymedialite.pl MyMediaLite 3.07] || GlobalAverage || 1.1256 || 1.1238
 |-
-| MyMediaLite 3.07 || UserAverage || 1.0437 || 1.0518
+| [https://github.com/zenogantner/recsys-benchmark/ml-100k/mymedialite.pl MyMediaLite 3.07] || UserAverage || 1.0437 || 1.0518
 |-
-| MyMediaLite 3.07 || ItemAverage || 1.0246 || 1.0453
+| [https://github.com/zenogantner/recsys-benchmark/ml-100k/mymedialite.pl MyMediaLite 3.07] || ItemAverage || 1.0246 || 1.0453
 |-
-| MyMediaLite 3.07 || UserItemBaseline || 0.9457 || 0.9695
+| [https://github.com/zenogantner/recsys-benchmark/ml-100k/mymedialite.pl MyMediaLite 3.07] || UserItemBaseline || 0.9413 || 0.9656
 |}
 == kNN-based Collaborative Filtering ==
 == Matrix Factorization ==

@@ Line 53: / Line 53: @@
 * MovieLens 100k is one of the oldest existing collaborative filtering datasets, and it was dominating the literature for years, because it was one of the few available datasets. It could be that methods developed in that period have a certain bias towards this dataset. The dataset is also quite small by today's standards.
 [[Category: Benchmark]]
 [[Category: Dataset]]