MediaEval Placing Task 2013 Data Set

Introduction

This data set was used by the participants of the MediaEval 2013 Placing Task. It contains more than 8.5 million training images and more than 250,000 test images which were crawled from Flickr (all with Creative Commons license). MediaEval is a multimedia benchmark initiative and the Placing Task was one of several tasks in 2013.

The Placing Task required participants to estimate the geographical coordinates (latitude and longitude) of photos, as well as to indicate how "placeable" they actually are by estimating the error of the predicted locations.
Pinpointing the exact or approximate coordinates of a photo is not merely an intellectual challenge. It has clear utility and potential for far-reaching impact in multimedia and beyond. As an example, rescue teams may be able to narrow down where exactly a family disappeared in a remote area by discovering the locations shown in photos uploaded to a social network before they lost contact. The placing task integrates various aspects of multimedia: textual metadata, image content, location, time, users and context.

The Placing Task 2013 was organized by:

  • Bart Thomee (Yahoo! Research Barcelona, Spain)
  • Michele Trevisiol (Yahoo! Research Barcelona, Spain)
  • Claudia Hauff (Delft University of Technology, Netherlands)
with Xinchao Li (Delft University of Technology, Netherlands) as task auxiliary.

Proceedings & Citation

The proceedings of the workshop are available online.

If you use the data set, please cite the Placing Task 2013 overview paper:

@inproceedings{HTT13PlacingTask,
author = {Claudia Hauff and Bart Thomee and Michele Trevisiol},
title = {Working Notes for the Placing Task at MediaEval 2013},
booktitle = {MediaEval},
year = {2013},
ee = {http://ceur-ws.org/Vol-1043/mediaeval2013_submission_95.pdf},
}
				

Extracted Features

The training data contains more than 8.5 million images. The test data contains more than a quarter of a million images. All images were crawled from Flickr and come with a Creative Commons license.
Provided are the textual metadata of each image as well as image features extracted with LIRE (version 0.9.3). Due to their size, the data has been split into several files.
Please note, that the following files contain the features of both, the training and test images.

Raw Images

To facilitate the exploitation of different visual features, we also provide a download of the raw images (i.e. the image files retrieved through photoLink). They can be found here: http://petamedia.p2p-next.org/imagesPlacing2013/. Since not all raw images were captured during the original crawl (some had to be retrieved later again), a handful of images that are available in the metadata are missing from this download.

Latitude/Longitude of Training Data

The following files contain the latitude/longitude information for the training data. It is given in the format: photoID latitude longitude.

Test Data

The test data is distributed in a Russian doll approach, which means that you should only process one of the five possible test sets (the largest one that you can manage). All test images of test1 are contained in test2 and so on.
The set of users contributing to the test data is in all cases distinct from the set of users contributing to the training data.
The format is: photoID latitude longitude.

To evaluate the estimation accuracy, you can use our evaluation script (Perl). It was used to generate the results published in the MediaEval Working Notes 2013.

Results

This section provides an overview of results achieved on the test data. Note, that here we only include results that make exclusive use of the training data, i.e. approaches that exploit additional data for training are ignored (it should be mentioned though that they tend to improve the results even further - as more data is available).
When using the data for comparative purposes, the matching test set should be processed (results vary depending on the test set!).