Carrot searches for documents, just like usual searching engine (it actually uses Google for that). After completing the list of documents (so-called ranked list), Carrot will attempt to group documents which have similar contents. This similarity is based on common phrases, which appear in the description part of a document. Therefore if you have two references like this:

Then these two documents could possibly be found in a group containing phrase "dance salsa". Different forms of a word are considered the same by Carrot, therefore "salsa dance" and "salsa dancing" would be considered identical and would appear in the same group.

After you type a query, you are presented with the standard ranked-list view of results. Remember, if you are in the process of evaluating Carrot, you should not be using this view (because it isn't anything new). To switch to Carrot's specific views, you use the simple menu just below the top-panel:

Clustered view

This view of data presents you with the groups of documents found (clusters mean groups). A sample group may look like this:

This group counts 48 documents, 97% of them contains the word "computer" in their description, 47% of them contains also "computer science". To display all documents of this group, click "display all", to see just the documents with phrase "computer science", click directly on that phrase.

Dynamic index view

If you are interested to find just an overview of what phrases were found in your query result, this view does that. You will be presented with all the phrases, which were found in the result of your query. You may sort these phrases alphabeticaly or according to length, or the number of documents they cover. A sample fragment of this view is shown below:

As you can see, phrase "Artificial intelligence" exists in 11 documents, from all the results obtained for the query, which happened to be "machine learning". To display a full list of phrase's documents, simply click on the one you want to explore.

Hints before you begin...

Carrot is not a fully tuned system, some results may not make mych sense. According to the experience we have, the best way to find information quickly is to:

1. Type most-generic questions first. E.g. when looking for types of salsa (the sauce), you could simply ask the query "salsa" and go through phrases found in dynamic index, which would tell you there are tomato salsas, green salsas, fruit salsas etc...
2. Do not get confused if the first groups in "clusters view" don't make much sense. They are sorted, but the sorting process is sometimes misleading.
3. If you get 'results could not be interpreted' system exceptions, please reformulate your query - some of Google pages have changed the structure and Carrot's parser fails to interpret them.

You may now begin the evaluation form. The process is pretty simple: two browser windows will open up after you click the "Begin evaluation" button below, one with the evaluation form and the other with Carrot main page. The locale will be set to English to achieve consistent interfaces between different evaluators.

Read instructions on the form and follow them. When searching, use the Carrot search page only - no other search engines may be used during the experiment. The evaluation, depending on your search experience, may take from 15 to 45 minutes. Please do not start it if you don't have the time necessary to finish the form - breaks are not allowed during evaluation as they distort statistical data collected by the system.

Thank you again for your time.