Comments on: Similarity Measure: Cosine Similarity or Euclidean Distance or Both

By: Richard

Richard — Mon, 04 Jan 2010 06:48:05 +0000

Hi All,
Thanks for providing this greate tutorial. I am new and just start learning about data mining. I am sorry if my question sounds childish. I know how to calculate tf/idf. Thanks for egarcia, now i also know how to calculate cosine similarity. But I have a question.
Lets say i have two documents. each document has 2 words. 1 word is similar in both documents. So the tf/idf would be like doc A (0,0.5) doc B (0,0.5). Should I calculate cosine similarity on tf/idf values? If I do it return 1 while i can see both documents are 50% same not 100%.
Am I missing some thing? Should I calculate cosine similarity only on IDF values or Tf values?
Thanks

By: egarcia

egarcia — Thu, 30 Oct 2008 23:57:04 +0000

Anand: I’m a he, not a she.

Hooman: With a positional inverted index, word orden and positional information can be stored into vectors and all kind of similarity scores computed.

BTW, at http://irthoughts.wordpress.com/2008/10/29/similarity-pearson-and-spearman-coefficients/ we show the connection between Pearson and Spearman’s Correlation Coefficients with cosine angles and dot products. None of these are distances. The links listed in that post might help to clear up the difference between similarity and distance.

Another thing. Few IRs have, unfortunately, used the expression ‘Similarity Distance’. Avoid it. This expression is an oxymoron as Distance is Dissimilarity.

I hope this help.

By: Hooman

Hooman — Thu, 25 Sep 2008 08:35:49 +0000

Dear All
As you know, cosine similarity measure loses the word order – while is exploited for measuring the similarity of texts. What other methods exist that integrate some syntactic information like word order in measuring the similarity?
Thanks very much

By: James

James — Mon, 21 Jul 2008 05:05:50 +0000

For more discussion see:
http://www.pui.ch/phred/automated_tag_clustering/

By: James

James — Mon, 21 Jul 2008 04:52:48 +0000

The first document has only 3/300 or 1 percent the number of mentions of the word “machine” and likewise only 4/400 or 1 percent the number of mentions of the word “learning” as compared to the second document. This small frequency is within the margin of error of most statistical measurements and therefore I recommend any supposedly “similarity” be expressed as a probability. To see my observation in a different light, imagine a triangle with only 3 points or vertices expressed. Now imagine a triangle with 300 points expressed filling out the 3 sides. Now introduce the same random point in both sets. In the first set, the “triangle” will most likely becomes a square or rhombus (quite a different classification) while the random point in the second set doesn’t greatly disturb our picture. It remains a triangle with some “noise”. Consequently, it would make sense to limit so-called “comparisons” or “similarity” to sets with the same order of magnitude of points to preserve statistical accuracy.

By: e.garcia

e.garcia — Thu, 03 Jul 2008 17:47:02 +0000

To all above:

You cannot blindfold exchange distance and similarity without knowing what is the model and the subject of that model.
http://www.miislita.com/searchito/binary-similarity-calculator.html

By: Anand Kishore

Anand Kishore — Sun, 15 Jun 2008 05:05:26 +0000

Jack: Well she was making the point that the two cannot be combined together to give you a number that takes both similarity and magnitude into account.

By: Jack

Jack — Thu, 12 Jun 2008 13:20:37 +0000

I’m not an expert but I’m not sure what point Dr. E. Garcia is trying to make here…cosine similarity does use magnitudes in the calculation but the calculation in fact normalizes the magnitudes, and the output of that calculation is an angle, and nothing else.

By: nthio

nthio — Fri, 16 May 2008 08:44:56 +0000

Hi Anand and Dr. E.Garcia,
May I quote the original post:
“For example, the cosine between a document which has ‘machine’ occurring 3 times and ‘learning’ 4 times and another document which has ‘machine’ occurring 300 times and ‘learning’ 400 times will hint that the two documents are pointing in almost the same direction. If magnitude (euclidean distance) was taken into account, the results would be quite different.”

I’m kind of getting what you (Anand) intended to say from the early post, which is the cosine similarity is kind of normalizing the magnitude of the vector in its calculation. For example the coordinates A(3,4), B(5,12) and B’(500,1200); the cosine similarity between A and B, and A and B’ are the same; however, the euclidean distance is of course different.
In a context of query and document searhing, the larger the document tends to scales up (in ideal case while it may not always practical; scales up the vector dimension equally, so although the magnitude i.e. the size of the document increases, the vector direction remains the same). In this case, cosine similarity may have a desirable normalizing effect compared to using the euclidean distance, which will tends to favor document with similar scale (as well as the direction) from the query. In such application, we may expect that the magnitude of query can be much less than the document searched – thus using euclidean will tend to favor smaller documents.
Perhaps if the application is different, such as comparing the strict similarity between two documents, euclidean distance may have advantage if we intend to account similar magnitude of both documents.

By: Robbie Haertel

Robbie Haertel — Fri, 25 Apr 2008 21:27:14 +0000

If vectors X and Y are L2 normalized, it is possible to prove that using the cosine distance (1 – cosine similarity) is interchangeable with the euclidean distance.