Finally, I’ve found it! After thorough testing and experimentation my conclusions are: The correct way is to calculate maximum possible difference between the two colors. Formulas with any kind of estimated average/typical difference had room for discontinuities. I was unable to find a working formula that calculates the distance without blending RGBA colors with some … Read more
you can use this function (cop^H^H^Hadapted from http://www.artfulsoftware.com/infotree/queries.php#552): CREATE FUNCTION `levenshtein`( s1 text, s2 text) RETURNS int(11) DETERMINISTIC BEGIN DECLARE s1_len, s2_len, i, j, c, c_temp, cost INT; DECLARE s1_char CHAR; DECLARE cv0, cv1 text; SET s1_len = CHAR_LENGTH(s1), s2_len = CHAR_LENGTH(s2), cv1 = 0x00, j = 1, i = 1, c = 0; IF … Read more
It sounds like you want a binary delta or perhaps an index derived from the application of a binary delta (like it’s size). You could then compare this index to some baseline that you determine experimentally to decide if it’s a “clone” or not. There are a lot of similarities between compression and delta creation, … Read more
I have found out that the Levenshtein distance may be good when you are searching a full string against another full string, but when you are looking for keywords within a string, this method does not return (sometimes) the wanted results. Moreover, the SOUNDEX function is not suitable for languages other than english, so it … Read more
Expanding on my wiki-walk comment in the errata and noting some of the ground-floor literature on the comparability of algorithms that apply to similar problem spaces, let’s explore the applicability of these algorithms before we determine if they’re numerically comparable. From Wikipedia, Jaro-Winkler: In computer science and statistics, the Jaro–Winkler distance (Winkler, 1990) is a … Read more
The best package I’ve seen for this is Gensim, found at the Gensim Homepage. I’ve used it many times, and overall been very happy with it’s ease of use; it is written in Python, and has an easy to follow tutorial to get you started, which compares 9 strings. It can be installed via pip, … Read more
I’ve recently done a similar task, although I was matching new data to existing names in a database, rather than looking for duplicates within one set. Name matching is actually a well-studied task, with a number of factors beyond what you’d consider for matching generic strings. First, I’d recommend taking a look at a paper, … Read more
The way you have it, similarity between every element and every other element of the table has to be calculated (almost a cross join). If your table has 1000 rows, that’s already 1,000,000 (!) similarity calculations, before those can be checked against the condition and sorted. Scales terribly. Use SET pg_trgm.similarity_threshold and the % operator … Read more
You could take a look at two answers on SO itself: this one is about image comparison itself, offering links to stuff in C++ (if I read correctly) while this one offers links to broader approaches, one being in C. I would suggest starting with the second link since there’s links on that discussion that’ll … Read more