SimHashPHP

This is the second version of SimHashPHP. If you are using the version 1 and don't want to update your code, please refer to the 1.0-security branch (https://github.com/tgalopin/SimHashPhp/tree/1.0-security). The 1.0 branch will be maintained until the release of a v3 but only the v2 will have lastest features.

What is SimHashPHP ?

SimHashPHP is a PHP library that port the SimHash algorithm in PHP. This algorithm, created by Moses Charikar, provides an efficient way to compute a similarity index between two texts. It is used by Google internally to detect dupplicate content.

See "SimHash or the way to compare quickly two datasets" for more informations.

How to use it ?

Install it with Composer:

composer require tga/simhash-php

Once installed, include vendor/autoload.php to load the library.

The concept of SimHash is described in this article. Here are few examples:

<?phprequire 'vendor/autoload.php';$text1 = <<<EOTGeorge Headley (1909–1983) was a West Indian cricketer who played 22 Test matches, mostly before the Second World War.Considered one of the best batsmen to play for West Indies and one of the greatest cricketers of all time, he alsorepresented Jamaica and played professionally in England. Headley was born in Panama but raised in Jamaica where hequickly established a cricketing reputation as a batsman. West Indies had a weak cricket team through most of Headley'scareer; as their one world-class player, he carried a heavy responsibility, and they depended on his batting. He battedat number three, scoring 2,190 runs in Tests at an average of 60.83, and 9,921 runs in all first-class matches at anaverage of 69.86. He was chosen as one of the Wisden Cricketers of the Year in 1934.EOT;$text2 = <<<EOTGeorge Headley was a West Indian cricketer who played 22 Test matches, mostly before the Second World War.Considered one of the best batsmen to play for West Indies and one of the greatest cricketers of all time, he alsorepresented Jamaica and played professionally in England. Headley was born in Panama but raised in Jamaica where hequickly established a cricketing reputation as a batsman. West Indies had a weak cricket team through most of Headley'scareer; as their one world-class player, he carried a heavy responsibility, and they depended on his batting. He battedat number three, scoring 2,190 runs in tests at an average of 60.83, and 9,921 runs in all first-class matches at anaverage of 69.86. He was chosen as one of the Wisden Cricketers of the Year.EOT;$simhash = new TgaSimHashSimHash();$extractor = new TgaSimHashExtractorSimpleTextExtractor();$comparator = new TgaSimHashComparatorGaussianComparator(3);$fp1 = $simhash->hash($extractor->extract($text1), TgaSimHashSimHash::SIMHASH_64);$fp2 = $simhash->hash($extractor->extract($text2), TgaSimHashSimHash::SIMHASH_64);var_dump($fp1->getBinary());var_dump($fp2->getBinary());// Index between 0 and 1 : 0.80073740291681var_dump($comparator->compare($fp1, $fp2));