<a name="benchmarks"></a> Benchmarks ------------------------- The reference system uses an Intel i7-9700K cpu, and runs Ubuntu x64 20.04. The [open source benchmark program] is compiled with `clang` v10.0 using `-O3` flag. | Hash Name | Width | Bandwidth (GB/s) | Small Data Velocity | Quality | Comment | | --------- | ----- | ---------------- | ----- | --- | --- | | __XXH3__ (SSE2) | 64 | 31.5 GB/s | 133.1 | 10 | __XXH128__ (SSE2) | 128 | 29.6 GB/s | 118.1 | 10 | _RAM sequential read_ | N/A | 28.0 GB/s | N/A | N/A | _for reference_ | City64 | 64 | 22.0 GB/s | 76.6 | 10 | T1ha2 | 64 | 22.0 GB/s | 99.0 | 9 | Slightly worse [collisions] | City128 | 128 | 21.7 GB/s | 57.7 | 10 | __XXH64__ | 64 | 19.4 GB/s | 71.0 | 10 | SpookyHash | 64 | 19.3 GB/s | 53.2 | 10 | Mum | 64 | 18.0 GB/s | 67.0 | 9 | Slightly worse [collisions] | __XXH32__ | 32 | 9.7 GB/s | 71.9 | 10 | City32 | 32 | 9.1 GB/s | 66.0 | 10 | Murmur3 | 32 | 3.9 GB/s | 56.1 | 10 | SipHash | 64 | 3.0 GB/s | 43.2 | 10 | FNV64 | 64 | 1.2 GB/s | 62.7 | 5 | Poor avalanche properties | Blake2 | 256 | 1.1 GB/s | 5.1 | 10 | Cryptographic | SHA1 | 160 | 0.8 GB/s | 5.6 | 10 | Cryptographic but broken | MD5 | 128 | 0.6 GB/s | 7.8 | 10 | Cryptographic but broken [open source benchmark program]: https://github.com/Cyan4973/xxHash/tree/release/tests/bench [collisions]: https://github.com/Cyan4973/xxHash/wiki/Collision-ratio-comparison#collision-study note 1: Small data velocity is a _rough_ evaluation of algorithm's efficiency on small data. For more detailed analysis, please refer to [the wiki](https://github.com/Cyan4973/xxHash/wiki/Performance-comparison#benchmarks-concentrating-on-small-data-). note 2: On 32-bit applications, the ranking between hashes differs drastically, strongly favoring 32-bit hashes, See [this section of the wiki](https://github.com/Cyan4973/xxHash/wiki/Performance-comparison#bandwidth-on-x86-with-various-data-size-of-len-2n-32-bit-friendliness) for illustration. note 3: some algorithms feature _faster than RAM_ speed. In which case, they can only reach their full speed when input data is already in CPU cache (L3 or better). Otherwise, they max out at RAM speed limit. <br/> <a name="other-languages"></a> ## Multiple languages The following versions produce xxHash-compatible results in different languages. |Language |Author |URL | |-- |-- |-- | |__Java__ (XXH32 and XXH64) |Adrien Grand |https://github.com/jpountz/lz4-java |__Java__ (XXH64) |Vsevolod Tolstopyatov|https://github.com/OpenHFT/Zero-Allocation-Hashing |__JavaScript__ (port) |Pierre Curto |https://npmjs.org/package/xxhashjs |__JavaScript__ (nodeJS) |Brian White |https://npmjs.org/package/xxhash |__JavaScript__ (nodeJS, xxh3)|Nhan Khong |https://github.com/ktrongnhan/xxhash-addon |__JSX__ (static JavaScript)|Yoshiki Shibukawa |https://www.npmjs.org/package/xxhash.jsx |__Python__ |Yue Du |https://pypi.python.org/pypi/xxhash/ |__C#__ (XXH32) |Seok-ju Yun |https://github.com/noricube/xxHashSharp |__C#__ (XXH64) |Brandon Dahler |http://www.nuget.org/packages/System.Data.HashFunction.xxHash |__C#__ (endian independent)|Otaku |https://github.com/differentrain/YYProject.XXHash |__C#__ (.net std 2.0) |Sedat Kapanoğlu |https://github.com/ssg/HashDepot#xxhash |__C++ 17__ |Red Gavin |https://github.com/RedSpah/xxhash_cpp |__PHP__ |Nir Heimann |https://github.com/nheimann1/php-xxhash |__PHP7__ |Craig R Megasaxon |https://github.com/Megasaxon/php-xxhash |__PHP__ (port) |Scott Dutton |https://github.com/exussum12/xxhash |__R__ |Dirk Eddelbuettel |https://github.com/eddelbuettel/digest |__R__ (XXH3) |mikefc |https://github.com/coolbutuseless/xxhashlite |__Swift__ |Daisuke T |https://github.com/daisuke-t-jp/xxHash-Swift |__Go__ (XXH32) |Stéphane Bunel |https://bitbucket.org/StephaneBunel/xxhash-go |__Go__ (XXH64) |Ahmed Waheed |https://github.com/OneOfOne/xxhash |__Go + ASM__ |Caleb Spare |https://github.com/cespare/xxhash |__Go__ (XXH3) |Jeff Wendling |https://github.com/zeebo/xxh3 |__Perl__ |Sanko Robinson |https://metacpan.org/module/Digest::xxHash |__Perl__ (streaming) |Bela Bodecs |https://github.com/DoubleBB/digest-xxhash64 |__Ruby__ (port) |Justin W Smith |http://rubygems.org/gems/ruby-xxHash |__Ruby__ (wrapper) |Vasiliy Ermolovich |https://rubygems.org/gems/xxhash |__Ruby__ (digest::class) |konsolebox |https://rubygems.org/gems/digest-xxhash |__Rust__ |Jake Goulding |https://libraries.io/cargo/twox-hash |__D__ |Masahiro Nakagawa |https://github.com/repeatedly/xxhash-d |__Pascal__ |Vojtěch Čihák |http://sourceforge.net/projects/xxhashfpc |__Scala__ |Desmond Yeung |https://github.com/desmondyeung/scala-hashing |__Lua__ (binding) |Masatoshi Teruya |https://github.com/mah0x211/lua-xxhash |__Lua__ (jit, XXH32) |szensk |https://github.com/szensk/luaxxhash |__Lua__ (jit, XXH64) |Soojin Nam |https://github.com/sjnam/luajit-xxHash |__Julia__ |Hanan Rosemarin |https://github.com/hros/XXhash.jl |__Erlang__ |Pierre Matri |https://github.com/pierresforge/erlang-xxhash |__Haskell__ |Christian Marie |http://hackage.haskell.org/package/xxhash |__PicoLisp__ |mpech |https://git.envs.net/mpech/xxhash-picolisp |__Elixir__ |Mykola Konyk |https://github.com/ttvd/elixir-xxhash |__Tcl__ (XXH32, port) |D. Bohdan |https://wiki.tcl-lang.org/48790 |__Bash__ (port) |Devin Hussey |https://github.com/easyaspi314/xxbash |__sh__ (port) |Jan Chren |https://gitlab.com/rindeal/xxHashish |__Crystal__ |Lucjan Suski |https://github.com/methyl/xxhash |__Zig__ |clownpriest |https://github.com/clownpriest/xxhash <br/> <a name="references"></a> ## xxHash is used by #### Databases <div id="xxhashusedby_databases" class="container"> <ul class="list-inline"> <li><a href="http://prestodb.io/" ><img src="images/logo50/prestoDB.png" /><span> PrestoDB</span></a></li> <li><a href="https://rocksdb.org/" ><img src="images/logo50/rocksdb.png" /><span> RocksDB</span></a></li> <li><a href="https://www.mysql.com/" ><img src="images/logo50/mysql.png" /><span> MySQL</span></a></li> <li><a href="https://www.arangodb.org/" ><img src="images/logo50/arangoDB.png" /><span> ArangoDB</span></a></li> <li><a href="https://pgroonga.github.io/" ><img src="images/logo50/pgroonga.png" /><span> PGroonga</span></a></li> <li><a href="http://spark.apache.org/" ><img src="images/logo50/spark.png" /><span> Spark</span></a></li> </ul> </div> #### Games <div id="xxhashusedby_games" class="container"> <ul class="list-inline"> <li><a href="https://mafiagame.fandom.com/wiki/Mafia:_Definitive_Edition_Credits"><img src="images/logo50/mafiade.png"/><span> Mafia</span></a></li> <li><a href="https://ppsspp.org/" ><img src="images/logo50/ppsspp.png" /><span> PPSSPP</span></a></li> <li><a href="https://dolphin-emu.org/" ><img src="images/logo50/dolphin.png" /><span> Dolphin</span></a></li> <li><a href="https://cxbx-reloaded.co.uk"><img src="images/logo50/cxbx.png" /><span> Cxbx-reloaded</span></a></li> <li><a href="http://www.cocos2d.org/" ><img src="images/logo50/cocos2D.png" /><span> Cocos2D</span></a></li> <li><a href="https://www.lwjgl.org/" ><img src="images/logo50/lwjgl.png" /><span> LWJGL</span></a></li> <li><a href="http://fc-solve.shlomifish.org/"><span> Freecell Solver</span></a></li> </ul> </div> #### Filters <div id="xxhashusedby_filters" class="container"> <ul class="list-inline"> <li><a href="https://www.haproxy.org/" ><img src="images/logo50/haproxy.png" /><span> HAProxy</span></a></li> <li><a href="https://rspamd.com/" ><img src="images/logo50/rspamd.png" /><span> Rspamd</span></a></li> <li><a href="https://www.pfsense.org/" ><img src="images/logo50/pfsense.png" /><span> pfSense</span></a></li> <li><a href="http://freecode.com/projects/fio/"><img src="images/logo50/placeholder.png"/><span> fio</span></a></li> <li><a href="https://npmjs.org/package/bloomxx/"><img src="images/logo50/placeholder.png"/><span> bloomxx</span></a></li> <li><a href="http://devisedbydavid.com/open_source/bloom_filter"><img src="images/logo50/placeholder.png"/><span> C & Python Bloom Filter</span></a></li> <li><a href="https://github.com/mozilla-services/lua_bloom_filter"><img src="images/logo50/mozilla.png"/><span> LUA Bloom Filter</span></a></li> </ul> </div> #### File Transfer <div class="container"> <ul class="list-inline"> <li><a href="https://partnerhelp.netflixstudios.com/hc/en-us/articles/360000581207-Production-Assets-Data-Management"><img src="images/logo50/netflix.png"/><span> Netflix</span></a></li> <li><a href="https://rsync.samba.org/" ><img src="images/logo50/rsync.png" /><span> rsync</span></a></li> <li><a href="http://www.lz4.org/" ><img src="images/logo50/lz4.png" /><span> LZ4</span></a></li> <li><a href="https://pomfort.com/silverstack/" ><img src="images/logo50/silverstack.png" /><span> Silverstack</span></a></li> <li><a href="http://www.lespace.co.jp/file_bl/rapidcopy/rapidcopy.html"><img src="images/logo50/rapidcopy.png"/><span> Rapidcopy</span></a></li> <li><a href="https://www.hedgeformac.com/" ><img src="images/logo50/syncFactory.png" /><span> Hedge</span></a></li> <li><a href="https://ipmsg.org/tools/fastcopy.html.en"><img src="images/logo50/fastcopy.png" /><span> fastcopy</span></a></li> <li><a href="https://codesector.com/teracopy" ><img src="images/logo50/teracopy.png" /><span> TeraCopy</span></a></li> <li><a href="https://www.dragonflybsd.org/hammer/" ><img src="images/logo50/hammer.png" /><span> Hammer 2</span></a></li> </ul> </div> #### Other <div id="xxhashusedby_other" class="container"> <ul class="list-inline"> <li><a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=59e1a2f4bf83744e748636415fde7d1e9f557e05"><img src="images/logo50/linux.png"/><span> Linux</span></a></li> <li><a href="https://docs.microsoft.com/en-us/azure/kusto/query/hashfunction"><img src="images/logo50/azure50.png"/><span> Microsoft Azure</span></a></li> <li><a href="https://devblogs.microsoft.com/cppblog/linker-throughput-improvement-in-visual-studio-2019/"><img src="images/logo50/microsoft.png"/><span> Visual Studio</span></a></li> <li><a href="https://forum.huawei.com/enterprise/en/how-to-understand-inline-deduplication-and-compression/thread/604642-891"><img src="images/logo50/huawei.png"/><span> Dorado</span></a></li> <li><a href="https://wiki.freebsd.org/pkgng" ><img src="images/logo50/freebsd.png" /><span> PKG</span></a></li> <li><a href="https://www.qemu.org/" ><img src="images/logo50/qemu.png" /><span> Qemu</span></a></li> <li><a href="https://btrfs.wiki.kernel.org/index.php/Main_Page"><img src="images/logo50/btrfs.png"/><span> btrfs</span></a></li> <li><a href="https://www.xpra.org/" ><img src="images/logo50/xpra.png" /><span> Xpra</span></a></li> <li><a href="http://www.teamviewer.com/" ><img src="images/logo50/teamviewer.png" /><span> TeamViewer</span></a></li> <li><a href="http://factorcode.org/" ><img src="images/logo50/factor.png" /><span> Factor</span></a></li> <li><a href="http://nvlabs.github.io/nvbio/index.html"><img src="images/logo50/nvbio.png" /><span> nVBio</span></a></li> <li><a href="http://dvisvgm.de/"> <img src="images/logo50/placeholder.png" /><span> dvisvgm</span></a></li> <li><a href="http://www.fastbuild.org/"> <img src="images/logo50/fastbuild.png" /><span> FastBuild</span></a></li> <li><a href="http://keypirinha.com/"> <img src="images/logo50/keypirinha.png" /><span> Keypirinha</span></a></li> <li><a href="http://quickhash-gui.org/"> <img src="images/logo50/quickhash.png" /><span> QuickHash</span></a></li> </ul> </div> ### Special Thanks to - [Takayuki Matsuoka](https://github.com/t-mat/) for creating `xxhsum -c` and the canvas of this webpage. - [Mathias Westerdahl](https://github.com/JCash) for introducing the first version of XXH64. - [Devin Hussey](https://github.com/easyaspi314) for the excellent low-level optimizations on XXH3 and XXH128. ---