This website is a virtual proof that I'm awesome http://martin.ankerl.com/ Where I've spent way too much time creating benchmarks of C++ hashmaps - It’s been over 3 years since I’ve spent considerable time finding the best C++ hashmap. After several requests I finally gave in and redid the benchmark with state of C++ hashmaps as of August 2022. This took much more work than I initially anticipated, mostly due to the fact that benchmarks take a looong time, and writing everything up and creating a representation that is actually useful takes even more time. Thanks everyone who annoyingly kept asking me for updates :wink: TL;DR: Here are the benchmark results! Table of Contents Benchmark Infrastructure Hardware Software Benchmarks Stable References Modifying Numbers Benchmarks... Sat, 27 Aug 2022 00:00:00 +0200 http://martin.ankerl.com/2022/08/27/hashmap-bench-01/ http://martin.ankerl.com/2022/08/27/hashmap-bench-01/ Some helpful Tricks - I recently upgraded from my 8 year old Kindle Keyboard (2011) to a brand new Kindle Paperwhite (2019, generation 10). So far, it is a great upgrade. Here are some tricks that I found useful: Custom Fonts Connect the kindle with USB, you’ll see a folder fonts. I’ve tried a few fonts, and really like Literata. It’s very similar to Amazon’s Bookerly, and made for e-books. It’s also the font you’ll see in the header for this post. Here’s how to install it: Go to https://fonts.google.com/specimen/Literata Click “Select this font” Click on the new popup Click the download arrow to... Fri, 23 Aug 2019 00:00:00 +0200 http://martin.ankerl.com/2019/08/23/kindle-paperwhite-tips/ http://martin.ankerl.com/2019/08/23/kindle-paperwhite-tips/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion 👈 In conclusion, I can only say one thing with certainty: there is not one single best hashmap. What’s best for you will depend on your usecase, and I advise you to create a representative benchmark for your use... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-05-conclusion/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-05-conclusion/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating 👈 Conclusion Iteration benchmark works like this: I create a Map<uint64_t, uint64_t>, and each time I insert one random element the whole map is iterated. Once 50k elements are inserted, elements are removed one by one and each time the... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-06-result-IterateIntegers/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-06-result-IterateIntegers/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string 👈 Iterating Conclusion Again much like Find 1-2000 size_t, except that this time we use 13 byte strings. Every time we insert 4 strings, 800 lookups are performed. This is repeated until 1 million entries are inserted. Results Hashes In Insert... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-05-result-RandomFindString_1000000/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-05-result-RandomFindString_1000000/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string 👈 Find 1 – 1M std::string Iterating Conclusion This test is practically exactly the same as Find 1-2000 size_t, except that it uses 100 byte long std::string. There are 4000 lookups every 4 inserts, until 100k entries inserts. Results Hashes In Insert & Erase std::string we... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-04-result-RandomFindString/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-04-result-RandomFindString/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t 👈 Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion This benchmark is the same as previous, but with 4000 lookups for every 4 inserts until the map contains 500k elements. Results Hashes Same behavior as for the other benchmarks: robin_hood::hash is fastest, absl::Hash second, and libstdc++-v3 problematic.... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-03-result-RandomFind_500000/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-03-result-RandomFind_500000/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t 👈 Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion In many use cases find performance is probably considered the most important benchmark. This benchmark was tricky to implement, as it should be as unbiased as possible. It tries to do the following: Lookup with different probability of... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-02-result-RandomFind_2000/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-04-02-result-RandomFind_2000/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string 👈 Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion This benchmark is very similar to Insert & Erase, but with a Map<std::string, std::string>. It constantly inserts and removes random strings. The benchmark is run with different sizes of std::string: 7, 8, 13, 100, and 1000 bytes. Each... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-04-result-RandomInsertEraseStrings/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-04-result-RandomInsertEraseStrings/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t 👈 Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion The core of the benchmark is this loop: Map<uint64_t, uint64_t> map; for (size_t i = 0; i < 50'000'000; ++i) { map.emplace(rng() & bitMask, i); map.erase(rng() & bitMask); } On first glance it looks similar to the previous... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-03-result-RandomInsertErase/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-03-result-RandomInsertErase/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int 👈 Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion This benchmark has been adapted from attractivechaos’ Revisiting hash table performance code. It basically contains this main part: Map<int, int> map; for (size_t i = 0; i < 50'000'000; ++i) { checksum += ++map[rng(max_rng)]; } Here rng(max_rng) creates... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-02-result-RandomDistinct2/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-02-result-RandomDistinct2/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int 👈 Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion Now it gets interesting. This benchmark benchmarks a few things at once: Insert 100 million random int into a Map<int, int>. Clear all entries with clear(). Reinsert 100 million random int into the same cleared map. Remove all... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-01-result-InsertHugeInt/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-03-01-result-InsertHugeInt/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction 👈 Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion Almost the same as Construction & Destruction, but this time a single element is inserted. All maps that do lazy initialization are now forced to actually provide storage memory, and they have to calculate the hash function at... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-02-02-result-CtorDtorSingleEntryMap/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-02-02-result-CtorDtorSingleEntryMap/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview Construction Benchmarks Construction & Destruction 👈 Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion To get started, the first benchmark is a very simple one: measure how fast the hashmap can be constructed and destructed. Some maps perform some kind of lazy initialization, where data is only allocated when the first element... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-02-01-result-CtorDtorEmptyMap/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-02-01-result-CtorDtorEmptyMap/ Finding the Fastest, Memory Efficient Hashmap - Table of Contents Overview 👈 Construction Benchmarks Construction & Destruction Construction & Insert 1 int & Destruction Modifying Benchmarks Insert & Erase 100M int Insert & Access with Varying Probability int Insert & Erase uint64_t Insert & Erase std::string Accessing Find 1 – 2000 uint64_t Find 1 – 500k uint64_t Find 1 – 100k std::string Find 1 – 1M std::string Iterating Conclusion I’ve spent a long time developing my robin_hood::unordered_map, and after claiming that it is now the fastest hashmap I understandably got quite a few skeptic comments. Some of the comments were quite right, and my benchmarks were not... Mon, 01 Apr 2019 00:00:00 +0200 http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-01-overview/ http://martin.ankerl.com/2019/04/01/hashmap-benchmarks-01-overview/ With lots of benchmarks - While playing around with fast random number generators, I’ve started contemplating what’s the fastest way to uniformly generate random boolean values. The correct solution is this (never mind the bad seeding of mt19937, but it’s just too cumbersome to do it correctly): std::mt19937 rng(std::random_device{}()); bool rand_bool = std::uniform_int_distribution<>{0, 1}(rng); But is this fast? unfortunately, not so much. It can’t be fast, because each call to the std::uniform_int_distribution<> object has to request a new random number from rng, which means from the 32 random bits that are generated only a single one is used. What a colossal waste. On my quest... Sat, 08 Dec 2018 00:00:00 +0100 http://martin.ankerl.com/2018/12/08/fast-random-bool/ http://martin.ankerl.com/2018/12/08/fast-random-bool/ UTXO in all it's glory - About 5 years ago I’ve created a nice graph of all the bitcoin addresses. I got this: A year later I’ve tried to recreate it, but couldn’t because my PC at that time did not have enough RAM for znort987’s parser that I had used. Fast forward to September 2018, bitcoin now has an a fantastic simple REST interface! So I’ve tried again to create an updated graph of all unspent bitcoin transactions. Unfortunately, my previously used gnuplot script didn’t cut it any more. Too much RAM useage even for my now 32GB machine. I wrote my own renderer and... Thu, 20 Sep 2018 00:00:00 +0200 http://martin.ankerl.com/2018/09/20/bitcoin-unspent-transaction-outputs-graph/ http://martin.ankerl.com/2018/09/20/bitcoin-unspent-transaction-outputs-graph/ I have just finished moving my homepage from WordPress to jekyll + GitHub + Cloudflare. I must say this is really a fantastic setup! I have multiple reasons for that: GitHub Pages are free. Thanks to Cloudflare we get free HTTPS and caching support. Security: WordPress is a constant hassle: my website has been hacked multiple times due to old plugins or not running a recent enough wordpress release. Speed: static websites are fast and practically maintainence free Ease of development: jekyll & git makes development very convenient. From WordPress to Jekyll Unfortunately converting this blog was quite a bit... Sat, 22 Jul 2017 13:03:00 +0200 http://martin.ankerl.com/2017/07/22/free-secure-web-jekyll-github-pages-cloudflare/ http://martin.ankerl.com/2017/07/22/free-secure-web-jekyll-github-pages-cloudflare/ While cleaning up my blog I have found an old ruby program from 2005: rtd.rb. What this does it it can create pretty graphs from ruby class hierarchy. Interestingly, it still works :smiley: It is quite easy to use, here is a small tutorial: Usage Tutorial Generate graph for the ruby module time. Create a file rtd-time.rb with this content: require "./rtd.rb" stats = RubyToDot.new stats.hide_current_state require 'time' puts stats.generate Now run this command (requires dot, install in Ubuntu with sudo apt install graphviz): ruby rtd-time.rb | dot -Tpng >time.png The generates this graph: This does not yet look very... Sat, 22 Jul 2017 08:26:58 +0200 http://martin.ankerl.com/2017/07/22/ruby-to-dot/ http://martin.ankerl.com/2017/07/22/ruby-to-dot/ I’ve recently started developing more in Linux, here is a collection of tools, tips and tricks to get the most out of the box. I’ll expand this list from time to time. Appearance Terminal & Editor font: Hack Here is a test pattern I am using to evaluate fonts: o0O s5S z2Z !|l1Iij {([|})] .,;: ``''"" a@#* vVuUwW <>;^°=-~ öÖüÜäÄßµ \/\/ the quick brown fox jumps over the lazy dog THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG 0123456789 &-+@ for (int i=0; i<j; ++i) { } This is how the pattern looks like with Hack, my favourite programming... Tue, 09 May 2017 08:20:30 +0200 http://martin.ankerl.com/2017/05/09/essential-linux-dev-stuff/ http://martin.ankerl.com/2017/05/09/essential-linux-dev-stuff/