Re: Stockfish 3 PA_GTB
Posted: Fri Jun 07, 2013 8:21 pm
user,
I did an implementation using LMDB today and it's pretty nice. Super API.
On OSX, the file size is much smaller and a better reflection of the tiny bits of data which are being stored (19 bytes per record, 8 byte key + 11 byte data). The major disadvantage vis a vis QDBM is that LMDB memory-mapped files on Windows don't grow as incrementally as the QDBM files. They start at 1MB and expand (presumably 1MB at a time) to fit the size of the data. Not a big deal, I guess, but it will change the way "Prune" works, since pruning will basically lead to the complete decimation of all data in the PH file on Windows.
On the other hand, the user can specify a hard limit to the file, and if I can come up with a good way to deal with overflow (new entries will probably just have to replace the first entry found with a lower depth), it's pretty cool. Prune will drop off, in that case, or be offered as a way to eliminate entries at a certain depth and below en masse.
I can't say all that much about speed, since the biggest time cost is doing the TT lookups on the interesting positions. PH file read/write is nothing compared to that.
Once I write a conversion from the QDBM format to LMDB and work out how to deal with pruning and overflow, I'll send you something to try, if you're interested (just PM me your system details).
Jeremy
I did an implementation using LMDB today and it's pretty nice. Super API.
On OSX, the file size is much smaller and a better reflection of the tiny bits of data which are being stored (19 bytes per record, 8 byte key + 11 byte data). The major disadvantage vis a vis QDBM is that LMDB memory-mapped files on Windows don't grow as incrementally as the QDBM files. They start at 1MB and expand (presumably 1MB at a time) to fit the size of the data. Not a big deal, I guess, but it will change the way "Prune" works, since pruning will basically lead to the complete decimation of all data in the PH file on Windows.
On the other hand, the user can specify a hard limit to the file, and if I can come up with a good way to deal with overflow (new entries will probably just have to replace the first entry found with a lower depth), it's pretty cool. Prune will drop off, in that case, or be offered as a way to eliminate entries at a certain depth and below en masse.
I can't say all that much about speed, since the biggest time cost is doing the TT lookups on the interesting positions. PH file read/write is nothing compared to that.
Once I write a conversion from the QDBM format to LMDB and work out how to deal with pruning and overflow, I'll send you something to try, if you're interested (just PM me your system details).
Jeremy
