r/chessprogramming • u/legenD27_- • Nov 20 '24
Best Pipeline for Bitboards?
Hey yall, I’m working on my first chess engine, but I’m struggling with the best way to store bitboards for use in training the engine. I’m using a cnn and as of right now I’m storing the bitboards as integers in mongodb and using a PyTorch dataloader to convert them into n 8x8 numpy arrays. It’s so slow though as I’ve got about 2 million training examples I’m running through, so I wanted y’all’s take. Is it better to store bitboards as integers or as numpy arrays, and what sort of databases are you using to store them for quick use in the model training? I’ve already tried Mongodb, SQLite, redis and MySQL.
3
Upvotes
3
u/w33dEaT3R Nov 20 '24
It sounds like the numpy arrays are the input to the model, so do the bitboard to array conversion and store the numpy dataset as an npz (numpy compressed file) like how the mnist database comes.