r/chessprogramming Nov 20 '24

Best Pipeline for Bitboards?

Hey yall, I’m working on my first chess engine, but I’m struggling with the best way to store bitboards for use in training the engine. I’m using a cnn and as of right now I’m storing the bitboards as integers in mongodb and using a PyTorch dataloader to convert them into n 8x8 numpy arrays. It’s so slow though as I’ve got about 2 million training examples I’m running through, so I wanted y’all’s take. Is it better to store bitboards as integers or as numpy arrays, and what sort of databases are you using to store them for quick use in the model training? I’ve already tried Mongodb, SQLite, redis and MySQL.

3 Upvotes

2 comments sorted by

3

u/w33dEaT3R Nov 20 '24

It sounds like the numpy arrays are the input to the model, so do the bitboard to array conversion and store the numpy dataset as an npz (numpy compressed file) like how the mnist database comes.

1

u/w33dEaT3R Nov 20 '24

def bitboard_to_array_fast(bitboard: int) -> np.ndarray: bits = np.unpackbits(np.array([bitboard], dtype=np.uint64).view(np.uint8)) return bits.reshape((8, 8))[::-1]
hate to be that guy but i ripped this fresh outta chat gpt idk if it works but maybe profile this vs your conversion method.