I picked up Keras, so the code for the network itself is simple. Complexity is around: dataset preparation, evaluation and model selection.
Initially I worked on my laptop with an extremely small values, for example, images were resized to 128 by 128, which feels too small. However, I managed to get reasonable performance numbers.
When I moved my code to a much more powerful machine with lots of memory, cpus and gpus, I got a GPU enabled TemsorFlow binary. That speed up the things, but still CPU was a bottleneck.
At that point TemsorFlow was complaining that the CPU supports instructions that the binary was not built with. Before building TF from source, I decided to update cuda and other Nvidia software. After a dance following outdated tutorials, I got it done. Speed improved, but still, while CPUs were at 100%, GPUs were barely loaded.
The server crashed shortly after I left, so no great results.
vmtouch is a nice tool to cache file content into memory.
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!