DeanoC thoughts and musings

The need for speed Coming from a real-time world (games and graphics), current machine learning training is a shock. Even small simple training tasks take a  long  time, which gives me plenty of time to think about the performance of my graphs. Currently most of the open source deep learning stacks consist of a C++ and Cuda back-end driven by a Python/R/Lua front end. The Cuda code tends to be fairly generic, which makes sense from an HPC and experimental point of view.  Its important to note HPC stuff is very optimised but tends to really on standard interfaces with relatively custom generic back-ends. For example BLAS is a old FORTRAN standard for linear algebra, it  has numerous back-ends included optimised x64, Cuda, OpenCL etc.  However it only accelerates the tensor multiplies in a few data formats, other more data specific optimisation like overlapping conversions isn't in its remit. Its quite a different optimisation strategy from the rea...

About 5 years ago, I came close to starting a tech company selling archival level storage systems. Part of the idea was an algorithm and technique I developed that efficiently stores large data reliably. I wrote up a patent, even pushed it through the 1st phase of the UK patent system, but then decided not to go ahead. Effectively the not-patent is now open for anyone. Its designed for large scale system, thousands of storage devices and isn't necessarily fast. However for large archival storage is saves significant storage devices over the tradition RAID systems. So if data retention is really important this is an idea that might be worth investigating... The best example of its utility is that a 1000 storage unit system would have higher resilience and more disks used for storage than a RAID71 (the best normal system) system of similar size. Anyway I found one of the drafts so though I'd publish it here (including typos!). I never actually published it though beyond t...