New ask Hacker News story: Ask HN: Release Path for 'Transformers Alternatives'?
Ask HN: Release Path for 'Transformers Alternatives'? 2 by adinhitlore | 0 comments on Hacker News. So, a side project I've spent/wasted ~1000 hours on, with 2 goals set in mind: 1. faster than transformers on CPU; 2. smarter than transformers. couple of screenshots below (the black/red part are censored on purpose...for now): https://ift.tt/XCRrD2q https://ift.tt/m4M6xHC https://ift.tt/ku2pTXF Summary: what the hell is this? Two architectures - 1. Linear RNN which solves the long memory problem in current front-runner RNN transformer alternatives (RWKV, Mamba), in addition to being cpu friendly and entirely in C from scratch, but not too big: ~4000 lines. 2. 2 SNN experimental programs (in C originally but also ported to C# and F#) that turned out to be better than expected but unfortunately for the time being: dumber than the linear RNN one (i need more tests). The question is: what to do with them? google gemini pro 3.1/sonnet 4.6 told me to patent, IP, estimating val...