Open-weight & owned stack · Training tuning
Developer Updates Small Model With Engram Techniques
A developer reported adding an Engram table, sparse-window attention, Attention Residuals, and compressed frozen components to a 2.6B model.
Read the original at reddit.comOpens the publisher's site in a new tab