New ask Hacker News story: Ask HN: Relatively SoTA LLM Agents from Scratch?
Ask HN: Relatively SoTA LLM Agents from Scratch?
2 by solsane | 0 comments on Hacker News.
As we know, OpenAI is not so open. In 2023, I was playing with transformers, RNNs and I had an understanding how it worked from top to bottom (e.g. made my own keras, could whiteboard small nets) and I can throw things together in keras or tf pretty quick I got a job and never touched that again. Data and compute notwithstanding, how hard would it be to make a pet project foundation model using the latest techniques? I’ve heard about MoE, things like that and I figure we’re not just throwing a bunch of layers and dropout in Keras anymore.
2 by solsane | 0 comments on Hacker News.
As we know, OpenAI is not so open. In 2023, I was playing with transformers, RNNs and I had an understanding how it worked from top to bottom (e.g. made my own keras, could whiteboard small nets) and I can throw things together in keras or tf pretty quick I got a job and never touched that again. Data and compute notwithstanding, how hard would it be to make a pet project foundation model using the latest techniques? I’ve heard about MoE, things like that and I figure we’re not just throwing a bunch of layers and dropout in Keras anymore.
Comments
Post a Comment