{"id":"fiju3_n8zhw","title":"Transformers Era Over?, Attention-Free Models Rival 14B Transformers, Forward-Deployed AI Engineers","description":"Join the NEAR Legion at https://nearlegion.com/\n\nFollow us on X:   / nearlegion  \n\n#NEAR #nearprotocol #NEARAI #ai #ainews","channel_name":"NEAR Protocol","published_at":"2025-11-03T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/fiju3_n8zhw/hqdefault.jpg","transcript_text":"Good morning Legion and Happy Monday. Last week was a busy one. The NEAR digital asset treasury play that halving vote passing and the announcement of the Zcash hackathon that NEAR is sponsoring and Illia will be judging at all broke news this week. We're expecting some announcements on the J1 races, with that game showing steady development from Mister.. We're also tentatively confident in the release of the Degen Arena in the coming two weeks. A nice surprise for degens on NEAR that will be closely connected to Vanguard missions. If you haven't seen the latest update to the Legion Gitbook, take a look. There are rewards listed and more to come as we continue to build out the vision of the NEAR Legion. As the world's premier decentralized AI community, focused on NEAR technology and the promise of open ASI on the NEAR blockchain, This brings us to our first story today. Are we at the end of the Transformers era? Smarter training in smaller models is making it look more and more that a future of fine tuning and specialized open source models is coming, and that maybe spending so much money on compute only makes sense If you want to power high amounts of inference. Jacob Buckman confirms this by detailing how his team trained a completely attention free foundation model at the 14 billion scale for only $4,000. The performance matches other models of similar scale, including transformers and hybrid models. AI is changing fast. Last up today a nice article from the Financial Times on the hottest AI job the forward deployed engineer. Forward deployed engineers are consultant style engineers building out custom AI solutions inside of other companies. In other words, these engineers take an existing law firm or financial services company and find a way to build an AI solution for their workflow or problem set. It's an interesting note showing that AI is maturing. And yes, it is likely coming for your job. Heads up for everyone waiting to complete your ascendant missions. Another mailbag starts tomorrow with the answers coming on Friday. Submit your question on NEAR the recent DAT strategy and all things NEAR ecosystem. We'll do this all again real soon.","transcript_synced_at":"2026-11-03T04:48:26.174Z","segments":[{"start_seconds":0,"text":"Good morning Legion and Happy Monday. Last week was a busy one.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=0s"},{"start_seconds":4,"text":"The NEAR digital asset treasury play that halving vote passing and the announcement of the Zcash hackathon that NEAR is sponsoring and Illia will be judging at all broke news this week.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=4s"},{"start_seconds":14,"text":"We're expecting some announcements on the J1 races, with that game showing steady development from Mister..","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=14s"},{"start_seconds":20,"text":"We're also tentatively confident in the release of the Degen Arena in the coming two weeks.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=20s"},{"start_seconds":24,"text":"A nice surprise for degens on NEAR that will be closely connected to Vanguard missions.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=24s"},{"start_seconds":29,"text":"If you haven't seen the latest update to the Legion Gitbook, take a look.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=29s"},{"start_seconds":32,"text":"There are rewards listed and more to come as we continue to build out the vision of the NEAR Legion.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=32s"},{"start_seconds":37,"text":"As the world's premier decentralized AI community, focused on NEAR technology and the promise of open ASI on the NEAR blockchain, This brings us to our first story today.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=37s"},{"start_seconds":48,"text":"Are we at the end of the Transformers era?","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=48s"},{"start_seconds":51,"text":"Smarter training in smaller models is making it look more and more that a future of fine tuning and specialized open source models is coming,","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=51s"},{"start_seconds":59,"text":"and that maybe spending so much money on compute only makes sense If you want to power high amounts of inference.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=59s"},{"start_seconds":65,"text":"Jacob Buckman confirms this by detailing how his team trained a completely attention free foundation model at the 14 billion scale for only $4,000.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=65s"},{"start_seconds":74,"text":"The performance matches other models of similar scale, including transformers and hybrid models. AI is changing fast.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=74s"},{"start_seconds":81,"text":"Last up today a nice article from the Financial Times on the hottest AI job the forward deployed engineer.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=81s"},{"start_seconds":88,"text":"Forward deployed engineers are consultant style engineers building out custom AI solutions inside of other companies.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=88s"},{"start_seconds":94,"text":"In other words, these engineers take an existing law firm or financial services company and find a way to build an AI solution for their workflow or problem set.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=94s"},{"start_seconds":103,"text":"It's an interesting note showing that AI is maturing.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=103s"},{"start_seconds":106,"text":"And yes, it is likely coming for your job.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=106s"},{"start_seconds":109,"text":"Heads up for everyone waiting to complete your ascendant missions.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=109s"},{"start_seconds":112,"text":"Another mailbag starts tomorrow with the answers coming on Friday.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=112s"},{"start_seconds":116,"text":"Submit your question on NEAR the recent DAT strategy and all things NEAR ecosystem. We'll do this all again real soon.","url":"https://www.youtube.com/watch?v=fiju3_n8zhw&t=116s"}],"youtube_url":"https://youtube.com/watch?v=fiju3_n8zhw","url":"https://near-pedia.xyz/videos/fiju3_n8zhw"}