{"id":"0GS2KO60HSk","title":"Token Processing: Overcoming Quantization Challenges","description":"The complexities of quantization in machine learning model inference. strategies to enhance token processing efficiency #MachineLearning #TokenProcessing #QuantizationChallenges #DataHandling #ModelInference #AI #TechInsights #MachineLearningTips #DataScience #OptimizationStrategies #NEARAI","channel_name":"NEAR Protocol","published_at":"2024-09-23T00:00:00.000Z","thumbnail_url":"https://img.youtube.com/vi/0GS2KO60HSk/hqdefault.jpg","transcript_text":"let's see let's see why is challenging and what can be done about it so let's zoom in into single Math mode right so let's say we look into this one right so we have Vector X here this is the model and usually we don't look at a single Vector we look at a bunch of them uh so there's your whole context window of data right so if you doing inference you you fit the prompt let's say the prompt is already 100 tokens so this is your 100 tokens your prompt uh we call it uh let","transcript_synced_at":"2026-11-03T05:32:34.516Z","segments":[{"start_seconds":0,"text":"let's see let's see why is challenging and what can be done about it so let's zoom in into single Math","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=0s"},{"start_seconds":9,"text":"mode right so let's say we look into this one right so we have Vector X here this","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=9s"},{"start_seconds":17,"text":"is the model and usually we don't look at a single Vector we look at a bunch of them","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=17s"},{"start_seconds":24,"text":"uh so there's your whole context window of data right so if you doing inference you you fit the prompt let's say the","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=24s"},{"start_seconds":33,"text":"prompt is already 100 tokens so this is your 100 tokens your prompt uh we call","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=33s"},{"start_seconds":39,"text":"it uh let","url":"https://www.youtube.com/watch?v=0GS2KO60HSk&t=39s"}],"youtube_url":"https://youtube.com/watch?v=0GS2KO60HSk","url":"https://near-pedia.xyz/videos/0GS2KO60HSk"}