[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f3lqmcmxi7o2tm":3},{"_id":4,"slug":5,"title":6,"subtitle":7,"kind":8,"cards":9,"tags":57,"categories":58,"source":59,"lang":62,"author":63,"audioState":66,"stats":67,"publishedAt":70,"renderer":71},"6abc80bfca21c797c7e9fef9","language-models-for-text-classification-from-bag-of-words-to-9ccfb6a6","Language Models for Text Classification: From Bag-of-Words to Jev","A Visual Guide to Bag-of-Words, RNNs, CNNs, Transformers, Jev-like APIs, and Calibration The recently released Jev AI model has been quite a cultural phenomenon in technical communities in the past 2 weeks.","news",[10,12,17,22,27,32,37,42,47,52],{"headline":6,"body":7,"imageUrl":11,"sourceImageUrl":11},"https:\u002F\u002Fsubstackcdn.com\u002Fimage\u002Ffetch\u002F$s_!jZ4X!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto\u002Fhttps%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0015e77a-5140-4139-b361-78e28d18df17_2494x1304.png",{"headline":13,"body":14,"imageUrl":15,"images":16},"While Jev aims to classify things, it’s easy","While Jev aims to classify things, it’s easy to dismiss Jev as “just a classifier,” and my own view of Jev has evolved quite a bit over the past few days. In particular, my thoughts went from “classifiers used to be my bread & butter; I can easily build this myself” (more on this later) to “wow, this actually works better than I thought.”","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F1.webp",{"local":15},{"headline":18,"body":19,"imageUrl":20,"images":21},"Sure, the latest state-of-the-art GPT and open-weight LLMs","Sure, the latest state-of-the-art GPT and open-weight LLMs can do the same kinds of classification tasks as Jev, while also being capable of much more general decision-making. But Jev’s advantage is that it can handle those classification tasks much faster and more cheaply.","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F2.webp",{"local":20},{"headline":23,"body":24,"imageUrl":25,"images":26},"At the other end of the spectrum, for","At the other end of the spectrum, for a narrow, well-defined problem, Jev probably won’t classify anything better, faster, or cheaper than a special-purpose classifier. But its selling point is that it is far more general than those task-specific models.","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F3.webp",{"local":25},{"headline":28,"body":29,"imageUrl":30,"images":31},"So, what is the methodology behind Jev (based","So, what is the methodology behind Jev (based on an educated guess), what can it do, and why is it so popular? I aim to answer all of these later in this article. However, I thought starting with a brief history of language models for decision-making would be a great way to begin. And it hopefully helps demystify some of the hype and show what Jev does very well (”Jev is essentially a text classifier,” but “Jev is also not ‘just’ a text classifier.”)","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F4.webp",{"local":30},{"headline":33,"body":34,"imageUrl":35,"images":36},"PS: I am not affiliated with Jev in","PS: I am not affiliated with Jev in any way. Also, I am not offered free access to Jev, and this is also not a product endorsement, just a technical article to offer some insights into the history of text classification to help you make sense of the recent hype.","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F5.webp",{"local":35},{"headline":38,"body":39,"imageUrl":40,"images":41},"Since this is a long article, I recommend","Since this is a long article, I recommend reading it in your browser, where you can access the table of contents menu on the left side. 1. Language modeling and classification in the pre-transformer era","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F6.webp",{"local":40},{"headline":43,"body":44,"imageUrl":45,"images":46},"For completeness, before we put Jev in context","For completeness, before we put Jev in context (no pun intended), I thought it made the most sense to start chronologically. In this section, I want to take a brief tour of applied text classification via naive Bayes, logistic regression, and the more classic (deep) neural networks before transformer-based models came along. 1.1 Bag-of-words: naive Bayes, logistic regression, and XGBoost","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F7.webp",{"local":45},{"headline":48,"body":49,"imageUrl":50,"images":51},"Back in the day, when I was a","Back in the day, when I was a grad student 15 years ago, even though recurrent neural networks already existed (more on that later), text classification was usually done with a bag-of-words representation because it was straightforward and could get good results on moderately sized datasets.","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F8.webp",{"local":50},{"headline":53,"body":54,"imageUrl":55,"images":56},"In short, we can think of the bag-of-words","In short, we can think of the bag-of-words representation as a method that makes free-form text input of different lengths compatible with classic classifiers (naive Bayes, Logistic Regression, SVMs, Random Forest, XGBoost, to name a few), which expect a fixed-size input vector.","\u002Fapi\u002Fmedia\u002Fposts\u002Flanguage-models-for-text-classification-from-bag-of-words-to-9ccfb6a6\u002F9.webp",{"local":55},[],[],{"name":60,"url":61},"Hacker News","https:\u002F\u002Fmagazine.sebastianraschka.com\u002Fp\u002Fclassifier-history-and-jev","en",{"handle":64,"displayName":65},"spots","Spots","queued",{"views":68,"likes":69,"saves":69,"shares":69,"completions":69,"opens":69,"skips":69,"depthSum":69},1,0,"2026-09-30T03:23:43.969Z","local"]