The world of Machine Learning Engineering (MLE) is undergoing a rapid transformation with the emergence of new foundation models across all modalities. While it may seem overwhelming, rest assured that these advancements will ultimately simplify our work and make it more efficient.

My journey into the realm of ML began during the rise of Deep Learning. I immersed myself in learning Theano on deeplearning.net, and during that time, AlexNet reigned as the ultimate model for ImageNet. However, it was an interview at ElementAI that completely revolutionized my perspective on ML.

TLDR: Our expertise lies in using the right tool that brings value to your users, whether it’s an LLM or a regex.

During the interview, I was tasked with developing a system for GIF categorization for a well-known company that rhymes with Jiphy. My initial approach involved computing embeddings for all frames and using clustering techniques to assign categories to the GIFs. While this solution was deemed superior to those proposed by other interviewees, who wanted to train models on a limited set of predetermined categories, I realized that simplicity could be our ally.

After careful consideration, I decided to focus on the tags associated with the GIFs. Most GIFs come with hashtags that describe their content. For example, if a GIF has the hashtag #funny, the odds are higher that it is a funny GIF. This simple insight turned out to be the real solution.

This experience taught me a valuable lesson: as MLE professionals, our ultimate goal is to deliver features to customers as quickly as possible. It is better to have an okay-ish version that works reasonably well than to spend months developing a model that may never be used at all.

So what?

Now, let’s circle back to LLMs. Of course, we will continue to employ techniques such as KNNs, Sentence Embeddings, and trained classifiers. However, if we can create a semi-reliable prompt using an LLM in just one afternoon, it becomes the ideal way of showcasing new features to our users. This approach allows us to iterate more frequently and gain a better understanding of our customers’ needs.

At Glowstick, our ML pipeline is a fusion of various components, including Similarity search with SBERT, trained classifiers, regexes, and LLMs. We leverage what makes the most sense at any given time. In a startup environment, it’s highly likely that we will discard models along the way, but that’s just part of the journey. :)

In the upcoming weeks, I will share examples of simple cases where we opted to use LLMs instead of training our own models. I will delve into our rationale and discuss the tradeoffs we observed. And of course, I’ll include exciting prompts to illustrate our findings! Let me know in the comments if interests you!

Stay tuned for more intriguing insights!