Jul 21, 2023
Hi Sanjay, thanks for the question. Sentence-transformers offer a host of embeddings models, and their performances can be found here in the MTEB leaderboard: https://huggingface.co/spaces/mteb/leaderboard. I chose 'all-MiniLM-L6-v2' mainly because it is simple, lightweight, fast, general-purpose, and has decent performance. The aim of this project was to focus on the inference aspect, so it was not to really optimize the embeddings component.