Larger datasets you want to be using more advanced techniques, FAISS is really good for big big datasets as it means you will not be searching all combinations, just a subset via approximate search (+ many other optimizations)