High-dimensional embedding vectors are fundamental building blocks in Machine Learning, particularly in transformers or word2vec. Typically, two vectors that are semantically similar point in roughly the same direction; if they are entirely dissimilar, they point in opposite directions; and if they’re nearly orthogonal, they are unrelated.
We usually think in two or three dimensions, but there are some unintuitive properties that only apply in higher dimensions. For example, two random vectors are expected to be near orthogonal in high dimensions. Intuitively, it makes sense for word2vec, as we expect that two words are unrelated in most instances.
I prepared an interactive post that explains why two random vectors are expected to be nearly orthogonal in high dimensions: https://maitbayev.github.io/posts/random-two-vectors/.
Here a random preview: