Embedding

An embedding is a dense vector of numbers (typically hundreds to thousands of dimensions) that represents a piece of data in a continuous space. The key property of a good embedding space is that semantically similar items are located close together: the embeddings for 'dog' and 'puppy' will be nearby, while 'dog' and 'interest rate' will be far apart.

Embeddings are learned during model training. In a language model, the embedding layer maps each token in the vocabulary to a vector. As the model trains, these vectors are adjusted so that tokens that appear in similar contexts end up with similar embeddings. This geometry captures semantic and syntactic relationships: embeddings encode analogies (king is to queen as man is to woman), subcategory relationships, and topical clusters.

Embeddings are used extensively in retrieval augmented generation (RAG), where documents are indexed by their embeddings and retrieved by computing similarity between the query embedding and document embeddings. This enables semantic search, finding documents that are conceptually relevant even if they do not share keywords with the query.

Beyond token embeddings, sentence embeddings and document embeddings (computed by aggregating token embeddings or using a specialized encoder) allow the similarity of longer texts to be compared. These are used in clustering, classification, de-duplication, and recommendation systems.

For AI evaluation platforms, embeddings power the job matching and profile recommendation features that surface relevant opportunities. Your stated skills and experience are embedded and matched against job requirement embeddings to identify the roles best aligned with your profile.