10 — Understanding Word2Vec 1: Word Embedding in NLP
In this article, we will talk about Word2vec one of the word embedding techniques. Before we start, I recommend you read the article I have previously explained on Word Embedding.
After reading word embedding article, now that you have a basic understanding of this topic, let us start Word2vec.
Word2Vec technique was developed by Google in 2013. It is a neural network with one input layer, one output layer and one hidden layer. When creating word vectors, it uses hyperparameters like window size and embedding size. Window size shows how many words to the left and right of the target word should be considered. Embedding size shows how many dimensions each word vector will have. Embedding size is also the number of neurons in the hidden layer.
The Word2Vec method is different from the One Hot Encoding method I explained in my previous article. Word2Vec creates word vectors by training data without labels using neural networks. Unlike other methods, the size of the vector is not as large as the number of unique words in the text.
For example, if there are 300,000 unique words in a big text, One Hot Encoding creates a vector with 300,000 elements for each word, with only one element being 1 and the rest being 0. But with Word2Vec, the vector size can be 300 (depending on the size of the text and the type of project, more or less may be selected, but usually between 100–300), avoiding unnecessary large vectors. This is very useful for big data.
Why we need Word2Vec?
Word2Vec is a popular and significant method for representing words as vectors in NLP.
- Similarity in Meaning: Word2Vec translates words into numbers, and words with similar meanings get similar numbers. For example, “king” and “queen” take similar numbers.
- Efficient Calculation: Word2Vec turns words into smaller, more useful numbers. It is easier to work with these numbers.
- Understanding Relationships: Word2Vec learns relationships between words. For example, the difference between “king” and “queen” is similar to the difference between “man” and “woman”.
- Working with Big Data: Word2Vec can work with large texts and extract meaningful information from these texts.
Word2Vec can still be used sometimes, but today, there are better word embedding methods for understanding context and doing complex NLP tasks. It is important to choose the right method based on your project needs, data set, and computing power.
There are two different Word2Vec approaches, Continuous Bag of Words (CBOW) and Skip-Gram.
To avoid making this article long, I will explain these two approaches with an example in my next article.
Conclusion
In conclusion, Word2Vec is a widely used method in NLP. Word2Vec uses a neural network to learn word embeddings by predicting the context of a given word. Word2Vec aims to capture semantic relationships between words based on their co-occurrence patterns in a large corpus of text data.
