In Natural Language Processing (NLP), understanding the relationships between words is crucial for various applications, such as text analysis, information retrieval, and machine learning. The co-occurrence matrix is one of the fundamental tools used to capture these relationships.

This article delves into the concept of the co-occurrence matrix, its construction, significance, and applications in NLP.

What is a Co-occurrence Matrix?

A co-occurrence matrix is a mathematical representation that captures the frequency with which pairs of words appear together within a specified context, such as a sentence, paragraph, or document. It is a square matrix where rows and columns represent unique words in the corpus, and each cell (i, j) contains the number of times word i appears in the context of word j.

Given a vocabulary of N unique words, a co-occurrence matrix C is an N x N matrix, where:

[Tex]C[i][j][/Tex] = the number of times word j appears in the context of word i.

Constructing a Co-occurrence Matrix

To construct a co-occurence matrix, we are going to use following steps:

Step 1: Import Necessary Libraries

First, we need to import the required libraries, including nltk for text preprocessing and pandas for creating a DataFrame.

import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
from collections import defaultdict, Counter
import numpy as np
import pandas as pd

# Download NLTK resources
nltk.download('punkt')
nltk.download('stopwords')

Step 2: Define Sample Text

We define a sample text that we will use to create the co-occurrence matrix.

# Sample text
text = """Apple is looking at buying U.K. startup for $1 billion.
The deal is expected to close by January 2022. Apple is very optimistic about the acquisition."""

Step 3: Preprocess the Text

In this step, we preprocess the text by converting it to lowercase, tokenizing it, removing stop words, and filtering out non-alphanumeric tokens.

# Preprocess the text
stop_words = set(stopwords.words('english'))
words = word_tokenize(text.lower())
words = [word for word in words if word.isalnum() and word not in stop_words]

Step 4: Define Window Size and Create Co-occurrence Pairs

We define the context window size and create a list of co-occurring word pairs within this window.

# Define the window size for co-occurrence
window_size = 2

# Create a list of co-occurring word pairs
co_occurrences = defaultdict(Counter)
for i, word in enumerate(words):
for j in range(max(0, i - window_size), min(len(words), i + window_size + 1)):
if i != j:
co_occurrences[word][words[j]] += 1

Step 5: Create List of Unique Words

We extract a list of unique words from the preprocessed text.

# Create a list of unique words
unique_words = list(set(words))

Step 6: Initialize and Populate the Co-occurrence Matrix

We initialize the co-occurrence matrix and populate it using the co-occurrence counts.

# Initialize the co-occurrence matrix
co_matrix = np.zeros((len(unique_words), len(unique_words)), dtype=int)

# Populate the co-occurrence matrix
word_index = {word: idx for idx, word in enumerate(unique_words)}
for word, neighbors in co_occurrences.items():
for neighbor, count in neighbors.items():
co_matrix[word_index[word]][word_index[neighbor]] = count

Step 7: Create a DataFrame for Better Readability

We create a DataFrame from the co-occurrence matrix for better readability and display it.

# Create a DataFrame for better readability
co_matrix_df = pd.DataFrame(co_matrix, index=unique_words, columns=unique_words)

# Display the co-occurrence matrix
co_matrix_df

Complete Code

Output: