One of the most well-liked and effective machine learning libraries for a range of applications, including regression and classification, is called XGBoost (Extreme Gradient Boosting). Data scientists and machine learning practitioners use it because of its excellent accuracy and capacity to handle massive datasets. One crucial parameter to comprehend in multi-class classification issues is ‘num_classes’. This parameter is essential to determining the target variable’s category count and, consequently, to properly configure the model. This post explores the ‘num_classes’ parameter in XGBoost when utilizing R, outlining its significance and providing practical implementation examples.

Overview of XGBoost in R

The optimized gradient boosting library XGBoost is made to be incredibly effective, adaptable, and portable. It uses the Gradient Boosting framework to implement machine learning algorithms. XGBoost is a flexible tool for a range of predictive modeling applications in R because it can be used for both regression and classification problems.

Key Features of XGBoost:

• Speed and Performance: The model performance and execution speed of XGBoost are well-known. It manages big datasets effectively.
• Regularisation: Prevents overfitting by using L1 (Lasso) and L2 (Ridge) regularisation.
• Tree Pruning: To guarantee that the trees are trimmed correctly, a max-depth parameter is used.
• Processing in Parallel: Facilitates processing in parallel to accelerate computation.
• Cross-Validation: Offers integrated cross-validation features.

Role of ‘num_classes’ Parameter

When performing multi-class classification tasks with XGBoost, the ‘num_classes’ argument is essential. It specifies how many unique classes or categories the target variable contains. This parameter aids in the model’s configuration by establishing the proper output layer structure and objective function to handle multi-class classification.

Why is ‘num_classes’ important ?

For every instance, the output layer of the model has to generate a probability distribution over the various classes. The output layer is guaranteed to have the appropriate number of units in accordance with the number of classes by using the ‘num_classes’ argument.

When to use ‘num_classes’ ?

Multi-class classification problems are those in which there are more than two different classes in the target variable. Examples of these problems include document categorization, species classification (using datasets like the iris), and digit recognition (0–9).

Now we will discuss step by step Implementation of num_classes for xgboost in R Programming Language.

Step 1: Prepare the Data

We will use the iris dataset for this example, as it is a classic example of a multi-class classification problem with three species of flowers.

install.packages("xgboost")
library(xgboost)

# Load the iris dataset
data(iris)

# Convert the Species column to a numeric factor
iris$Species <- as.numeric(as.factor(iris$Species)) - 1

# Split the dataset into features (X) and labels (y)
X <- as.matrix(iris[, -5])
y <- iris$Species

# Create a DMatrix for xgboost
dtrain <- xgb.DMatrix(data = X, label = y)

Step 2: Define Parameters and Train the Model

Set up the parameters for the XGBoost model, including the num_class parameter.

# Set parameters for the xgboost model
params <- list(
  objective = "multi:softmax",
  num_class = 3,  # Number of classes
  eval_metric = "mlogloss",
  max_depth = 3,
  eta = 0.1
)

# Train the model
set.seed(123)
xgb_model <- xgb.train(
  params = params,
  data = dtrain,
  nrounds = 50
)

summary(xgb_model)

Output:

              Length Class              Mode       
handle             1 xgb.Booster.handle externalptr
raw           137047 -none-             raw        
niter              1 -none-             numeric    
call               4 -none-             call       
params             6 -none-             list       
callbacks          1 -none-             list       
feature_names      4 -none-             character  
nfeatures          1 -none-             numeric  

Step 3: Make Predictions and Evaluate the Model

Make predictions on the training data and evaluate the accuracy.

# Make predictions
preds <- predict(xgb_model, dtrain)

# Convert predictions to factors
preds <- as.factor(preds)

# Calculate accuracy
accuracy <- sum(preds == y) / length(y)
print(paste("Accuracy:", round(accuracy * 100, 2), "%"))

Output:

[1] "Accuracy: 78 %"

Conclusion

XGBoost for R’s num_classes option must be understood and used correctly in order to solve multi-class classification issues. The significance of num classes, its function in the model configuration, and thorough illustrations of its use have all been discussed in this article. Furthermore, a thorough grasp of the model’s efficacy can be obtained by assessing its performance using a variety of indicators.

num_classes for xgboost in R-FAQs

What is the purpose of the num_classes parameter in XGBoost ?

The number of unique classes in the target variable is specified by the num_classes option, which is essential for setting up the model for multi-class classification jobs.

How do I install the XGBoost package in R ?

Install.packages(“xgboost”) is the command you can use in R to install XGBoost.

Can XGBoost handle multi-class classification ?

If you set the aim parameter to multi:softprob and provide the number of classes with num_classes, then XGBoost is capable of handling multi-class classification.

What evaluation metrics can be used for multi-class classification in XGBoost ?

For multi-class classification, accuracy, confusion matrix, log loss, precision, recall, F1-score, and AUC-ROC are common evaluation measures.

How do I convert a dataset to DMatrix format in XGBoost ?

Use xgb to convert a dataset to DMatrix format.DMatrix(data = X, label = y), where the target variable is denoted by y and the feature matrix by X.