Nonlinear regression models are powerful tools for capturing complex relationships between variables that linear models cannot adequately describe. However, assessing the goodness-of-fit for nonlinear models can be more challenging than for linear models. This article will explore various methods and metrics used to evaluate the goodness-of-fit for nonlinear models, providing a comprehensive guide for practitioners.

Introduction to Goodness-of-Fit

Goodness-of-fit refers to the degree to which a statistical model accurately represents the underlying data. While linear models have well-established goodness-of-fit measures (e.g., R-squared), nonlinear models present unique challenges due to their intricate mathematical forms. Evaluating nonlinear model fit involves a multi-faceted approach, encompassing both visual and numerical assessments.

A non-linear model is the type of model that explains the non-linear relationship between the experimental data. The model can be single-variate or multivariate depending on the number of dependent variables. They are usually assumed to be parametric where the model describes the non-linear equation.

• These models don’t assume a linear relationship between the dependent and independent variable rather they analyze the complex relationship in the model by making curves, exponentials, and other non-linear functions.
• Examples of non-linear models are decision trees, logistic regression, Support vector machines, and neural networks like CNN, ANN, etc. These models are usually used in image processing and in fields of physics and biology where the linear model won’t be enough to capture the complex relationship between the variables.

Visual Assessment

1. Residual Plots: Residuals are the differences between predicted values and actual observed values. Plotting residuals against predicted values or independent variables can reveal patterns indicative of model misspecification. Ideally, residuals should be randomly scattered around zero, with no discernible trends.
2. Fitted Value vs. Observed Value Plots: Plotting predicted values against observed values helps visualize the model’s overall accuracy. A well-fitting model should have points clustered closely around a 45-degree line.

Limitations of R-Squared for Nonlinear Models

R-squared, a widely used measure of goodness-of-fit, is based on the proportion of variance in the dependent variable explained by the independent variables. However, for nonlinear models, R-squared can be misleading and may not accurately reflect the model’s performance. This is because R-squared is not defined for nonlinear models and can produce values outside the [0,1] interval, making it difficult to interpret.

Different Measures of Goodness-of-Fit for Nonlinear Model

Several alternative measures have been proposed to evaluate the goodness-of-fit of nonlinear models. These include:

1. Bayesian Information Criterion:

The BIC often abbreviated as Bayesian Information Criterion is a measure used in model selection in a finite set of models. It is a goodness-of-fit test that helps balance a model’s complexity and its goodness-of-fit to the data. It allows researchers to choose the appropriate model and encourages the selection of simpler models.

The value of the Bayesian information criterion cannot be negative as it is derived from the likelihood function and includes a penalty based on the number of parameters. An example of BIC application is in placental dysfunction.

klog(n)-2log(L(θ))

Where n is the sample size, k is the number of parameters that the model estimated and L(θ) is the maximized value of the likelihood of the model

2. Cramer-von Mises Criterion

The CVM often abbreviated as the Cramer-von Mises Criterion is another goodness-of-fit test used to explain how well the set of observed data fits the hypothesized probability distribution. It has various applications in economics, engineering, and finance. It tests based on the cumulative distribution function of the hypothesized distribution and set of observed data. It is evaluated by comparing the observed data’s empirical cumulative distribution function and the hypothesized distribution’s theoretical cumulative distribution function.

The Cramer-von Mises has two types of tests for different numbers of samples given below,

Cramer-von Mises test for one sample:

Consider, there are ‘n’ number of observed values from x₁, x₂, x₃, …., x_n in the increasing order then the formula will be,

T = nω2 = (1/12n) + ∑[((2i-1)/2n)-F(xi)]2

Where n is the number of values, F(x_i) is the cumulative distribution function at x_i

Cramer-von Mises test for Two samples:

We will consider two sets of observed values for the following: x_1, x₂, x₃, …., x_N and y₁, y₂, y₃, …., y_M for the first and second samples respectively. Let the ranks be r_1, r₂, r₃, …, r_N for the first sample, and we consider r₁, r_2, r_{3, ….,} r_{M for the second sample.}

T = ((NM)/(N+M))ω2 = (U)/(NM(N+M))-((4MN-1)/(6(M+N)))

Where the formula for U is

U = N∑(ri-i)2 + M∑(sj-j)2 

3. Akaike Information Criterion:

The AIC often abbreviated as the Akaike Information Criterion measures the quality of the statistical model for a given set of data. It helps provide a trade-off between the goodness-of-fit of the model and its complexity. The Akaike information criterion is based on the information theory. It measures how much amount of information was lost by a model when it is used to estimate the true underlying distribution of the data.

The AIC method can be used when we want to figure out how drink influences body weight where the survey will contain multiple variables like age, gender, and others, etc. You can create and compare the possible models using this AIC method.

The formula the for Akaike Information Criterion is given below,

AIC = 2K - 2In(L)

Where

K= number of independent variables

L= log-likelihood estimate

4. Residual Analysis

The residual analysis is used to examine the differences between the observed values and predicted values. It is a technique used to assess the goodness-of-fit of a non-linear model. The residual refers to the data in the validation dataset that was not represented or explained by the model. It shows the differences between the observed data with its corresponding predicted value and also helps to identify the patterns that the model didn’t capture.

Assessing Goodness-of-Fit for Non-linear Models : Practical Implementation

The goal is to analyze the results of the non-linear model has the following steps, We need to determine how well the model fits the data how uncertain the values of parameters are, and how the equation fits data differently for different sets of data.

Initially, We need to assess how well the model fits the data by graphing a curve for the data and plotting the data points. If the algorithm worked correctly then the distance between the curve and data points appears to be minimal.

We plot the points in a residual plot where the curve is on the y-axis and the corresponding independent values are on the x-axis.

• A model is said to have a good fit if residuals are randomly distributed across the zero line.
• A model is said to not have a good fit if residuals represent a pattern like sinusoid or long runs of the same sign.

Implementation Using Python:

import numpy as np
from scipy.optimize import curve_fit 
import matplotlib.pyplot as plt 

#Define a Non-Linear model function
def non_linear_model(x, a, b, c):
    return a*np.exp(-b*x)+c

np.random.seed(0)

#Generate the synthetic data 
x_data= np.linspace(0, 4, 50)
y_data= non_linear_model(x_data, 2.5, 1.3, 0.5)+0.2*np.random.normal(size=len(x_data))

#Fitting of Non-Linear model to the generated data 
Optimal_param, covariance = curve_fit(non_linear_model, x_data, y_data)

#Making predictions using the Fitting model 
y_pred=non_linear_model(x_data, *Optimal_param)

#Calculate the sum of squares of residual
ss_res=np.sum((y_data-y_pred)**2) 

#Calculate the total sum of squares
ss_tot=np.sum((y_data-np.mean(y_data))**2)

#Calculate the R-Squared using the formula we discussed earlier 
r_squared= 1-(ss_res/ss_tot)

#Printing the R-squared score 
print("The R_Squared score for the model is",r_squared)

#Plotting the data 
plt.scatter(x_data, y_data, label='Data')
plt.plot(x_data, y_pred, color='green', label='Goodness-of-fit Model')
plt.xlabel("X-axis")
plt.ylabel("Y-axis")
plt.title("Non-Linear Model Fitting")
plt.legend()
plt.show()

Output:

The R_Squared score for the model is 0.923188676710105

The Goodness-of-fit Model

Considerations for Nonlinear Models

1. Model Complexity: Nonlinear models can be highly flexible, potentially leading to overfitting. Overfitting occurs when a model captures noise in the data rather than the true underlying relationship. Regularization techniques like ridge regression or LASSO can help mitigate overfitting.
2. Outliers: Outliers can unduly influence the fit of nonlinear models. Robust regression methods, which are less sensitive to outliers, may be appropriate in such cases.
3. Non-Normal Errors: Traditional goodness-of-fit measures often assume normally distributed errors. If errors are non-normal, alternative measures or transformations may be necessary.
4. Multiple Optima: Nonlinear optimization algorithms can converge to local optima rather than the global optimum. It’s crucial to explore multiple starting points or use global optimization techniques to ensure the best possible fit.

Conclusion

Assessing the goodness-of-fit of nonlinear models requires a combination of visual inspection and numerical measures. By carefully considering the unique characteristics of nonlinear models and employing appropriate techniques, researchers and analysts can confidently evaluate model performance and make informed decisions based on their findings.