Custom kernel SVM
Description
Custom Kernel SVM model is a generalization of Support Vector Machines (SVM) models to solve problems that are not linearly separable. It allows the user to select the kernel function.
Sometimes data is organized in such a way that separating it with a straight line is impossible. When data is not linearly separable, a solution is to project the data. Unfortunately, this can sometimes be computationally expensive, but fortunately, only inner products of the data points are needed to calculate the decision barrier for SVM, and this is where a kernel function comes into play.
Kernel functions are a way to solve non-linear problems with the help of linear classifiers.
Mathematically, the definition of the kernel function is:
K(x_i, x_j) = <Φ(x_i), Φ(x_j)>
where Φ is a function that would map the data onto higher dimensions. This is known as the kernel trick method.
The kernel functions are used as parameters in the SVM codes. They help to determine the shape of the hyperplane and decision boundary.
Example of kernels:
-
Linear:
K(x_i, x_j) = x_i * x_j -
Polynomial:
K(x_i, x_j) = (x_i * x_j)dwhere
dis the polynomial degree. -
Sigmoid:
K(x_i, x_j) = tanh(x_i * x_j)
Note
The linear kernel function is used by the Linear SVM model type, so it's not available as a choice for this model type as it would be redundant.
Hyperparameters
The hyperparameters for this model type are:
- Custom kernel type to be applied
- Inverse of regularization strength
- Degree of polynomial for polynomial kernel
- Stop condition tolerance
- Class weight