Interpretable support vector regression

As universal approximator for any multivariate function; support vector machine model is widely-used to capture highly nonlinear, unknown or partially known complex systems plants or processes. However, it should be pointed out that simplicity and complexity of this type of models are challenging tasks to perform, since these black box systems don’t give any insight to the model structure. Hence, incorporating robustness of support vector models with the transparency and interpretability of fuzzy models is a straightforward directional development step, confirmed by the fact that connection between support vector regression and fuzzy regression has been widely and extensively researched lately. Numerous techniques were reported to exploit the above listed synergies. Nowadays complexity of regression problems is increasingly growing, therefore it is necessary to deal with structural issues and linguistic interpretability in order to capture more information from nature of the modeled system, data or process.

Fuzzy logic helps to improve interpretability of knowledge based regression models through its semantics that provide insight in the model structure and decision making process. Application of support vector methods for the initialization of fuzzy models is not a completely new idea. Numerous methods have been proposed to build the connection between the SVR and the FIS. Chen and Wang [3, 4] propose a positive definite fuzzy system (PDFS). In the proposed fuzzy model, the PDFS is equivalent to a Gaussian-kernel SVM [3] if Gaussian membership functions are adopted. Antecedent of a fuzzy rule is obtained by a support vector (SV). Therefore the number of fuzzy rules is the same as the number of SVs. As the number of SVs is generally large, the size of the FIS based on an SVM is also large. To solve this problem researcher [5] proposed a learning algorithm to remove the irrelevant fuzzy rules. In spite of this, the generalization performance is degraded. The above methods are for the zero order FIS, which has one fuzzy singleton in the consequent of a fuzzy rule.

For the first order FIS, Leski [18] described a method for obtaining a FIS by means of the SVM with data-independent kernel matrix. Moreover Juang et al. used a combination of fuzzy clustering and the linear SVM to establish a fuzzy model with less parameter number and better generalization performance. However, negligible effort has been done to establish a HFIS (high order FIS) with kernel methods. It was presented a HFIS with high accuracy and good generalization performance [11]. It was shown how to obtain the formulation of the nonlinear function for the consequent part. Furthermore, Catala used prototype vectors to combine with the support vectors using geometric methods to define ellipsoids in the input space, which are later transformed to if then rules [2].

Special operator was utilized to achieve equivalency between support vector machines and fuzzy rule based system [6]. Utilization of support vector models is described to solve the convex optimization problem for multivariate linear regression models and it is also shown how multivariate fuzzy nonlinear regression model can be formalized for numerical inputs and fuzzy output [8]. Multiple types of kernels [6, 8] can be used to solve crisp nonlinear regression problems [1]. Juang and Hsieh [7] used a combination of fuzzy clustering and linear support vector regression to obtain Takagi-Sugeno type fuzzy rules. Support vector machines can be applied to determine the support vectors for each fuzzy cluster obtained by fuzzy c-means clustering algorithm [16]. Visualization of fuzzy regression models is also discussed lately. Interpretation of fuzzy regression is provided with an insight into regression intervals so that regression interval analysis, data type analysis and variable selections are analytically performed [15].

A visualization and interpretation tool is presented. Feature space is visualized with highlighting the corresponding variables in the original input data to show how they are associated to the output variable [17]. It is shown that which part of the input data can be utilized to estimate the output value. This technique also describes which input variable are responsible for the performance of the support vector regression. With the combination of visualization and interpretation the black-box support vector regression is identified in one step.

For full text: click here

(Author: Tamás Kenesei, János Abonyi

Published by Sciedu Press)