Bayes Wavelet Regression Approach to Solve Problems in Multivariable Calibration Modeling

In the multiple regression modeling, a serious problems would arise if the independent variables a re correlated among each other (the problem of ill conditioned) and the number of observations is much smaller than the number of independent variables (t he problem of singularity). Bayes Regression (BR) is a n approach that can be used to solve the problem of i ll conditioned, but computing constraints will be expe rienced, so pre-processing methods will be necessary in the form of dimensional reduction of independent variables. The results of empirical studies and literature shows that the discrete wavelet transform (WT) gives estimation results of regression model which is better than the other pre processing methods. This experiment will study a combination of BR with WT as pre-processing method t o solve the problems ill conditioned and singularities. One application of calibration in the field of chemistry is relationship modeling between the concentration of active substance as measured by High Performance Liquid Chromatography (HPLC) with Fourier Transform Infrared (FTIR) absorbance spectrum. Spectrum pattern is expected to predict the value of the concentration of active substance. The exploration of Continuum Regression Wavelet Transform (CR-WT), and Partial Least Squares Regression Wavelet Transform (PLS-WT), and Bayes Regression Wavelet Transform (BR-WT) shows that the BR-WT has a good performance. BR-WT is superior than PLS-WT method, and relatively is as good as CR-WT method. KeywordsBayes, wavelet, ill conditioned, singularity


I. INTRODUCTION
n general, the statistical model requires the number of observations greater than the number of independent variables (predictors).On some phenomena (cases) these requirements can't always be fulfilled.One of the reasons is the high cost that is used to obtain the data, so it does not allow for observation of samples in large numbers.Trooped with these problems there are also other problems on the occurrence of the independent variables of high correlation among independent variables.If there is a high correlation between independent variables X, there will present an classical issue called multicollinearity, hence the use of Classical Regression (Ordinary Least Squares Regression, OLS) will lead to ill-conditioned raw alleged error resulting enlarged (over estimate).In other words, multicollinearity can cause a very low accuracy of the estimated parameters [8].Meanwhile, if the number of independent variables is much greater than the number of observations, the structure becomes singular matrix of independent variables (the problem of singularity).This results affect in X T X matrix has no unique inverse (typical), which is the main requirement in OLS.Consequently, a method to solve both of problems is necessarily obtained.
Ill-conditioned and the singularity problem occur in many real problems, for example in the calibration model.Calibration models are widely used in chemistry, especially Chemo metrics that is a field of science that are a combination of mathematics, statistics, and chemistry.
Several methods to solve the problem of ill conditioned and the singularity has been developed.Setiawan and Notodiputro (2007b) developed a method with the combination of Wavelet Transformation Continuum Regression (CR-WT) which resulted in a relatively satisfactory model to solve the problem of ill conditioned and the singularity on various structures of the correlation matrix independent variables.Bayes Regression method is one of alternative to solve the multicollinearity problems.This paper will discuss about how the performance of a combination of Bayes Regression and Wavelet Transform.The focus of this research is to examine the Bayes Regression with pre-processing Wavelet Transform (BR-WT) as one alternative for dealing with the high correlation between independent variables and the number of observations is smaller than the number of independent variables.This is because both problems are common in real problems.This paper is based on the results of research that specifically aims to study and develop Bayes Regression methods with Wavelet Transform to solve the problem of high correlation between independent variables and the number of observations is smaller than the number of independent variables.
Recently, the use of medical plants is not only limited in the making of herbal medicine, but also in pharmacy, supplement products (nutraceuticals), herbal extracts, etc. the quality of the source (medical plants) must be kept in order to keep the quality of the products of the herbal medicine company and of the pharmacy and in order to fulfill the standard (Danutirto, 2001).The information of the use of the medical plants can be observed through the active compounds within them.Therefore, the study about the active compounds is necessary.

A. Multicollinearity
In the classical regression model requires the number of observations greater than the number of independent variables (predictors) and there is no multicolinearity.Multicolinerity is high correlation between its independent variables.One of the methode that can be used to

Bayes Wavelet Regression Approach to Solve Problems in Multivariable Calibration Modeling
Setiawan 1 and Sutikno 1 I identify multicolinearity is value of Varians Inflation Factor (VIF).
Where R 2 j is determination coefficient for Xj that constitute function from other X variable.If the VIF's value is greater than 10 shows multicolinearity.

B. Bayes Regression with Normal Prior
Bayes approach in the regression is done by forming the posterior distribution of parameters.This posterior is the product of priors with the likelihood function.General multiple regression model with k independent variables (including intercept) is: (1) Where y = observation of the dependent variable vector (n x 1) X= observation matrix of independent variables (n x k) β = regression coefficient vector (kx1) ε = random variable vector error (nx1) and In this paper assumed β ~ N (θ, V) where V is a variance-covariance matrix β so symmetrical, then the prior function: The maximum value of the function is obtained by finding the first derivative of β and equated to zero.
where V is variance-covariance matrix β, θ is vector of regression parameters which are approximated by least squares regression coefficient,σ 2 = the middle of square error which is approximated by the middle of square error of the least square regression.
Statistical properties of β ˆand y ˆ : 1. β ˆis a bias estimator for β ( ) Variance and standard deviation for β ˆand y ˆ ( ) 3. Var(ŷ) is expected by the middle of the square error, so that the estimation of ( ) Variance for y ˆ is : The detailed algorithms Bayes approach in the following regression model are: a. Standardization of variables b.Least squares regression method to obtain θ and σ 2 c.Conducting a resampling and regression coefficients assumed for each sample by the least square method to obtain V. d.Compute β ˆ with formula : bagi β i , Var(ŷ), s(ŷ), and interval confidence (1-α) 100% for y i .

C. Discrete Wavelet Transform
Suppose there is a vector data x = (x 0 , x 1 , …, x q-i ) T with q = 2 M , M>0 is an integer.Discrete Wavelet Transform (DWT) defined as follows: j = 0,1,2, …, (M-1) and k = 0,1,…, (2 j -1), thus obtained (q-1) coefficients and one coefficient c 0,0 same matrix dimensions variable X.With matrix notation, DWT in the equation ( 6) can be written d = Bx (7) because B is orthogonal, hence can be written: B T is a matrix that the elements of his column is the value of φ(t) and Ψ j,k (t) for various t ∈ [0, 1].Special properties of the matrix B T is orthogonal, the first column of the same value, and number of other elements of each column equal to zero.
Vector data x can be connected with the function f in the interval [0, 1] and defined as : This function is known with a stair and included in L 2 ([0,1]) so that the wavelet decomposition of f(t) is : called the scale function for Haar wavelet.The equation ( 10) is called discrete wavelet transform, because the value of j is only taken on the positive integers.Numbers of j in the equation ( 10) is called a resolution level, and f(t) can be obtained accurately, if taken by all levels of resolution for the decomposition, i.e. the resolution level 0 up to (M-1).Coefficient c 0,0 called the coefficient of smoothing or part of a function approach, whereas d j,k called wavelet coefficients or also referred to the detail of a function.
For example, suppose there are four observations (q=4, M=2), x = (x 0 , x 1 , x 2 , x 3 ) T , hence can be written as follows: With DWT obtained: )) 4 In matrix notation can be written: B is called a wavelet transform matrix.In order to obtain matrix B T which is ortogonal, then selected φ(t) and Ψ j,k (t) such that : for j= l and k=m not occur simultaneously.

∫
Then matrix B T obtained by multiplying all components φ(t) and Ψ j,k (t) with Suppose for the Haar wavelet: In the case p = 4 then for the Haar wavelet is : Because the dimensions of B very large (p>>), then choose a particular resolution levels such that the number of wavelet coefficients selected for ' p with p'< (n-1) < p and the result will be obtained D * (n x p) = X (n x p) B *T (n x p) (11) which reduces the observation of p vertices of each sample into ' p wavelet coefficients selected point.

D. Bayes Regression with Pre-processing Discrete Wavelet Transformation
Bayes regression with pre-processing Discrete Wavelet Transformation (BR-DWT), is essentially Bayes regression between the response variable Y with independent variable D from DWT results.Matrix D from Wavelet Transformation not guaranteed independent.Sunaryo (2005) show that the new variable result from DWT still have a fairly high correlation, although its value is smaller than the correlation between variables of origin.Therefore, the regression model between the response variable (Y) with variable results from DWT cannot using OLS method because there are still multicollinearity problems.
Data analysis using the BB-WT divided into two phases that are to build a model calibration and validation model.

Phase 1
Build a calibration model with the following steps : a. Search for a wavelet coefficient matrix i. Matrix X (nxp) is the results of discretizing of the infrared spectrum of gingerol is the independent variable, whereas the response variable y (nx1) is the concentration of active compounds of ginger powder from HPLC measurements.Data were divided into two groups: the first group is n 1 data to build models, and the second group is n 2 data for validation.ii.Get DWT matrix B with the involvement of 1024 points (M = 10), which is determined based on the mother wavelet.Wavelet matrix calculations in this study using the software Wavetresh (Nason, 1998).iii.Get the matrix of wavelet coefficients D with the formula D (n1xp) = X (n1xp) B T (pxp) then select a specific resolution levels such that the number of wavelet coefficients selected for p'with p' < (n 1 xp) < p. Subsequently obtained D * (n1xp) = X (n1xp') B *T (pxp') which reduces the observation of p vertices of each sample into p' point wavelet coefficients are selected.In software Wavetresh 3 (Nason, 1998 Validation of the model with the following steps : a. Perform predictive value of y in the group of validation data using the model produced in phase 1, with the following steps: b.Further validating the model with RMSEP criterion, coefficient of determination of prediction results (R predict 2 ).

A. Data
The research is a step in continuous research about tatistics modelling.The aim of the long-term research is to solve the issues of classical assumptions in statistics modelling.The assumption studied here is the issue of multicollinearity, the number of observations which is much less than the number of independent variables, and the presence of outliers.In order to get a suitable model for solving these issues, a method combination is done.
The first step is to study the combination of Bayes Regression and Wavelet Transform (BR-WT) theoretically, which is expected to be an alternative for solving ill-conditioned and singularity problems.
The second step is to study the use of BR-WT method by using simulation data.And the last step is to use BR-WT method in the calibration modeling.
The calibration model in this research is calibration model of the level of gingerol compounds.The sample is gingerol from (a) medical plants farmers in Kulonprogo, Centre Java and Karanganyar, DIY, (b) the experiments in Biofarmaka experimental field IPB Bogor, and (c) Balitro, Bogor, Majalengka, and Sukabumi.The gingerol will be chemical-analyzed in three laboratories in Laboratorium Kimia Analitik Jurusan Kimia IPB, Laboratoriun Terpadu IPB, a Laboratorium Pusat Studi Biofarmaka LPPM-IPB.
The research sample is 20 gingerols.The composition of gingerol will be analysed through HPLC method, which will be used as the dependent variable.FTIR method gives the spectrum of infrared.The discretization process will gives the percentage of transmittance, which is observed in 1866 points for wavelength 4000 -400 cm -1 .This value represents the level of gingerol, which is used as the independent variable.DWT must have 2 M (M is positive integers), so that the observations are only taken 1024 points from the actual 1866 points, by considering the region of infrared spectrum.
The chosen 1024 points of observations will be transformed by using Discrete Wavelet Transform (DWT), by considering the possible resolution which gives wavelet coefficients size less than the sample size.Daubechies wavelet is used as the mother wavelet because it has been used in most applications and brought a better result [18].

B. Data
The analysis is using BR-WT method, which is divided into 2 steps, i.e. building the calibration model and the model validation.The criterions used to evaluate the model performance are R 2 , RMSE, R 2 predict , RMSEP, and plot between the actual data and the prediction.The best model is a model which gives much R 2 and R 2 predict , less RMSE and RMSEP, and the fittest prediction to the actual data.BR-WT performance will be compared with CR-WT (Continuum Regression-Wavelet Transform) and PLSR-WT (Partial Least Square Regression-Wavelet Transform).

C. Multicolinearity Identification
According to VIF's value, shows that all of the independent variables are identified multicollinearity, cause of VIF's value is greater than 20.

i. Multiplication of the i-th observation vector x (i)
T have a measurement of (1xp) with the wavelet transform matrix B * thus obtained vector d (i) *T with formula d (i) ii. Predict the value y by the formula: ; .

D. Model Calibration Levels Gingerol
The measurement results obtained by FTIR data percent transmitans ginger powder to 20 samples at 1866 points as shown in Fig. 1.Furthermore, 1024 was chosen for each observation point.This is to meet the requirements of DWT that need M 2 (M=10) and the spectrum presented in Fig. 2.
Fig. 1 and Fig. 2 show that the spectrum is in tune (almost parallel).As a result, the process of discrediting the infrared spectrum that produces 1024 points and used to obtain independent variables X 1 to X 1024 as a compiler matrix X, multicollinearity problems will arise because of the high correlation between X 1 and X 1024 .In addition to the large number of points produced (1024) results in the independent variables is much greater than the number of observations.Therefore it needs to be done using data compression in order to obtain TWD wavelet transform coefficient matrix D * .
Calibration model is built using 16 data.From the results of data analysis concluded that the best model for predicting levels of gingerol is built using 11 wavelet coefficients (for the mother wavelet Daubechies-10) at a resolution of 0, 1 and 3 and a coefficient function of scale discrete wavelet transforms.This is because in these conditions result whose behavior is relatively better than others, that can capture good measures of such models are relatively better R 2 .The results proved the vectors that form the matrix D * still has a high correlation, so if regressed between the response variable Y with D * arise the multicollinearity problems.To solve these problems, Bayes Regression method is used.Summary results of the processing method with BR-WT MINITAB macro programs are presented in Table 1.Fig. 3 presents the scatter diagram of observed data (y insample ) with the prediction results using BR-WT, CR-WT, and PLS-WT.While Fig. 4 presents the scatter diagram data observations (y outsample ) with predicted results using BR-WT, CR-WT, and PLS-WT.
From Table 1 we can see that the results of BR-WT method obtained good value model that is better than PLS-WT method, and relatively as good as CR-WT methods.Furthermore, estimated by the method derived BR-WT R 2 = 97.9%, R predict 2 = 99.0 %, an RMSEP = 0,0254.This is reinforced by Fig. 3 which shows that the dots prediction results using RB-TW obtained the points closer to the observational data points.So also in Fig. 4 shows that the points outcome prediction BR-WT method closer to the observational data points.So it can be said that the model for prediction of external data (data that is not involved in the modeling) is quite satisfactory.BR-WT method gives better results than the PLS-WT method.Meanwhile, when compared with CR-WT method is relatively equally well.Thus it can be said that the BR-WT method is one alternative that can be used to overcome the multicollinearity and singularity.
Thus we can conclude that the CR-DWT has very good potential for modeling calibration.Gingerol content of the calibration model using the CR-DWT approach the results of this study can be used to predict the content of the active compound (gingerol) in all kinds of ginger, if known FTIR spectrum.This is because all kinds of ginger have gingerol content of active substances that have the same chemical formula and the same spectral pattern.The difference is in the high and low percent of transmittance.Therefore, if the pattern resembles the pattern spectrum of FTIR spectra in Fig. 3, then this model can be used to predict the gingerol content of ginger powder.
Gingerol content of the calibration model can be used to predict the concentration of active compound gingerol in ginger with a high degree of accuracy, but cannot be used to predict the levels of active compounds in the rhizomes of plants of other drugs (e.g.turmeric).This is because the type of active compounds in each different medical plant means that its chemical formula is also different, resulting FTIR spectrum is also different with gingerol.Infrared spectrum of organic compounds has characteristic physical properties, which means the possibility of the two compounds has the same spectrum is very small (Nur and Adijuwana 1989).These differences affect the spectrum pattern of each active compound has a different calibration models.However, BR-WT method can be used to obtain calibration models for all levels of the active compounds of medicinal plant species.

V. CONCLUSION
The conclusion from the theoretical and empirical studies is the combination of Bayes Regression and preprocessing Discrete Wavelet Transform, which is called BR-WT, has a good effect in solving illconditioned and singularity.The application of CR-WT in the gingerol calibration brings R 2 = 97.9%,R 2 predict = 99%, and RMSEP 0.0254.Therefore, gingerol calibration model can be used predict the level of gingerol compounds, but it can not be used to predict other medical plants.
CR-TW method is the best method for solving ill conditioned and singularity issues, but it is not good for data having outliers.Therefore, a robust method is preferred and the combination with CR-TW, which will be called Robust Continuum Regression-Wavelet Transform (RCR-WT), is an interesting solution.
B T B = BB T = I.Because of B is orthogonal, the wavelet coefficients can be calculated by d = Bx.If x has a large dimension, computation of wavelet coefficients matrix inefficient manner, so that the pyramid algorithm is used.From the simple example shown the number of elements of each column of the matrix B T equal to zero except the first column associated with the scale function (father wavelet).This is because the elements other than the first column of the matrix B T obtained from the mother wavelet function has the general properties ∫ (nxp) = [x 1 ,x 2 , …,x p ] T where p = q = 2 M (M= positive integers) the DWT can be written: D (n x p) = X (n x p) B T (n x p) There are 10 levels of mother wavelet Daubechies (D-1 to D-10) so the matrix will be obtained 10.

TABLE 1 .
CONCENTRATION GINGEROL OBSERVATION, ALLEGATION AND THE RESULTS OF PREDICTION RESULTS