{"id":2618,"date":"2018-09-27T15:14:55","date_gmt":"2018-09-27T13:14:55","guid":{"rendered":"https:\/\/blogs.fu-berlin.de\/reseda\/?page_id=2618"},"modified":"2018-10-01T12:34:45","modified_gmt":"2018-10-01T10:34:45","slug":"svm-regression","status":"publish","type":"page","link":"https:\/\/blogs.fu-berlin.de\/reseda\/svm-regression\/","title":{"rendered":"SVM Regression"},"content":{"rendered":"

There are several R packages that provide SVM regression, or Support Vector Regression (SVR), support, e.g., caret<\/a>, e1071<\/a>, or kernLab<\/a>. We will use the e1071 package, as it offers an interface to the well-known libsvm <\/a>implementation. <\/p>\n

Below you can see a complete code implementation. Read the previous section<\/a> to learn about pre-processing the training data. While this is already executable with the given input data, you should read the in-depth guide in the following to understand the code in detail.<\/p>\n

\r\n# import\r\nlibrary(raster)\r\nlibrary(rgeos)\r\nlibrary(e1071)\r\n\r\nsetwd(\"\/media\/sf_exchange\/regression\/\")\r\nimg <- brick("LC081930232017060201T1-SC20180613160412_subset.tif")\r\nshp <- shapefile("reg_train_data.shp")\r\n\r\n# training sample preprocessing (see last section for details)\r\nnames(img) <- c("b1", "b2", "b3", "b4", "b5", "b6", "b7")\r\nshp <- gBuffer(shp, byid=TRUE, width=0)\r\nimg.subset <- crop(img, shp)\r\nimg.mask <- rasterize(shp, img.subset, getCover = TRUE)\r\nimg.mask[img.mask < 100] <- NA\r\nimg.subset <- mask(img.subset, img.mask)\r\ngrid <- rasterToPolygons(img.subset)\r\ngrid$ID <- seq.int(nrow(grid))\r\n\r\nsmp <- data.frame()\r\nfor (i in 1:length(grid)) {\r\n  cell <- intersect(grid[i, ], shp)\r\n  cell <- cell[cell$Class_name == "building" | cell$Class_name == "impervious", ]\r\n  if (length(cell) > 0) {\r\n    areaPercent <- sum(area(cell) \/ area(grid)[1])\r\n  } else {\r\n    areaPercent <- 0\r\n  }\r\n  newsmp <- cbind(grid@data[i, 1:nlayers(img)], areaPercent)\r\n  smp <- rbind(smp, newsmp)\r\n}\r\n\r\n# model training\r\n# define search ranges\r\ngammas = 2^(-8:3)\r\ncosts = 2^(-5:8)\r\nepsilons = c(0.1, 0.01, 0.001)\r\n# start training via gridsearch\r\nsvmgs <- tune(svm,\r\n              train.x = smp[-ncol(smp)],\r\n              train.y = smp[ncol(smp)],\r\n              type = "eps-regression",\r\n              kernel = "radial", \r\n              scale = TRUE,\r\n              ranges = list(gamma = gammas, cost = costs, epsolon = epsilons),\r\n              tunecontrol = tune.control(cross = 5)\r\n              )\r\n\r\n# pick best model\r\nsvrmodel <- svmgs$best.model\r\n\r\n# use best model for prediciton\r\nresult[result < 0] = 1\r\nresult[result < 0] = 0\r\n<\/pre>\n
<\/br><\/a><\/p>\n
In-depth Guide<\/font><\/h1>\n
In order to be able to use the regression functions of the e1071 package, we must additionally load the library into the current session via library()<\/span>. If you do not use our VM, you must first download and install the packages with install.packages()<\/span>:<\/p>\n
\r\n#install.packages(\"raster\")\r\n#install.packages(\"rgeos\")\r\n#install.packages(\"e1071\")\r\nlibrary(raster)\r\nlibrary(rgeos)\r\nlibrary(e1071)\r\n<\/pre>\n
First, it is necessary to process the training samples in the form of a data frame. The necessary steps are shown in line 10-31 and described in detail in the previous section<\/a>. <\/p>\n
After the preprocessing, we can train our Support Vector Regression with the training dataset smp<\/span>. We will utilize an epsilon Support Vector Regressions, which requires three parameters: one gamma \\(\\gamma\\) value, one cost \\(C\\) value as well as a epsilon \\(\\varepsilon\\) value (for more details refer to the SVM section<\/a>). These hyperparameters significantly determine the performance of the model. Finding the best hyparameters is not trivial and the best combination can not be determined in advance. Thus, we try to find the best combination iteratively by trial and error. Therefore, we create three vectors comprising all values that should be tested:<\/p>\n
\r\ngammas = 2^(-8:3)\r\ncosts = 2^(-5:8)\r\nepsilons = c(0.1, 0.01, 0.001)\r\n<\/pre>\n
So we have 14 different values for \\(\\gamma\\), 14 different values for \\(C\\) and three different values for \\(\\varepsilon\\). Thus, the whole training process is tested for 588 (14 * 14 * 3) models. Conversely, this means that the more parameters we check, the longer the training process takes.
\nWe start the training with the tune()<\/span> function. We need to specify the training samples as train.x<\/span>, i.e., all columns of our smp<\/span> dataframe except the last one, and the corresponding class labels as train.y<\/span>, i.e. the last column of our smp<\/span> dataframe, which holds the areaPercentage<\/span>of our target class imperviousness<\/em>:<\/p>\n
\r\nsvmgs <- tune(svm,\r\n              train.x = smp[-ncol(smp)],\r\n              train.y = smp[ncol(smp)],\r\n              type = "eps-regression",\r\n              kernel = "radial", \r\n              scale = TRUE,\r\n              ranges = list(gamma = gammas, cost = costs, epsolon = epsilons),\r\n              tunecontrol = tune.control(cross = 5)\r\n              )\r\n<\/pre>\n
We have to set the type<\/span> of the SVM to “eps-regression”<\/span> in line 4 in order to perform a regression task. Furthermore, we set the kernel used in training and predicting to a RBF kernel<\/a> via “radial”<\/span> (have a look at the SVM section<\/a> for more details). We set the argument scale<\/span> to TRUE<\/span> in order to initiate the z-transformation of our data<\/a>. The argument ranges<\/span> in line 7 takes a named list of parameter vectors spanning the sampling range. We put our vectors gammas<\/span>, costs<\/span> and epsilons<\/span> in this list. By using the tunecontrol<\/span> argument in line 8, you can set k for the k-fold cross validation on the training data<\/a>, which is necessary to assess the model performance. <\/p>\n
Depending on the complexity of the data, this step may take some time. Once completed, you can check the output by calling the resultant object name:<\/p>\n
\r\nsvmgs \r\n##\r\n## Parameter tuning of \u2018svm\u2019:\r\n##\r\n## - sampling method: 5-fold cross validation \r\n##\r\n## - best parameters:\r\n##       gamma cost epsolon\r\n##  0.00390625    8     0.1\r\n## \r\n## - best performance: 0.03157376 \r\n<\/pre>\n
In the course of the cross-validation, the overall accuracies were compared and the best parameters were determined: In our example, those are 0.0039, 8 and 0.1 for \\(\\gamma\\), \\(C\\) and \\(\\varepsilon\\), respectively. Furthermore, the Mean Absolute Error (MAE) of the best model is displayed, which is 0.0316, or 3.12%, in our case.<\/p>\n
We can extract the best model out of our svmgs<\/span> to use for image prediction:<\/p>\n
\r\nsvrmodel <- svmgs$best.model\r\nsvrmodel \r\n## \r\n## Call:\r\n## best.tune(method = svm, train.x = smp[-ncol(smp)], train.y = smp[ncol(smp)], ranges = list(gamma = gammas, cost = costs, \r\n##     epsolon = epsilons), tunecontrol = tune.control(cross = 5), type = "eps-regression", kernel = "radial", scale = TRUE)\r\n## \r\n## \r\n## Parameters:\r\n##    SVM-Type:  eps-regression \r\n##  SVM-Kernel:  radial \r\n##        cost:  8 \r\n##       gamma:  0.00390625 \r\n##     epsilon:  0.1 \r\n## \r\n## \r\n## Number of Support Vectors:  78\r\n<\/pre>\n
Save the best model by using the save()<\/span> function. This function saves the model object svrmodel<\/span> to your working directory, so that you have it permanently stored on your hard drive. If needed, you can load it any time with load()<\/span>.<\/p>\n
\r\nsave(svrmodel, file = \"svrmodel.RData\")\r\n#load(\"svrmodel.RData\")\r\n<\/pre>\n
Since your SVR model is now completely trained, you can use it to predict all the pixels in your image. The command method predict()<\/span> takes a lot of work from you: It is recognized that there is an image which will be processed pixel by pixel. As with the training pixels, each image pixel is now individually regressed and finally reassembled into your final regression image. Save the output as raster object result<\/span> and have a look at its minimum and maximum values (line 11):<\/p>\n
\r\nresult <- predict(img, svrmodel)\r\n\r\nresult\r\n## class       : RasterLayer \r\n## dimensions  : 504, 1030, 519120  (nrow, ncol, ncell)\r\n## resolution  : 30, 30  (x, y)\r\n## extent      : 369795, 400695, 5812395, 5827515  (xmin, xmax, ymin, ymax)\r\n## coord. ref. : +proj=utm +zone=33 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 \r\n## data source : in memory\r\n## names       : layer \r\n## values      : -0.7562021, 1.15675  (min, max)\r\n<\/pre>\n
You may will notice some “super-positive” (values above 1.0 or 100%) and\/ or some negative (values below 0.0 or 0%) values. Such values are not uncommon, since the SVR implementation of the e1071 package is not subject to any non-negative constraints. You could simply fix this issue by overriding all adequate entries with meaningful minimum and maximum values (0 for 0% and 1 for 100%) by adding the following two lines:<\/p>\n
\r\nresult[result > 1] = 1\r\nresult[result < 0] = 0\r\n<\/pre>\n
Finally, save your regression raster output using the writeRaster()<\/span> function and plot your result in R:<\/p>\n
\r\nwriteRaster(result, filename="regression.tif")\r\n\r\nplot(result, col=gray.colors(100))\r\n<\/pre>\n
\"\"<\/a><\/p>\n
Done! You now got a map, which indicates the percentage of imperviousness, i.e., subpixel-information, for every single pixel in your image data.<\/p>\n
<\/br><\/br><\/p>\n
\n
\n