{"id":1966,"date":"2018-07-09T15:23:31","date_gmt":"2018-07-09T13:23:31","guid":{"rendered":"https:\/\/blogs.fu-berlin.de\/reseda\/?page_id=1966"},"modified":"2018-09-25T19:33:09","modified_gmt":"2018-09-25T17:33:09","slug":"rf-classification","status":"publish","type":"page","link":"https:\/\/blogs.fu-berlin.de\/reseda\/rf-classification\/","title":{"rendered":"RF Classification"},"content":{"rendered":"

There are some packages available containing the possibility to perform the standard RF algorthm described by Breiman (2001)<\/a>, e.g., in the caret<\/a>, randomForest<\/a>, ranger<\/a>, xgboost<\/a>, or randomForestSRC <\/a>packages. However, we will use the package called “randomForest” because it is the most common and therefore best supported.<\/p>\n

Below you can see a complete code implementation. While this is already executable with your input data, you should read the following comprehensive in-depth guide to understand the code in detail. Even better: You will learn how to generate numerous useful plots, which do great in each thesis!<\/p>\n

\r\n# import packages\r\nlibrary(raster)\r\nlibrary(randomForest)\r\n \r\n# import image (img) and shapefile (shp)\r\nsetwd(\"\/media\/sf_exchange\/landsatdata\/\")\r\nimg <- brick("LC081930232017060201T1-SC20180613160412_subset.tif")\r\nshp <- shapefile("training_data.shp")\r\n \r\n# extract samples with class labels and put them all together in a dataframe\r\nnames(img) <- c("b1", "b2", "b3", "b4", "b5", "b6", "b7")\r\nsmp <- extract(img, shp, df = TRUE)\r\nsmp$cl <- as.factor( shp$classes[ match(smp$ID, seq(nrow(shp)) ) ] )\r\nsmp <- smp[-1]\r\n\r\n# tune and train rf model\r\nsmp.size <- rep(min(summary(smp$cl)), nlevels(smp$cl))\r\nrfmodel <- tuneRF(x = smp[-ncol(smp)],\r\n                  y = smp$cl,\r\n                  sampsize = smp.size,\r\n                  strata = smp$cl,\r\n                  ntree = 250,\r\n                  importance = TRUE,\r\n                  doBest = TRUE\r\n                  )\r\n\r\n# save rf model \r\nsave(rfmodel, file = \"rfmodel.RData\")\r\n\r\n# predict image data with rf model\r\nresult <- predict(img,\r\n                  rfmodel,\r\n                  filename = "classification_RF.tif",\r\n                  overwrite = TRUE\r\n                  )\r\n<\/pre>\n
<\/br><\/a><\/p>\n
In-depth Guide<\/h1>\n
In order to be able to use the functions of the randomForest package, we must additionally load the library into the current session via library()<\/span>. If you do not use our VM, you must first download and install the packages with install.packages()<\/span>:<\/p>\n
\r\n#install.packages(\"raster\")\r\n#install.packages(\"randomForest\")\r\nlibrary(raster)\r\nlibrary(randomForest)\r\n<\/pre>\n
First, it is necessary to process the training samples in the form of a data frame. The necessary steps are described in detail in the previous section<\/a>.<\/p>\n
\r\nnames(img) <- c("b1", "b2", "b3", "b4", "b5", "b6", "b7")\r\nsmp <- extract(img, shp, df = TRUE)\r\nsmp$cl <- as.factor( shp$classes[ match(smp$ID, seq(nrow(shp)) ) ] )\r\nsmp <- smp[-1]\r\n<\/pre>\n
After that, you can identify the number of available training samples per class with summary()<\/span>. There is often an imbalance in the number of those training pixels, i.e., one class is represented by a large number of pixels, while another class has very few samples:<\/p>\n
\r\nsummary(smp$cl)\r\n##  baresoil    forest grassland  urban_hd  urban_ld     water \r\n##       719      2074      1226      1284       969       763\r\n<\/pre>\n
This often leads to the problem that classifiers favor and overclass strongly-represented classes in the classification. However, the Random Forest Algorithm, as an ensemble classifier, provides an ideal solution to compensate for this imbalance. For each decision tree, we draw a bootstrap sample from the minority class (class with the fewest samples). Then, we randomly draw the same number of cases, with replacement, from all other classes. This technique is called down-sampling.
\nIn our example this is the class baresoil<\/em> with 719 samples. With the rep()<\/span> function we form a vector where the length corresponds to the number of target classes. We will use this vector to tell the classifier how many samples it should randomly draw per class for each decision tree during training:<\/p>\n
\r\nsmp.size <- rep(min(summary(smp$cl)), nlevels(smp$cl))\r\nsmp.size\r\n## [1] 719 719 719 719 719 719\r\n<\/pre>\n
The complete training takes place via just one function call of tuneRF()<\/span>! This function automatically searches for the best parameter setting for mtry<\/em> – the number of variables available for each tree node. So we just have to worry about ntree<\/span>, i.e., the number of trees to grow. 250-1000 trees are usually sufficient. Basically, the more the better, but many trees will increase the calculation time. When tuneRF()<\/span> is called, we need to specify the training samples as x<\/span>, i.e., all columns of our smp<\/span> dataframe except the last one, and the corresponding class labels as y<\/span>, i.e. the last column of our smp<\/span> dataframe called “cl”:<\/p>\n
\r\nrfmodel <- tuneRF(x = smp[-ncol(smp)],\r\n                  y = smp$cl,\r\n                  sampsize = smp.size,\r\n                  strata = smp$cl,\r\n                  ntree = 250,\r\n                  importance = TRUE,\r\n                  doBest = TRUE\r\n                  )\r\n## mtry = 2  OOB error = 2.5% \r\n## Searching left ...\r\n## mtry = 1     OOB error = 2.54% \r\n## -0.01704545 0.05 \r\n## Searching right ...\r\n## mtry = 4     OOB error = 2.7% \r\n## -0.07954545 0.05                                       \r\n<\/pre>\n
In line 3, we pass our smp.size<\/span> vector as argument the sampsize =<\/span> to define how many samples it should draw per class, and strata =<\/span> at line 4 defines the column which should use for this stratified sampling. The argument importance = <\/span> in line 6 allows the subsequent assessment of the variable importance when set to TRUE<\/span>. By setting the argument doBest<\/span> to TRUE<\/span>, the RF with the optimal mtry <\/em> is output directly from the function.<\/p>\n
\"\"<\/a><\/p>\n
If you use tuneRF<\/span>, you will automatically get a plot that will tell you the OOB errors in the dependency of different mtry <\/em>settings. As mentioned, the feature automatically identifies the best mtry<\/em> setting and uses this to generate the optimal RF.<\/p>\n
When the model is created, we get some really useful information by executing the object name. First we get the command call with which we trained the model and the final number of variables tried at each split, i.e. the mtry<\/em> parameter. Furthermore, we get an averaged out of bag (OOB) estimate, as well as a complete confusion matrix based on the training data! The column headers contain the classes of training pixels and the rows describe the corresponding classification. For a more detailed description of a confusion matrix, please refer to the chapter Validate Classifiers<\/a>.<\/p>\n
\r\nrfmodel\r\n## \r\n## Call:\r\n##  randomForest(x = smp[-ncol(smp)], y = smp$cl, ntree = 300, mtry = mtry.opt,      \r\n##               strata = smp$cl, sampsize = smpsize, importance = TRUE) \r\n##                Type of random forest: classification\r\n##                      Number of trees: 300\r\n## No. of variables tried at each split: 2\r\n## \r\n##         OOB estimate of  error rate: 2.52%\r\n## Confusion matrix:\r\n##           baresoil forest grassland urban_hd urban_ld water class.error\r\n## baresoil       708      0         2        3        6     0 0.015299026\r\n## forest           0   2060         2        0       11     1 0.006750241\r\n## grassland        4      0      1220        0        2     0 0.004893964\r\n## urban_hd        18      0         0     1204       62     0 0.062305296\r\n## urban_ld         7     10         9       39      904     0 0.067079463\r\n## water            0      0         0        1        0   762 0.001310616\r\n<\/pre>\n
Before we started the training procedure, we set the argument importance = true<\/span>, which now allow us to have a look at the importance variable using the varImpPlot<\/span> command:<\/p>\n
\r\nvarImpPlot(rfmodel)\r\n<\/pre>\n
\"\"<\/a><\/p>\n
The variable importance <\/a>shows the most significant or important features for the classification, which are band 5 and 6 in this case. For details of those metrics, please refer to the Random Forest Section<\/a>. However, it is interesting to note that the most important bands also provide the largest spectral differences between the classes in the previous section<\/a>.<\/p>\n
Additionally, you can plot the RF model itself, which shows you the relationship between OOB Error<\/a> and the number of trees used. We can color the lines by passing a vector of hex-colors<\/a> whose length equals the number of classes + 1 – the first color is the average OOB line, which is also plotted automatically:<\/p>\n
\r\nplot(rfmodel, col = c(\"#000000\", \"#fbf793\", \"#006601\", \"#bfe578\", \"#d00000\", \"#fa6700\", \"#6569ff\"))\r\n<\/pre>\n
\"\"<\/a><\/p>\n
We can see a decrease in the error with increasing number of trees, where the urban classes have the highest OOB error values.<\/p>\n
Save the model by using the save()<\/span> function. This function saves the model object rfmodel<\/span> to your working directory, so that you have it permanently stored on your hard drive. If needed, you can load it any time with load()<\/span>.<\/p>\n
\r\nsave(rfmodel, file = \"rfmodel.RData\")\r\n#load(\"rfmodel.RData\")\r\n<\/pre>\n
Since your model is now fully trained, you can use it to predict all the pixels in your image. The command method predict()<\/span> takes a lot of work from you: It is recognized that there is an image which then passes through your Random Forest pixel by pixel. As with the training pixels, each image pixel is now individually classified and finally reassembled into your final classification image. Use the argument filename = <\/span> to specify the name of your output map:<\/p>\n
\r\nresult <- predict(img,\r\n                  rfmodel,\r\n                  filename = "classification.tif",\r\n                  overwrite = TRUE\r\n                  )\r\n<\/pre>\n
Random Forest classification completed!
\nIf you store the result of the predict()<\/span> function in an object, e.g., result<\/span>, you can plot the map using the standard plot command and passing this object:<\/p>\n
\r\nplot(result, \r\n     axes = FALSE, \r\n     box = FALSE,\r\n     col = c("#fbf793", # baresoil\r\n             "#006601", # forest\r\n             "#bfe578", # grassland\r\n             "#d00000", # urban_hd\r\n             "#fa6700", # urban_ld\r\n             "#6569ff"  # water\r\n             )\r\n     )\r\n<\/pre>\n
\"\"<\/a><\/p>\n
<\/br><\/br><\/p>\n
\n
\n