Newer
Older
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Testing the Forrest\n",
"\n",
"## Importing the Basics"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from matplotlib import pyplot as plt\n",
"from machineLearning.metric import ConfusionMatrix, RegressionScores\n",
"from machineLearning.utility import ModelIO\n",
"from machineLearning.rf import (\n",
" RandomForest, DecisionTree,\n",
" Gini, Entropy, MAE, MSE,\n",
" CART, ID3, C45,\n",
" AdaBoosting, GradientBoosting,\n",
" Majority, Confidence, Average, Median\n",
")"
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generating Test Data\n",
"\n",
"Here I generate random test data. It's two blocks shifted very slightly in some dimensions. For classifier tasks each block gets a label, for regressor tasks each block gets the average coordinates plus some random value as a traget. It's a very simple dummy data set meant for testing the code.\n",
"\n",
"Here one can change the dimensionallity and amount of the data."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def dataShift(dims):\n",
" offSet = [5, 1.5, 2.5]\n",
" diffLen = abs(len(offSet) - dims)\n",
" offSet.extend([0] * diffLen)\n",
" random.shuffle(offSet)\n",
" return offSet[:dims]\n",
"\n",
"# Initialize some parameters\n",
"totalAmount = 6400\n",
"dims = 5\n",
"evalAmount = totalAmount // 4\n",
"trainAmount = totalAmount - evalAmount\n",
"offSet = dataShift(dims)\n",
"\n",
"# Create covariance matrix\n",
"cov = np.eye(dims) # This creates a covariance matrix with variances 1 and covariances 0\n",
"\n",
"# Generate random multivariate data\n",
"oneData = np.random.multivariate_normal(np.zeros(dims), cov, totalAmount)\n",
"twoData = np.random.multivariate_normal(offSet, cov, totalAmount)\n",
"\n",
"# Split the data into training and evaluation sets\n",
"trainData = np.vstack((oneData[:trainAmount], twoData[:trainAmount]))\n",
"validData = np.vstack((oneData[trainAmount:], twoData[trainAmount:]))\n",
"\n",
"# Labels for classification tasks\n",
"trainLabels = np.hstack((np.zeros(trainAmount), np.ones(trainAmount)))\n",
"validLabels = np.hstack((np.zeros(evalAmount), np.ones(evalAmount)))\n",
"\n",
"# Targets for regression tasks\n",
"trainTargets = np.sum(trainData, axis=1) + np.random.normal(0, 0.1, 2*trainAmount)\n",
"validTargets = np.sum(validData, axis=1) + np.random.normal(0, 0.1, 2*evalAmount)\n",
"\n",
"# Shuffle the training data\n",
"trainIndex = np.random.permutation(len(trainData))\n",
"trainData = trainData[trainIndex]\n",
"trainLabels = trainLabels[trainIndex]\n",
"trainTargets = trainTargets[trainIndex]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating the Forrest\n",
"\n",
"Here the forrest is created. One can set the number of trees and set the maximum depth. Depending on the task, we add a different impurity function and a different leaf function. Finally we add the split algorithm and set the feature percentile. Higher numbers look at more possible splits, but decreases speed. Lower numbers look at less possible splits, speeding up the algorithm. Depending on the data set this can have a strong impact on the performance.\n",
"\n",
"One can set a different depth, leaf function, splitting algorithm and impurity function for each tree. Here in this simple case we create all trees with same parameters."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"task = 'classifier' # 'classifier'/'regressor'\n",
"forrest = RandomForest(bootstrapping=False, retrainFirst=False)\n",
johannes bilk
committed
"for i in range(5):\n",
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
" tree = DecisionTree(maxDepth=7, minSamplesSplit=2)\n",
" if task == 'regressor':\n",
" tree.setComponent(MSE())\n",
" tree.setComponent(Mean())\n",
" elif task == 'classifier':\n",
" tree.setComponent(Entropy())\n",
" tree.setComponent(Mode())\n",
" tree.setComponent(CART(featurePercentile=90))\n",
" forrest.append(tree)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Trainining the tree\n",
"\n",
"Again, depending on the task we train the forrest with targets or labels. Then we make a prediction and plot the tree."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tree 1 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 18%\n",
"tree 2 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 18%\n",
"tree 3 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 18%\n",
"tree 4 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 18%\n",
"tree 5 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 18%\n",
"━━━━━━━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: GradientBoosting, bootstrapping: False\n",
johannes bilk
committed
"—————————————————————— tree: 1/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 4 <= -0.88, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 2/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.86, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 3/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 3 <= 2.22, samples: 7\n",
" │ │ ├─feat: 4 <= 0.63, samples: 6\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 1 <= 2.37, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
johannes bilk
committed
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 4/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 1 <= 2.37, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 4 <= 0.49, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 5/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.86, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
johannes bilk
committed
"\n"
]
}
],
"source": [
"if task == 'regressor':\n",
" forrest.train(trainData, trainTargets)\n",
"elif task == 'classifier':\n",
" forrest.train(trainData,trainLabels)\n",
johannes bilk
committed
"prediction = forrest.eval(validData)\n",
"print(forrest)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Accuracy(name='tree: 0', accuracy=0.9959375),\n",
" Accuracy(name='tree: 1', accuracy=0.9959375),\n",
" Accuracy(name='tree: 2', accuracy=0.9953125),\n",
" Accuracy(name='tree: 3', accuracy=0.9953125),\n",
" Accuracy(name='tree: 4', accuracy=0.9959375)]"
johannes bilk
committed
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"forrest.accuracy(validData, validLabels)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABXqklEQVR4nO3de1xVVd4/8M/aW+EIglxUjhJ4QYNM1LyOl7SLE5VZPj2llc94Se2Xo5PlNJWVGpVazeTY5K3pYk1Tk03zWM9MpeNY3srxmkkoZCCB6QEVEMQBdO/1+4M4euRsA4S9YJ/P+/Xy9YrFPoe1PnvR+bL3WucIKaUEERERkUNoqjtARERE1JBY3BAREZGjsLghIiIiR2FxQ0RERI7C4oaIiIgchcUNEREROQqLGyIiInIUFjdERETkKCxuiIiIyFFY3BAREZGjsLghamLefPNNCCH8/nvsscca5Wd++eWXeOqpp1BcXNwoz38pqvPYtWuX6q7U2/Lly/Hmm2+q7gZRwGihugNE5N/TTz+NLl26+LT17NmzUX7Wl19+idTUVEyaNAkRERGN8jMC2fLly9G2bVtMmjRJdVeIAgKLG6Im6qabbkL//v1Vd+OSlJWVITQ0VHU3lDl9+jRCQkJUd4Mo4PC2FFEz9emnn+Lqq69GaGgowsLCMGrUKKSnp/scs2/fPkyaNAldu3aFy+WC2+3GvffeixMnTniPeeqpp/Cb3/wGANClSxfvLbCcnBzk5ORACOH3looQAk899ZTP8wghsH//ftxzzz2IjIzEsGHDvN//85//jH79+qFVq1aIiorCXXfdhby8vHqNfdKkSWjdujVyc3Nxyy23oHXr1oiNjcWyZcsAAGlpabjuuusQGhqKTp064d133/V5fPWtrs2bN+P//b//h+joaISHh2PChAkoKiqq8fOWL1+OK6+8EsHBwejYsSNmzJhR4xbeNddcg549e2L37t0YPnw4QkJC8Pjjj6Nz585IT0/Hpk2bvNlec801AIDCwkI8/PDDSE5ORuvWrREeHo6bbroJX3/9tc9zb9y4EUIIvP/++1iwYAEuu+wyuFwuXH/99fjuu+9q9Hf79u24+eabERkZidDQUPTq1QsvvfSSzzEZGRm44447EBUVBZfLhf79++P//u//6noqiJokXrkhaqJOnjyJ48eP+7S1bdsWAPD2229j4sSJSElJwfPPP4/Tp09jxYoVGDZsGL766it07twZALB+/XpkZ2dj8uTJcLvdSE9Pxx//+Eekp6fj3//+N4QQuP322/Htt9/iL3/5C37/+997f0a7du1w7NixOvf7zjvvRPfu3bFw4UJIKQEACxYswNy5czF27FhMnToVx44dw8svv4zhw4fjq6++qtetMMMwcNNNN2H48OF44YUX8M4772DmzJkIDQ3FE088gfHjx+P222/HypUrMWHCBAwePLjGbb6ZM2ciIiICTz31FDIzM7FixQp8//333mICqCraUlNTMXLkSEyfPt173M6dO/HFF1+gZcuW3uc7ceIEbrrpJtx11134n//5H8TExOCaa67Br371K7Ru3RpPPPEEACAmJgYAkJ2djQ8//BB33nknunTpgvz8fLzyyisYMWIE9u/fj44dO/r097nnnoOmaXj44Ydx8uRJvPDCCxg/fjy2b9/uPWb9+vW45ZZb0KFDB8yaNQtutxsHDhzAP/7xD8yaNQsAkJ6ejqFDhyI2NhaPPfYYQkND8f7772PMmDH429/+hv/6r/+q8/kgalIkETUpq1atkgD8/pNSytLSUhkRESGnTZvm8ziPxyPbtGnj03769Okaz/+Xv/xFApCbN2/2tv32t7+VAOShQ4d8jj106JAEIFetWlXjeQDI+fPne7+eP3++BCDvvvtun+NycnKkrutywYIFPu1paWmyRYsWNdqt8ti5c6e3beLEiRKAXLhwobetqKhItmrVSgoh5Hvvvedtz8jIqNHX6ufs16+frKys9La/8MILEoD86KOPpJRSFhQUyKCgIHnDDTdIwzC8xy1dulQCkG+88Ya3bcSIERKAXLlyZY0xXHnllXLEiBE12svLy32eV8qqzIODg+XTTz/tbfv8888lAHnFFVfIiooKb/tLL70kAci0tDQppZRnz56VXbp0kZ06dZJFRUU+z2uapve/r7/+epmcnCzLy8t9vj9kyBDZvXv3Gv0kam54W4qoiVq2bBnWr1/v8w+o+su8uLgYd999N44fP+79p+s6Bg0ahM8//9z7HK1atfL+d3l5OY4fP46f/exnAIA9e/Y0Sr/vv/9+n6//93//F6ZpYuzYsT79dbvd6N69u09/62rq1Kne/46IiEBiYiJCQ0MxduxYb3tiYiIiIiKQnZ1d4/H33Xefz5WX6dOno0WLFvjkk08AAP/6179QWVmJBx98EJp27n+X06ZNQ3h4OD7++GOf5wsODsbkyZNr3f/g4GDv8xqGgRMnTqB169ZITEz0e34mT56MoKAg79dXX301AHjH9tVXX+HQoUN48MEHa1wNq74SVVhYiM8++wxjx45FaWmp93ycOHECKSkpOHjwIH744Ydaj4GoKeJtKaImauDAgX4XFB88eBAAcN111/l9XHh4uPe/CwsLkZqaivfeew8FBQU+x508ebIBe3vOhbd+Dh48CCklunfv7vf484uLunC5XGjXrp1PW5s2bXDZZZd5X8jPb/e3lubCPrVu3RodOnRATk4OAOD7778HUFUgnS8oKAhdu3b1fr9abGysT/HxU0zTxEsvvYTly5fj0KFDMAzD+73o6Ogax8fHx/t8HRkZCQDesWVlZQG4+K667777DlJKzJ07F3PnzvV7TEFBAWJjY2s9DqKmhsUNUTNjmiaAqnU3bre7xvdbtDj3az127Fh8+eWX+M1vfoM+ffqgdevWME0TN954o/d5LubCIqHa+S/CFzr/alF1f4UQ+PTTT6Hreo3jW7du/ZP98Mffc12sXf64/qcxXTj2n7Jw4ULMnTsX9957L5555hlERUVB0zQ8+OCDfs9PQ4yt+nkffvhhpKSk+D2mW7dutX4+oqaIxQ1RM5OQkAAAaN++PUaOHGl5XFFRETZs2IDU1FTMmzfP21595ed8VkVM9ZWBC3cGXXjF4qf6K6VEly5dcPnll9f6cXY4ePAgrr32Wu/Xp06dwtGjR3HzzTcDADp16gQAyMzMRNeuXb3HVVZW4tChQxfN/3xW+X7wwQe49tpr8frrr/u0FxcXexd210X13Pjmm28s+1Y9jpYtW9a6/0TNDdfcEDUzKSkpCA8Px8KFC3HmzJka36/e4VT9V/6Ff9UvWbKkxmOq34vmwiImPDwcbdu2xebNm33aly9fXuv+3n777dB1HampqTX6IqX02ZZutz/+8Y8+Ga5YsQJnz57FTTfdBAAYOXIkgoKC8Ic//MGn76+//jpOnjyJUaNG1ernhIaG+n33Z13Xa2Ty17/+td5rXvr27YsuXbpgyZIlNX5e9c9p3749rrnmGrzyyis4evRojeeozw45oqaGV26Impnw8HCsWLECv/jFL9C3b1/cddddaNeuHXJzc/Hxxx9j6NChWLp0KcLDw73bpM+cOYPY2Fj885//xKFDh2o8Z79+/QAATzzxBO666y60bNkSo0ePRmhoKKZOnYrnnnsOU6dORf/+/bF582Z8++23te5vQkICnn32WcyZMwc5OTkYM2YMwsLCcOjQIaxZswb33XcfHn744QbLpy4qKytx/fXXY+zYscjMzMTy5csxbNgw3HrrrQCqtsPPmTMHqampuPHGG3Hrrbd6jxswYAD+53/+p1Y/p1+/flixYgWeffZZdOvWDe3bt8d1112HW265BU8//TQmT56MIUOGIC0tDe+8847PVaK60DQNK1aswOjRo9GnTx9MnjwZHTp0QEZGBtLT07Fu3ToAVYvVhw0bhuTkZEybNg1du3ZFfn4+tm3bhsOHD9d4nx2iZkfRLi0isuBv67M/n3/+uUxJSZFt2rSRLpdLJiQkyEmTJsldu3Z5jzl8+LD8r//6LxkRESHbtGkj77zzTnnkyJEaW6OllPKZZ56RsbGxUtM0n23hp0+fllOmTJFt2rSRYWFhcuzYsbKgoMByK/ixY8f89vdvf/ubHDZsmAwNDZWhoaEyKSlJzpgxQ2ZmZtY5j4kTJ8rQ0NAax44YMUJeeeWVNdo7deokR40aVeM5N23aJO+77z4ZGRkpW7duLcePHy9PnDhR4/FLly6VSUlJsmXLljImJkZOnz69xlZrq58tZdU2/VGjRsmwsDAJwLstvLy8XP7617+WHTp0kK1atZJDhw6V27ZtkyNGjPDZOl69Ffyvf/2rz/NabdXfunWr/PnPfy7DwsJkaGio7NWrl3z55Zd9jsnKypITJkyQbrdbtmzZUsbGxspbbrlFfvDBB37HQNScCCltWGVHRNSEvPnmm5g8eTJ27tzZ7D/igohq4pobIiIichQWN0REROQoLG6IiIjIUbjmhoiIiByFV26IiIjIUVjcEBERkaME3Jv4maaJI0eOICwszPIt0YmIiKhpkVKitLQUHTt2hKZd/NpMwBU3R44cQVxcnOpuEBERUT3k5eXhsssuu+gxAVfchIWFAagKJzw8XHFvGp9hGPj+++/RqVMny08UpobH3NVg7mowdzUCLfeSkhLExcV5X8cvJuCKm+pbUeHh4QFT3ABV4w2Eyd9UMHc1mLsazF2NQM29NktKuKCYiIiIHEVpcbN582aMHj0aHTt2hBACH3744U8+ZuPGjejbty+Cg4PRrVs3vPnmm43eTyIiImo+lBY3ZWVl6N27N5YtW1ar4w8dOoRRo0bh2muvxd69e/Hggw9i6tSpWLduXSP3tPkSQiAuLo47w2zG3NVg7mowdzWYu7Um8w7FQgisWbMGY8aMsTzm0Ucfxccff4xvvvnG23bXXXehuLgYa9eurdXPKSkpQZs2bXDy5MmAWHNDRETkBHV5/W5Wa262bduGkSNH+rSlpKRg27ZtinrU9BmGgYyMDO/CM7IHc1eDuavB3NVg7taa1W4pj8eDmJgYn7aYmBiUlJTgP//5D1q1alXjMRUVFaioqPB+XVJSAqBqUlRPCCEENE2DaZo4/0JWdfuFE8eqXdM0CCH8tgNVbyBYm3Zd1yGl9Nt+YR+t2s8f03/+858aY23uY2oO5+n83J0ypqZ+nqSUfnNvzmPy197UxgSgRu7NfUzN4TwBNXNv7mOqy3m6mGZV3NTHokWLkJqaWqM9PT0drVu3BgBERUUhPj4ehw8fRmFhofcYt9sNt9uNnJwclJaWetvj4uIQHR2NgwcPory83NvetWtXhIeHY//+/T4nITExEUFBQUhLS/PpQ3JyMiorK5GZmelt03UdycnJKC0tRXZ2trfd5XIhKSkJRUVFyMvL87aHhYUhISEBBQUF8Hg83vbqMR05cgSFhYVIT0+HEMIRY2oO56m4uNgndyeMqTmcp4qKCp/cnTCm5nCeQkNDUVRU5M3dCWNqDuepXbt2KC0t9cm9uY/pYucpKysLtdWs1twMHz4cffv2xZIlS7xtq1atwoMPPoiTJ0/6fYy/KzdxcXEoLCz03rNzcrV/5swZfPPNN7jyyiuh67ojxtQcztOFuTthTM3hPBmGgbS0tBq5N+cx+WtvamMyTRP79u3zyb25j6k5nCcpZY3cm/uYLnaeioqKEBUVVas1N83qys3gwYPxySef+LStX78egwcPtnxMcHAwgoODa7Trul7jTY+sPqvC6s2RGrO9+q/OC1n10aq9RYsWSEhIQMuWLX1W1DfnMTWH82SVe3MeU3M4T7qu+829OY/Jqr0pjUnTNL+51/V5mtKYGqq9McckpWyQ3Ova3tTOk9/nrvWRjeDUqVPYu3cv9u7dC6Bqq/fevXuRm5sLAJgzZw4mTJjgPf7+++9HdnY2HnnkEWRkZGD58uV4//338dBDD6nofrMghEB4eDi3CtqMuavB3NVg7mowd2tKi5tdu3bhqquuwlVXXQUAmD17Nq666irMmzcPAHD06FFvoQMAXbp0wccff4z169ejd+/eePHFF/Haa68hJSVFSf+bg+rL9FxNby/mrgZzV4O5q8HcrSm9LXXNNdfUuD93Pn/vPnzNNdfgq6++asReOQ8nvhrMXQ3mrgZzV4O5+9es3ueGiIiI6KewuCEiIiJHaTJbwe0SaB+/IKVEeXk5XC4XF53ZiLmrwdzVYO5qBFrudXn9blZbwal+goKCVHchIDF3NQIh95RnPlbdhQtI6AIwJAA0rRfZdXNHqe5CowqE+V4fvC3lcKZpIi0trcYbLlHjYu5qMHc1dAEM62hCb1p1jeNxvltjcUNERESOwuKGiIiIHIXFDRERETkKixuH0zQNycnJlp/hQY2DuavB3NUwJLD1iPbjgmKyC+e7NSYSACorK1V3ISAxdzWYuxrBtf9MQ2pAnO/+sbhxONM0kZmZydX0NmPuajB3NXQBDIjhbim7cb5bY3FDREREjsLihoiIiByFxU0A0HXeDFeBuavB3NXgYmI1ON/948cvOJyu60hOTlbdjYDD3NVg7moYUmDrEb7I2o3z3Rqv3DiclBIlJSUIsM9HVY65q8HcVZGIDJYAmLudON+tsbhxONM0kZ2dzdX0NmPuajB3NXQB9GrL3VJ243y3xuKGiIiIHIXFDRERETkKi5sA4HK5VHchIDF3NZi7/SSA02e54kYFznf/uFvK4XRdR1JSkupuBBzmrgZzV8OUAjvzuVvKbpzv1njlxuFM08SJEye44MxmzF0N5q6GgIQ7RELw2o2tON+tsbhxOCkl8vLyuFXQZsxdDeauhiaAxEgTGndL2Yrz3RqLGyIiInIUFjdERETkKCxuAkBYWJjqLgQk5q4Gc7efBFBUzhU3KnC++8fdUg6n6zoSEhJUdyPgMHc1mLsaphTYd4ILbuzG+W6NV24czjRNeDwerqa3GXNXg7mrISDRKczkbimbcb5bY3HjcFJKeDwerqa3GXNXg7mroQmgc7jkbimbcb5bY3FDREREjsLihoiIiByFxY3DCSEQFRUFIXi92E7MXQ3mroaUgKdMgHdH7MX5bo27pRxO0zTEx8er7kbAYe5qMHc1TAhkFvMF1m6c79Z45cbhTNNEbm4uV9PbjLmrwdzV0CCRGGFC424pW3G+W2Nx43BSShQWFnI1vc2YuxrMXQ0hAHeoBO+O2Ivz3RqLGyIiInIUFjdERETkKCxuHE4IAbfbzdX0NmPuajB3NUwJ5JQImLw7YivOd2vcLeVwmqbB7Xar7kbAYe5qMHc1JAS+L+ULrN04363xyo3DGYaBrKwsGIahuisBhbmrwdzV0IREr2gTmuClGztxvltjcRMASktLVXchIDF3NZi7/QSASJcEr93Yj/PdPxY3RERE5CgsboiIiMhRWNw4nBACcXFxXE1vM+auBnNXw5RAZpHG3VI243y3xt1SDqdpGqKjo1V3I+AwdzWYuxoSAp7TqnsReDjfrfHKjcMZhoGMjAyuprcZc1eDuauhCYkBMQZ3S9mM890ai5sAUF5erroLAYm5q8Hc7ScAhLQAd0spwPnuH4sbIiIichQWN0REROQoLG4cTtM0dO3aFZrGU20n5q4Gc1fDkMC+4xoMLrmxFee7Ne6WcjghBMLDw1V3I+AwdzWYuyoCRRWq+xB4ON+tsdxzOMMwkJaWxtX0NmPuajB3NXQhMayjAZ27pWzF+W6NxU0A4MRXg7mrwdzV0LlVSgnOd/9Y3BAREZGjsLghIiIiR2Fx43CapiExMZGr6W3G3NVg7moYEtiZz91SduN8t8ZEAkBQUJDqLgQk5q4Gc1ejgks/lOB894/FjcOZpom0tDSYpqm6KwGFuavB3NXQBTCso8lFxTbjfLfG4oaIiIgchcUNEREROQqLGyIiInIUFjcOp2kakpOTuZreZsxdDeauhiGBrUe4W8punO/WlCeybNkydO7cGS6XC4MGDcKOHTsuevySJUuQmJiIVq1aIS4uDg899BDKy8tt6m3zVFlZqboLAYm5q8Hc1QjWVfcgMHG++6e0uFm9ejVmz56N+fPnY8+ePejduzdSUlJQUFDg9/h3330Xjz32GObPn48DBw7g9ddfx+rVq/H444/b3PPmwzRNZGZmcjW9zZi7GsxdDV0AA2K4W8punO/WlBY3ixcvxrRp0zB58mT06NEDK1euREhICN544w2/x3/55ZcYOnQo7rnnHnTu3Bk33HAD7r777p+82kNERESBo4WqH1xZWYndu3djzpw53jZN0zBy5Ehs27bN72OGDBmCP//5z9ixYwcGDhyI7OxsfPLJJ/jFL35h+XMqKipQUVHh/bqkpARA1YeNVX/gmBACmqbBNE1Iee6mcXX7hR9MZtWuaRqEEH7bAdSorq3adV2HlNJv+4V9tGq/cEwXjtUJY/qpvjeFMZ3/M5wypp9qVzkmKaXf3JvzmPy1V5E+V0okAFMKCEhotWmXgAkBDRLivHZTAhICmpA4/0KMVXv1OhsB6fOp4NXtF17NsW4XjTImwzAc+/tUNWbf+d7cx1SX83Qxyoqb48ePwzAMxMTE+LTHxMQgIyPD72PuueceHD9+HMOGDYOUEmfPnsX9999/0dtSixYtQmpqao329PR0tG7dGgAQFRWF+Ph4HD58GIWFhd5j3G433G43cnJyUFpa6m2Pi4tDdHQ0Dh486LPep2vXrggPD8f+/ft9TkJiYiKCgoKQlpbm04fk5GRUVlYiMzPT26brOpKTk1FaWors7Gxvu8vlQlJSEoqKipCXl+dtDwsLQ0JCAgoKCuDxeLzt1WM6cuQIioqKkJ6eDiGEI8bUHM5TcXGxT+5OGFNzOE8VFRU+uTthTP7OEwBEBgO92p57gTl9FtiZryMmBEiMPNdeVC6w74RAfJhE5/BzLySeMoHMYoHuERLu0HPtOSUC35cK9IySiHSda88s0uA5DfRrbyLkvFeOfcc1lFQC7UOAIR0kJKoeszNfQ4VR9eZ+59t6REOwXnUbq1rVgmS9UcaUlpbm2N+ndu3aoayszDvfnTCmi52nrKws1JaQF5ZZNjly5AhiY2Px5ZdfYvDgwd72Rx55BJs2bcL27dtrPGbjxo2466678Oyzz2LQoEH47rvvMGvWLEybNg1z5871+3P8XbmJi4tDYWEhwsPDATS9v8qaUmXMMXFMHFPTG9PNC9eiaV25ET5Xbc61q79y8/c5N3HuOWRMRUVFiIqKwsmTJ72v31aUXblp27YtdF1Hfn6+T3t+fj7cbrffx8ydOxe/+MUvMHXqVABVlWVZWRnuu+8+PPHEE363wwUHByM4OLhGu67r3r+Aqlltp7vwODvaq//qvJBVH63ahRA4deoUwsLCvJV9Q/Wxru0NNabmcJ6scm/OY2oO50lKibKyshq5N+cxWbcLv1uvZR3bTQjAX7v0vzrYf7tEeBBQVFHVr/NZbQ/3397wYzo/O6f9PlnN97r2sa7tTe33ye9z1/rIBhYUFIR+/fphw4YN3jbTNLFhwwafKznnO336dI0wqger6AJUk2eaJrKzs7ma3mbMXQ3mroYuqm4ncbeUvTjfrSm7cgMAs2fPxsSJE9G/f38MHDgQS5YsQVlZGSZPngwAmDBhAmJjY7Fo0SIAwOjRo7F48WJcddVV3ttSc+fOxejRo+tU0REREZFzKS1uxo0bh2PHjmHevHnweDzo06cP1q5d611knJub63Ol5sknn4QQAk8++SR++OEHtGvXDqNHj8aCBQtUDYGIiIiaGKXFDQDMnDkTM2fO9Pu9jRs3+nzdokULzJ8/H/Pnz7ehZ87hcrlUdyEgMXc1mLv9JKp2NnFxgP043/1TXtxQ49J1HUlJSaq7EXCYuxrMXQ1TCuzM59IAu3G+W1P+2VLUuEzTxIkTJ7jgzGbMXQ3mroaAhDtEQvDaja04362xuHE4KSXy8vK4m8xmzF0N5q6GJqreZE/jbilbcb5bY3FDREREjsLihoiIiByFxU0ACAsLU92FgMTc1WDu9pOo+rwn3hyxH+e7f9wt5XC6riMhIUF1NwIOc1eDuathyqoPsiR7cb5b45UbhzNNEx6Ph6vpbcbc1WDuaghIdAozuVvKZpzv1ljcOJyUEh6Ph6vpbcbc1WDuamgC6BwuuVvKZpzv1ljcEBERkaOwuCEiIiJHYXHjcEIIREVFQQheL7YTc1eDuashJeApE+DdEXtxvlvjbimH0zQN8fHxqrsRcJi7GsxdDRMCmcV8gbUb57s1XrlxONM0kZuby9X0NmPuajB3NTRIJEaY0Lhbylac79ZY3DiclBKFhYVcTW8z5q4Gc1dDCMAdKsG7I/bifLfG4oaIiIgchcUNEREROQqLG4cTQsDtdnM1vc2YuxrMXQ1TAjklAibvjtiK890ad0s5nKZpcLvdqrsRcJi7GsxdDQmB70v5Ams3zndrvHLjcIZhICsrC4ZhqO5KQGHuajB3NTQh0SvahCZ46cZOnO/WWNwEgNLSUtVdCEjMXQ3mbj8BINIlwWs39uN894/FDRERETkKixsiIiJyFBY3DieEQFxcHFfT24y5q8Hc1TAlkFmkcbeUzTjfrXG3lMNpmobo6GjV3Qg4zF0N5q6GhIDntOpeBB7Od2u8cuNwhmEgIyODq+ltxtzVYO5qaEJiQIzB3VI243y3xuImAJSXl6vuQkBi7mowd/sJACEtwN1SCnC++8fihoiIiByFxQ0RERE5Cosbh9M0DV27doWm8VTbibmrwdzVMCSw77gGg0tubMX5bo27pRxOCIHw8HDV3Qg4zF0N5q6KQFGF6j4EHs53ayz3HM4wDKSlpXE1vc2YuxrMXQ1dSAzraEDnbilbcb5bY3ETADjx1WDuajB3NXRulVKC890/FjdERETkKCxuiIiIyFFY3DicpmlITEzkanqbMXc1mLsahgR25nO3lN04360xkQAQFBSkugsBibmrwdzVqODSDyU43/1jceNwpmkiLS0Npmmq7kpAYe5qMHc1dAEM62hyUbHNON+tsbghIiIiR2FxQ0RERI7C4oaIiIgchcWNw2mahuTkZK6mtxlzV4O5q2FIYOsR7payG+e7NSYSACorK1V3ISAxdzWYuxrBuuoeBCbOd/9Y3DicaZrIzMzkanqbMXc1mLsaugAGxHC3lN04362xuCEiIiJHYXFDREREjsLiJgDoOm+Gq8Dc1WDuanAxsRqc7/61UN0Baly6riM5OVl1NwIOc1eDuathSIGtR/giazfOd2u8cuNwUkqUlJRASv5ZZSfmrgZzV0UiMlgCYO524ny3xuLG4UzTRHZ2NlfT24y5q8Hc1dAF0Kstd0vZjfPdGosbIiIichQWN0REROQoLG4CgMvlUt2FgMTc1WDu9pMATp/lihsVON/9424ph9N1HUlJSaq7EXCYuxrMXQ1TCuzM524pu3G+W+OVG4czTRMnTpzggjObMXc1mLsaAhLuEAnBaze24ny3xuLG4aSUyMvL41ZBmzF3NZi7GpoAEiNNaNwtZSvOd2ssboiIiMhRWNwQERGRo7C4CQBhYWGquxCQmLsazN1+EkBROVfcqMD57h93SzmcrutISEhQ3Y2Aw9zVYO5qmFJg3wkuuLEb57s1XrlxONM04fF4uJreZsxdDeauhoBEpzCTu6Vsxvlurd7Fzdtvv42hQ4eiY8eO+P777wEAS5YswUcffVSn51m2bBk6d+4Ml8uFQYMGYceOHRc9vri4GDNmzECHDh0QHByMyy+/HJ988kl9h+F4Ukp4PB6uprcZc1eDuauhCaBzuORuKZtxvlurV3GzYsUKzJ49GzfffDOKi4thGAYAICIiAkuWLKn186xevRqzZ8/G/PnzsWfPHvTu3RspKSkoKCjwe3xlZSV+/vOfIycnBx988AEyMzPx6quvIjY2tj7DICIiIgeqV3Hz8ssv49VXX8UTTzwBXT/3rpT9+/dHWlparZ9n8eLFmDZtGiZPnowePXpg5cqVCAkJwRtvvOH3+DfeeAOFhYX48MMPMXToUHTu3BkjRoxA79696zMMIiIicqB6LSg+dOgQrrrqqhrtwcHBKCsrq9VzVFZWYvfu3ZgzZ463TdM0jBw5Etu2bfP7mP/7v//D4MGDMWPGDHz00Udo164d7rnnHjz66KM+Rdb5KioqUFFR4f26pKQEAGAYhveKkxACmqbBNE2fy3vV7dXH/VS7pmkQQvhtB1DjvqhVu67rkFL6bb+wj1bt1X2UUiIiIsL7XE4YU3M4Txfm7oQxNZfz5C/35j4m/2sqJHRx/ldVC3sFfG8PWbZLwISABglxXrspAQkBTUicf5fJqt2QVc9VUAZokKj+pvFjTPoFt6qs20WjjMkwDMf+Pgkhasz35j6mupyni6lXcdOlSxfs3bsXnTp18mlfu3Ytrrjiilo9x/Hjx2EYBmJiYnzaY2JikJGR4fcx2dnZ+OyzzzB+/Hh88skn+O677/DLX/4SZ86cwfz58/0+ZtGiRUhNTa3Rnp6ejtatWwMAoqKiEB8fj8OHD6OwsNB7jNvthtvtRk5ODkpLS73tcXFxiI6OxsGDB1FeXu5t79q1K8LDw7F//36fk5CYmIigoKAaV7WSk5NRWVmJzMxMb5uu60hOTkZpaSmys7O97S6XC0lJSSgqKkJeXp63PSwsDAkJCSgoKIDH4/G2V4/pyJEjKC4uRnFxsWPG1BzO08mTJ31yd8KYmst5Oj93p4zpwvMEAJHBQK+2515gTp8FdubriAmperfgakXlVTuZ4sMkOoefeyHxlAlkFgt0j5Bwh55rzykR+L5UoGeURKTrXHtmkQbPaaBfexMh571y7DuuoahCIDoEaB8qUf3xmTvzNVQYwLCOvi+CW49oCNaBATHn2g0JbD2iN8qY0tLSHP37ZBgG0tPTHTUmq/OUlZWF2hKyHiuRXnvtNTz11FN48cUXMWXKFLz22mvIysrCokWL8Nprr+Guu+76yec4cuQIYmNj8eWXX2Lw4MHe9kceeQSbNm3C9u3bazzm8ssvR3l5OQ4dOuT9BV+8eDF++9vf4ujRo35/jr8rN3FxcSgsLER4eDiApvdXWUNWxmfPnsXhw4cRGxvr7V9zH1NzOE8X5u6EMTWH82SaJvLy8mrk3pzH5K/95oVr0ZSu3GgAEiNMfHdSwPzxu03lys3f59zk2N8nAMjNzfWZ7819TBc7T0VFRYiKisLJkye9r99W6nXlZurUqWjVqhWefPJJnD59Gvfccw86duyIl156qVaFDQC0bdsWuq4jPz/fpz0/Px9ut9vvYzp06ICWLVv63IK64oor4PF4UFlZiaCgoBqPCQ4ORnBwcI12Xddr3MqqPpn+jrW7XQjht92qj1btQggUFxcjLi7O5/ma85iaw3myyr05j6m5nCd/uTf3MflvF95C4Xyyju0mRPXFFt926X/rk792ISTahwLfnhQ/Finn+PuZ1u0NPyYn/3/PMAy/872ufaxre1P7ffL73LU+8gLjx4/HwYMHcerUKXg8Hhw+fBhTpkyp9eODgoLQr18/bNiwwdtmmiY2bNjgcyXnfEOHDsV3333nUzF+++236NChg9/ChoiIiAJPvYqbQ4cO4eDBgwCAkJAQtG/fHgBw8OBB5OTk1Pp5Zs+ejVdffRVvvfUWDhw4gOnTp6OsrAyTJ08GAEyYMMFnwfH06dNRWFiIWbNm4dtvv8XHH3+MhQsXYsaMGfUZBhERETlQvW5LTZo0Cffeey+6d+/u0759+3a89tpr2LhxY62eZ9y4cTh27BjmzZsHj8eDPn36YO3atd5Fxrm5uT6XreLi4rBu3To89NBD6NWrF2JjYzFr1iw8+uij9RlGQBBCwO12e+/Pkj2YuxrMXQ1TVi1ENvlecrbifLdWrwXF4eHh2LNnD7p16+bT/t1336F///7enQpNUUlJCdq0aVOrBUlERE1RyjMfq+5Cs7Fu7ijVXaAGUpfX73rdlhJC+GzTqnby5Mk67UOnxmcYBrKysnhebMbc1WDuamhCole0CU3w0o2dON+t1au4GT58OBYtWuQTqGEYWLRoEYYNG9ZgnaOG4a8QpcbH3NVg7vYTACJdvlvEyR6c7/7Va83N888/j+HDhyMxMRFXX301AGDLli0oKSnBZ5991qAdJCIiIqqLel256dGjB/bt24exY8eioKAApaWlmDBhAjIyMtCzZ8+G7iMRERFRrdXryg0AdOzYEQsXLmzIvlAjEEIgLi6Oq+ltxtzVYO5qmLLq4xm4W8penO/W6l3cFBcXY8eOHSgoKKjxNswTJky45I5Rw9A0DdHR0aq7EXCYuxrMXQ0JAc9p1b0IPJzv1upV3Pz973/H+PHjcerUKYSHh/tUjUIIFjdNiGEYOHjwILp3716nt66mS8Pc1WDuamhCol97E7sLNMuPbaCGx/lurV5rbn7961/j3nvvxalTp1BcXIyioiLvv/M/4ZOahvM/GZbsw9zVYO72EwBCWoC7pRTgfPevXsXNDz/8gAceeAAhISEN3R8iIiKiS1Kv4iYlJQW7du1q6L4QERERXbJ6rbkZNWoUfvOb32D//v1ITk5Gy5Ytfb5/6623Nkjn6NJpmoauXbtafrQ8NQ7mrgZzV8OQwL7jGgzulrIV57u1ehU306ZNAwA8/fTTNb4nhOBbQTchQgh+hpYCzF0N5q6KQFGF6j4EHs53a/Uq90zTtPzHwqZpMQwDaWlpPC82Y+5qMHc1dCExrKMBnZ8tZSvOd2u8lhUAOPHVYO5qMHc1dG6VUoLz3b96v4lfWVkZNm3ahNzcXFRWVvp874EHHrjkjhERERHVR72Km6+++go333wzTp8+jbKyMkRFReH48eMICQlB+/btWdwQERGRMvW6LfXQQw9h9OjRKCoqQqtWrfDvf/8b33//Pfr164ff/e53Dd1HugSapiExMZGr6W3G3NVg7moYEtiZz91SduN8t1avRPbu3Ytf//rX0DQNuq6joqICcXFxeOGFF/D44483dB/pEgUFBanuQkBi7mowdzUquPRDCc53/+pV3LRs2dJbKbZv3x65ubkAgDZt2iAvL6/hekeXzDRNpKWl1fhwU2pczF0N5q6GLoBhHU0uKrYZ57u1eq25ueqqq7Bz5050794dI0aMwLx583D8+HG8/fbb6NmzZ0P3kYiIiKjW6nXlZuHChejQoQMAYMGCBYiMjMT06dNx7NgxvPLKKw3aQSIiIqK6qNeVm/79+3v/u3379li7dm2DdYiIiIjoUtTrys11112H4uLiGu0lJSW47rrrLrVP1IA0TUNycjJX09uMuavB3NUwJLD1CHdL2Y3z3Vq9Etm4cWONN+4DgPLycmzZsuWSO0UNy9+5osbH3NVg7moE66p7EJg43/2r022pffv2ef97//798Hg83q8Nw8DatWsRGxvbcL2jS2aaJjIzM5GcnAxd5/997MLc1WDuaugCGBBj8uqNzTjfrdWpuOnTpw+EEBBC+L391KpVK7z88ssN1jkiIiKiuqpTcXPo0CFIKdG1a1fs2LED7dq1834vKCgI7du3Z/VIREREStWpuOnUqRPOnDmDiRMnIjo6Gp06dWqsflEDYsGpBnNXg7mrwdtRanC++1fnBcUtW7bEmjVrGqMv1Ah0Xef9WAWYuxrMXQ1DCmw9osOQfItiO3G+W6vXbqnbbrsNH374YQN3hRqDlBIlJSWQkn9W2Ym5q8HcVZGIDJYAmLudON+t1etN/Lp3746nn34aX3zxBfr164fQ0FCf7z/wwAMN0jm6dKZpIjs7m9W9zZi7GsxdDV0Avdpyt5TdON+t1au4ef311xEREYHdu3dj9+7dPt8TQrC4ISIiImXqVdwcOnSooftBRERE1CAu+T2bpZS839fEuVwu1V0ISMxdDeZuPwng9FmuuFGB892/ehc3f/rTn5CcnIxWrVqhVatW6NWrF95+++2G7Bs1AF3XkZSUxPuxNmPuajB3NUwpsDNfh8ndUrbifLdWr+Jm8eLFmD59Om6++Wa8//77eP/993HjjTfi/vvvx+9///uG7iNdAtM0ceLECZimqborAYW5q8Hc1RCQcIdICF67sRXnu7V6rbl5+eWXsWLFCkyYMMHbduutt+LKK6/EU089hYceeqjBOkiXRkqJvLw8REREqO5KQGHuajB3NTQBJEaaOPYf7payE+e7tXpduTl69CiGDBlSo33IkCE4evToJXeKiIiIqL7qVdx069YN77//fo321atXo3v37pfcKSIiIqL6qtdtqdTUVIwbNw6bN2/G0KFDAQBffPEFNmzY4LfoIbXCwsJUdyEgMXc1mLv9JICicq64UYHz3b96FTf//d//je3bt+P3v/+992MYrrjiCuzYsQNXXXVVQ/aPLpGu60hISFDdjYDD3NVg7mqYUmDfCe6Ushvnu7V6FTcA0K9fP/z5z39uyL5QIzBNEwUFBWjfvj007ZLf1ohqibmrwdzVEJCID5PILRWQYJFjF853a/UubgzDwJo1a3DgwAEAQI8ePXDbbbehRYt6PyU1AiklPB4P2rVrp7orAYW5q8Hc1dAE0Dlc4vApwd1SNuJ8t1avSiQ9PR233norPB4PEhMTAQDPP/882rVrh7///e/o2bNng3aSiIiIqLbqdR1r6tSpuPLKK3H48GHs2bMHe/bsQV5eHnr16oX77ruvoftIREREVGv1unKzd+9e7Nq1C5GRkd62yMhILFiwAAMGDGiwztGlE0IgKioKQvA+uJ2YuxrMXQ0pAU+ZAD9m0F6c79bqdeXm8ssvR35+fo32goICdOvW7ZI7RQ1H0zTEx8dzsZnNmLsazF0NEwKZxRpMLia2Fee7tXolsmjRIjzwwAP44IMPcPjwYRw+fBgffPABHnzwQTz//PMoKSnx/iO1TNNEbm4uP3vEZsxdDeauhgaJxAgTGt/pxlac79bqdVvqlltuAQCMHTvWezlM/ng9cvTo0d6vhRAwDKMh+kn1JKVEYWEhYmNjVXcloDB3NZi7GkIA7lCJ704KsL6xD+e7tXoVN59//nlD94OIiIioQdSruBkxYkRD94OIiIioQdT7HffKy8uxb98+FBQU1Ljfd+utt15yx6hhCCHgdru5mt5mzF0N5q6GKYGcEgGTt6RsxflurV7Fzdq1azFhwgQcP368xve4zqZp0TQNbrdbdTcCDnNXg7mrISHwfSlfYO3G+W6tXrulfvWrX+HOO+/E0aNHYZqmzz8WNk2LYRjIysriebEZc1eDuauhCYle0SY0wUs3duJ8t1av4iY/Px+zZ89GTExMQ/eHGkFpaanqLgQk5q4Gc7efABDp4kdmqsD57l+9ips77rgDGzdubOCuEBEREV26eq25Wbp0Ke68805s2bIFycnJaNmypc/3H3jggQbpHBEREVFd1au4+ctf/oJ//vOfcLlc2Lhxo89KbSEEi5smRAiBuLg4rqa3GXNXg7mrYUogs0jjbimbcb5bq1dx88QTTyA1NRWPPfYYP9OiidM0DdHR0aq7EXCYuxrMXQ0JAc9p1b0IPJzv1upVmVRWVmLcuHEsbJoBwzCQkZHB1fQ2Y+5qMHc1NCExIMbgbimbcb5bq1d1MnHiRKxevbqh+0KNpLy8XHUXAhJzV4O5208ACGkB7pZSgPPdv3rdljIMAy+88ALWrVuHXr161VhQvHjx4jo937Jly/Db3/4WHo8HvXv3xssvv4yBAwf+5OPee+893H333bjtttvw4Ycf1ulnEhERkTPVq7hJS0vDVVddBQD45ptvLqkDq1evxuzZs7Fy5UoMGjQIS5YsQUpKCjIzM9G+fXvLx+Xk5ODhhx/G1VdffUk/n4iIiJxFSCmV3iQdNGgQBgwYgKVLlwIATNNEXFwcfvWrX+Gxxx7z+xjDMDB8+HDce++92LJlC4qLi2t95aakpARt2rTByZMnER4e3lDDaLKklCgtLUVYWBhX1NuIuasRKLmnPPOx6i5cQCIyGCiqAJrazal1c0ep7kKjCZT5Xq0ur991unJz++23/+QxQgj87W9/q9XzVVZWYvfu3ZgzZ463TdM0jBw5Etu2bbN83NNPP4327dtjypQp2LJly0V/RkVFBSoqKrxfl5SUAKgqkKoXYQkhoGkaTNPE+bVedfuFi7Ws2jVN8/vZWtULry/8gFGrdl3XIaX0235hH63aq/sopURoaKj3uZwwpuZwni7M3Qljai7nyV/uzX1MF7ZXkdDF+V8BphQQkNBq0y4BEwIaJM5/XTRl1e4nTfi+47BVuyEBQKCksro/8rx2+PTx4u2iUcZkGIajf58unO9OGFNtz9PF1Km4adOmTV0O/0nHjx+HYRg1PsYhJiYGGRkZfh+zdetWvP7669i7d2+tfsaiRYuQmppaoz09PR2tW7cGAERFRSE+Ph6HDx9GYWGh9xi32w23242cnByft7iOi4tDdHQ0Dh486LOYq2vXrggPD8f+/ft9TkJiYiKCgoKQlpbm04fk5GRUVlYiMzPT26brOpKTk1FaWors7Gxvu8vlQlJSEoqKipCXl+dtDwsLQ0JCAgoKCuDxeLzt1WPKy8tDVlYWIiMjvZ8g29zH1BzO04kTJ5CWlubN3Qljag7nqby8HNu3b/fm7oQx+TtPABAZDPRqe+4F5vRZYGe+jpgQIDHyXHtRucC+EwLxYRKdw8+9kHjKBDKLBbpHSLhDz7XnlFR9CGbPKIlI17n2zCINntNAv/YmQs575dh3XENJpcR/dzNQcFqg+hE78zVUGMCwjr4vgluPaAjWgQEx59oNCWw9ojfKmNLS0hz7+9SuXTts27YNoaGh3is3zX1MFztPWVlZqC2lt6WOHDmC2NhYfPnllxg8eLC3/ZFHHsGmTZuwfft2n+NLS0vRq1cvLF++HDfddBMAYNKkSRe9LeXvyk1cXBwKCwu9l7Wa2l9lDVkZnzlzBt988w2uvPJK6LruiDE1h/N0Ye5OGFNzOE+GYSAtLa1G7s15TP7ab164Fk3pyo0ugKs7GvjyqPbjFZimc+Xm73Nucuzvk5QS+/bt85nvzX1MFztPRUVFiIqKavjbUg2tbdu20HUd+fn5Pu35+fl+P8Y9KysLOTk5GD16tLetOuAWLVogMzMTCQkJPo8JDg5GcHBwjefSdd07GapZvW/PhcfZ0V79V+eFrPp4sfbq5zr/+Zr7mBqrj3Vtv9iY/OXe3MfkT1MakxDCb+7NeUzW7cJbKJxP1rHdhAD8tUv/azj8t8sfn194i5tq/n6mdXvDj8nJ/9+rvuXm7/WsuY6pPn30+9y1PrIRBAUFoV+/ftiwYYO3zTRNbNiwwedKTrWkpCSkpaVh79693n+33norrr32WuzduxdxcXF2dp+IiIiaIKVXbgBg9uzZmDhxIvr374+BAwdiyZIlKCsrw+TJkwEAEyZMQGxsLBYtWgSXy4WePXv6PD4iIgIAarRTFU3TkJiYyHeTthlzV4O5q2HIqjU2VldpqHFwvltTXtyMGzcOx44dw7x58+DxeNCnTx+sXbvWu8g4NzeXJ+4SBQUFqe5CQGLuajB3NSr4CQBKcL77p/x9buwWaO9zU73AMjk5uU73K+nSMHc1AiX3pvY+N7qQGNbRxNYjWo01N6o5+X1uAmW+V6vL6zcviRAREZGjsLghIiIiR2FxQ0RERI7C4sbhNE1DcnIyF2XbjLmrwdzVqHqHYe6WshvnuzUmEgAqKytVdyEgMXc1mLsawc5fz9okcb77x+LG4UzTRGZmpsUH71FjYe5qMHc1dFH1WVEXfqQCNS7Od2ssboiIiMhRWNwQERGRo7C4CQCB8OZOTRFzV4O5q8HFxGpwvvun/OMXqHHpuo7k5GTV3Qg4zF0N5q6GIQW2HuGLrN04363xyo3DSSlRUlKCAPuUDeWYuxrMXRWJyGAJgLnbifPdGosbhzNNE9nZ2VxNbzPmrgZzV0MXQK+23C1lN853ayxuiIiIyFFY3BAREZGjsLgJAC6XS3UXAhJzV4O5208COH2WK25U4Hz3j7ulHE7XdSQlJanuRsBh7mowdzVMKbAzn7ul7Mb5bo1XbhzONE2cOHGCC85sxtzVYO5qCEi4QyQEr93YivPdGosbh5NSIi8vj1sFbcbc1WDuamgCSIw0oXG3lK04362xuCEiIiJHYXFDREREjsLiJgCEhYWp7kJAYu5qMHf7SQBF5VxxowLnu3/cLeVwuq4jISFBdTcCDnNXg7mrYUqBfSe44MZunO/WeOXG4UzThMfj4Wp6mzF3NZi7GgISncJM7payGee7NRY3DielhMfj4Wp6mzF3NZi7GpoAOodL7payGee7NRY3RERE5CgsboiIiMhRWNw4nBACUVFREILXi+3E3NVg7mpICXjKBHh3xF6c79a4W8rhNE1DfHy86m4EHOauBnNXw4RAZjFfYO3G+W6NV24czjRN5ObmcjW9zZi7GsxdDQ0SiREmNO6WshXnuzUWNw4npURhYSFX09uMuavB3NUQAnCHSvDuiL04362xuCEiIiJHYXFDREREjsLixuGEEHC73VxNbzPmrgZzV8OUQE6JgMm7I7bifLfG3VIOp2ka3G636m4EHOauBnNXQ0Lg+1K+wNqN890ar9w4nGEYyMrKgmEYqrsSUJi7GsxdDU1I9Io2oQleurET57s1FjcBoLS0VHUXAhJzV4O5208AiHRJ8NqN/Tjf/WNxQ0RERI7C4oaIiIgchcWNwwkhEBcXx9X0NmPuajB3NUwJZBZp3C1lM853a9wt5XCapiE6Olp1NwIOc1eDuashIeA5rboXgYfz3Rqv3DicYRjIyMjganqbMXc1mLsampAYEGNwt5TNON+tsbgJAOXl5aq7EJCYuxrM3X4CQEgLcLeUApzv/rG4ISIiIkdhcUNERESOwuLG4TRNQ9euXaFpPNV2Yu5qMHc1DAnsO67B4JIbW3G+W+NuKYcTQiA8PFx1NwIOc1eDuasiUFShug+Bh/PdGss9hzMMA2lpaVxNbzPmrgZzV0MXEsM6GtC5W8pWnO/WWNwEAE58NZi7GsxdDZ1bpZTgfPePxQ0RERE5CosbIiIichQWNw6naRoSExO5mt5mzF0N5q6GIYGd+dwtZTfOd2tMJAAEBQWp7kJAYu5qMHc1Krj0QwnOd/9Y3DicaZpIS0uDaZqquxJQmLsazF0NXQDDOppcVGwzzndrLG6IiIjIUVjcEBERkaOwuCEiIiJHYXHjcJqmITk5mavpbcbc1WDuahgS2HqEu6XsxvlujYkEgMrKStVdCEjMXQ3mrkawrroHgYnz3T8WNw5nmiYyMzO5mt5mzF0N5q6GLoABMdwtZTfOd2ssboiIiMhRWNwQERGRozSJ4mbZsmXo3LkzXC4XBg0ahB07dlge++qrr+Lqq69GZGQkIiMjMXLkyIseT4Cu82a4CsxdDeauBhcTq8H57p/y4mb16tWYPXs25s+fjz179qB3795ISUlBQUGB3+M3btyIu+++G59//jm2bduGuLg43HDDDfjhhx9s7nnzoOs6kpOT+QtgM+auBnNXw5ACW4/oMCQX3diJ892a8uJm8eLFmDZtGiZPnowePXpg5cqVCAkJwRtvvOH3+HfeeQe//OUv0adPHyQlJeG1116DaZrYsGGDzT1vHqSUKCkpgZT8s8pOzF0N5q6KRGSwBMDc7cT5bk1pcVNZWYndu3dj5MiR3jZN0zBy5Ehs27atVs9x+vRpnDlzBlFRUY3VzWbNNE1kZ2dzNb3NmLsazF0NXQC92nK3lN043621UPnDjx8/DsMwEBMT49MeExODjIyMWj3Ho48+io4dO/oUSOerqKhARUWF9+uSkhIAgGEYMIyqj7EVQkDTNJim6VMBV7dXH/dT7ZqmQQjhtx1AjQlo1a7rOqSUftsv7KNV+4VjunCsThjTT/W9KYzp/J/hlDH9VLvKMUkp/ebenMfkr72K9CkmJABTCghIaLVpl4AJAQ0S4rx2UwISApqQOL9WsWqvXmsjIKELWaP9woLHul00ypgMw3Ds71PVmH3ne3MfU13O08UoLW4u1XPPPYf33nsPGzduhMvl8nvMokWLkJqaWqM9PT0drVu3BgBERUUhPj4ehw8fRmFhofcYt9sNt9uNnJwclJaWetvj4uIQHR2NgwcPory83NvetWtXhIeHY//+/T4nITExEUFBQUhLS/PpQ3JyMiorK5GZmeltq76HWlpaiuzsbG+7y+VCUlISioqKkJeX520PCwtDQkICCgoK4PF4vO3VYzpy5AgKCwuRnp4OIYQjxtQczlNxcbFP7k4YU3M4TxUVFT65O2FM/s4TAEQGV10tqXb6LLAzX0dMCJAYea69qFxg3wmB+DCJzuHnXkg8ZQKZxQLdIyTcoefac0oEvi8V6BklEek6155ZpMFzGujX3kTIea8c+45rKKkEYkKAIR0k5I+3pnbma6gwqj4t/Hxbj2gI1qveF6da1Tsc640yprS0NMf+PrVr1w6lpaXe+e6EMV3sPGVlZaG2hFR4s66yshIhISH44IMPMGbMGG/7xIkTUVxcjI8++sjysb/73e/w7LPP4l//+hf69+9veZy/KzdxcXEoLCxEeHg4gKb3V1lDVsZnzpzBwYMH0a1bN+i67ogxNYfzdGHuThhTczhPhmHg22+/rZF7cx6Tv/abF65FU7pyowmgf3sDXx3TYP64qLipXLn5+5ybHPv7JKVEZmamz3xv7mO62HkqKipCVFQUTp486X39tqK0uAGAQYMGYeDAgXj55ZcBVAUZHx+PmTNn4rHHHvP7mBdeeAELFizAunXr8LOf/axOP6+kpARt2rSpVThERE1RyjMfq+5Cs7Fu7ijVXaAGUpfXb+W7pWbPno1XX30Vb731Fg4cOIDp06ejrKwMkydPBgBMmDABc+bM8R7//PPPY+7cuXjjjTfQuXNneDweeDwenDp1StUQmjTTNHHixAkuOLMZc1eDuashIOEOkRDcLWUrzndrytfcjBs3DseOHcO8efPg8XjQp08frF271rvIODc31+cTT1esWIHKykrccccdPs8zf/58PPXUU3Z2vVmQUiIvLw8RERGquxJQmLsazF0NTVStiTn2H34yuJ04360pL24AYObMmZg5c6bf723cuNHn65ycnMbvEBERETVbym9LERERETUkFjcBICwsTHUXAhJzV4O520+ians270jZj/PdvyZxW4oaj67rSEhIUN2NgMPc1WDuapiy6n1nyF6c79Z45cbhTNOEx+PhanqbMXc1mLsaAhKdwkzulrIZ57s1FjcOJ6WEx+PhB6vZjLmrwdzV0ATQOdz3DfWo8XG+W2NxQ0RERI7C4oaIiIgchcWNwwkhEBUV5f1QNbIHc1eDuashZdWHVvLuiL04361xt5TDaZqG+Ph41d0IOMxdDeauhomqT+Mme3G+W+OVG4czTRO5ublcTW8z5q4Gc1dDg0RihAmNu6VsxflujcWNw0kpUVhYyNX0NmPuajB3NYQA3KESvDtiL853ayxuiIiIyFFY3BAREZGjsLhxOCEE3G43V9PbjLmrwdzVMCWQUyJg8u6IrTjfrXG3lMNpmga32626GwGHuavB3NWQEPi+lC+wduN8t8YrNw5nGAaysrJgGIbqrgQU5q4Gc1dDExK9ok1ogpdu7MT5bo3FTQAoLS1V3YWAxNzVYO72EwAiXRK8dmM/znf/WNwQERGRo7C4ISIiIkdhceNwQgjExcVxNb3NmLsazF0NUwKZRRp3S9mM890ad0s5nKZpiI6OVt2NgMPc1WDuakgIeE6r7kXg4Xy3xis3DmcYBjIyMria3mbMXQ3mroYmJAbEGNwtZTPOd2ssbgJAeXm56i4EJOauBnO3nwAQ0gLcLaUA57t/LG6IiIjIUVjcEBERkaOwuHE4TdPQtWtXaBpPtZ2YuxrMXQ1DAvuOazC45MZWnO/WuFvK4YQQCA8PV92NgMPc1WDuqggUVajuQ+DhfLfGcs/hDMNAWloaV9PbjLmrwdzV0IXEsI4GdO6WshXnuzUWNwGAE18N5q4Gc1dD51YpJTjf/WNxQ0RERI7C4oaIiIgchcWNw2mahsTERK6mtxlzV4O5q2FIYGc+d0vZjfPdGhMJAEFBQaq7EJCYuxrMXY0KLv1QgvPdPxY3DmeaJtLS0mCapuquBBTmrgZzV0MXwLCOJhcV24zz3RqLGyIiInIUFjdERETkKCxuiIiIyFFY3DicpmlITk7manqbMXc1mLsahgS2HuFuKbtxvltjIgGgsrJSdRcCEnNXg7mrEayr7kFg4nz3j8WNw5mmiczMTK6mtxlzV4O5q6ELYEAMd0vZjfPdGosbIiIichQWN0REROQoLG4CgK7zZrgKzF0N5q4GFxOrwfnuXwvVHaDGpes6kpOTVXcj4DB3NZi7GoYU2HqEL7J243y3xis3DielRElJCaTkn1V2Yu5qMHdVJCKDJQDmbifOd2ssbhzONE1kZ2dzNb3NmLsazF0NXQC92nK3lN04362xuCEiIiJHYXFDREREjsLiJgC4XC7VXQhIzF0N5m4/CeD0Wa64UYHz3T/ulnI4XdeRlJSkuhsBh7mrwdzVMKXAznzulrIb57s1XrlxONM0ceLECS44sxlzV4O5qyEg4Q6RELx2YyvOd2ssbhxOSom8vDxuFbQZc1eDuauhCSAx0oTG3VK24ny3xuKGiIiIHIXFDRERETkKi5sAEBYWproLAYm5q8Hc7ScBFJVzxY0KnO/+cbeUw+m6joSEBNXdCDjMXQ3mroYpBfad4IIbu3G+W+OVG4czTRMej4er6W3G3NVg7moISHQKM7lbymac79ZY3DiclBIej4er6W3G3NVg7mpoAugcLrlbymac79ZY3BAREZGjcM0NOULKMx+r7oIPXUgM62ji8X8chiGb1p+z6+aOUt0FIqJGxSs3DieEQFRUFIRoWi+wTicl4CkT4NVie3G+q8H5rgbnu7UmUdwsW7YMnTt3hsvlwqBBg7Bjx46LHv/Xv/4VSUlJcLlcSE5OxieffGJTT5sfTdMQHx8PTWsSpzpgmBDILNZggv/TsRPnuxqc72pwvltTnsjq1asxe/ZszJ8/H3v27EHv3r2RkpKCgoICv8d/+eWXuPvuuzFlyhR89dVXGDNmDMaMGYNvvvnG5p43D6ZpIjc3l6vpbaZBIjHChMbdI7bifFeD810NzndryoubxYsXY9q0aZg8eTJ69OiBlStXIiQkBG+88Ybf41966SXceOON+M1vfoMrrrgCzzzzDPr27YulS5fa3PPmQUqJwsJCrqa3mRCAO1SCV4vtxfmuBue7Gpzv1pQuKK6srMTu3bsxZ84cb5umaRg5ciS2bdvm9zHbtm3D7NmzfdpSUlLw4YcfNmZXa40LW2uPC1uJiKgxKC1ujh8/DsMwEBMT49MeExODjIwMv4/xeDx+j/d4PH6Pr6ioQEVFhffrkydPAgCKiopgGAaAqkVZmqbBNE2fCri6vfq4n2rXNA1ny09DE75VtPnjlxe+B4R1uwBQ8z0jTCkgUPOvI7/tsuo+OISJiv+YMCs0mLJqwZ+EqNFHq/aqPlq1X9qYioqKoOt6jdwB+G2/2Hnyl7uKMVW3ywtyr263Ok8aJM5frtCY56moqMh7j/7Cy9m6rkNK6be9Ic5TXX+fhBB+2/31vfo5SktLvXPLCWPy1362/DQa6v8RDTH3TAFU/Me4YL7/OAYF/987f0xFRUUNdp7u+N36JjEmoOp8CAEM6WDgsb9lev94Vfn/vfP7/v6vfw6gYX+fioqKfhzjT1+pcvxW8EWLFiE1NbVGe+fOne3vjCL/VN0BC1ELVPegca1X3QELTs+d1PiX6g5YcPp8D8TcS0tL0aZNm4seo7S4adu2LXRdR35+vk97fn4+3G6338e43e46HT9nzhyf21imaaKwsBDR0dEBsX2upKQEcXFxyMvLQ3h4uOruBAzmrgZzV4O5qxFouUspUVpaio4dO/7ksUqLm6CgIPTr1w8bNmzAmDFjAFQVHxs2bMDMmTP9Pmbw4MHYsGEDHnzwQW/b+vXrMXjwYL/HBwcHIzg42KctIiKiIbrfrISHhwfE5G9qmLsazF0N5q5GIOX+U1dsqim/LTV79mxMnDgR/fv3x8CBA7FkyRKUlZVh8uTJAIAJEyYgNjYWixYtAgDMmjULI0aMwIsvvohRo0bhvffew65du/DHP/5R5TCIiIioiVBe3IwbNw7Hjh3DvHnz4PF40KdPH6xdu9a7aDg3N9fnDYqGDBmCd999F08++SQef/xxdO/eHR9++CF69uypaghERETUhCgvbgBg5syZlrehNm7cWKPtzjvvxJ133tnIvXKG4OBgzJ8/v8atOWpczF0N5q4Gc1eDuVsTku/+Q0RERA6i/B2KiYiIiBoSixsiIiJyFBY3RERE5CgsboiIiMhRWNw43LJly9C5c2e4XC4MGjQIO3bsUN0lR9u8eTNGjx6Njh07QgjRZD7Q1ekWLVqEAQMGICwsDO3bt8eYMWOQmZmpuluOt2LFCvTq1cv7JnKDBw/Gp59+qrpbAeW5556DEMLnjW2JxY2jrV69GrNnz8b8+fOxZ88e9O7dGykpKSgoKFDdNccqKytD7969sWzZMtVdCSibNm3CjBkz8O9//xvr16/HmTNncMMNN6CsrEx11xztsssuw3PPPYfdu3dj165duO6663DbbbchPT1dddcCws6dO/HKK6+gV69eqrvS5HAruIMNGjQIAwYMwNKlSwFUfbRFXFwcfvWrX+Gxxx5T3DvnE0JgzZo13o8WIfscO3YM7du3x6ZNmzB8+HDV3QkoUVFR+O1vf4spU6ao7oqjnTp1Cn379sXy5cvx7LPPok+fPliyZInqbjUZvHLjUJWVldi9ezdGjhzpbdM0DSNHjsS2bdsU9oyo8Z08eRJA1Qst2cMwDLz33nsoKyuz/Kw/ajgzZszAqFGjfP4fT+c0iXcopoZ3/PhxGIbh/RiLajExMcjIyFDUK6LGZ5omHnzwQQwdOpQfy2KDtLQ0DB48GOXl5WjdujXWrFmDHj16qO6Wo7333nvYs2cPdu7cqborTRaLGyJylBkzZuCbb77B1q1bVXclICQmJmLv3r04efIkPvjgA0ycOBGbNm1igdNI8vLyMGvWLKxfvx4ul0t1d5osFjcO1bZtW+i6jvz8fJ/2/Px8uN1uRb0ialwzZ87EP/7xD2zevBmXXXaZ6u4EhKCgIHTr1g0A0K9fP+zcuRMvvfQSXnnlFcU9c6bdu3ejoKAAffv29bYZhoHNmzdj6dKlqKiogK7rCnvYNHDNjUMFBQWhX79+2LBhg7fNNE1s2LCB98PJcaSUmDlzJtasWYPPPvsMXbp0Ud2lgGWaJioqKlR3w7Guv/56pKWlYe/evd5//fv3x/jx47F3714WNj/ilRsHmz17NiZOnIj+/ftj4MCBWLJkCcrKyjB58mTVXXOsU6dO4bvvvvN+fejQIezduxdRUVGIj49X2DNnmzFjBt5991189NFHCAsLg8fjAQC0adMGrVq1Utw755ozZw5uuukmxMfHo7S0FO+++y42btyIdevWqe6aY4WFhdVYSxYaGoro6GiuMTsPixsHGzduHI4dO4Z58+bB4/GgT58+WLt2bY1FxtRwdu3ahWuvvdb79ezZswEAEydOxJtvvqmoV863YsUKAMA111zj075q1SpMmjTJ/g4FiIKCAkyYMAFHjx5FmzZt0KtXL6xbtw4///nPVXeNAhzf54aIiIgchWtuiIiIyFFY3BAREZGjsLghIiIiR2FxQ0RERI7C4oaIiIgchcUNEREROQqLGyIiInIUFjdERJdICIEPP/xQdTeI6EcsbojIr0mTJkEIUePf+R8vcSnefPNNRERENMhz1dekSZMwZswYpX0goobHj18gIks33ngjVq1a5dPWrl07Rb2xdubMGbRs2VJ1N4ioieCVGyKyFBwcDLfb7fOv+lOHP/roI/Tt2xculwtdu3ZFamoqzp49633s4sWLkZycjNDQUMTFxeGXv/wlTp06BQDYuHEjJk+ejJMnT3qvCD311FMA/N/iiYiI8H42V05ODoQQWL16NUaMGAGXy4V33nkHAPDaa6/hiiuugMvlQlJSEpYvX16n8V5zzTV44IEH8MgjjyAqKgput9vbr2oHDx7E8OHD4XK50KNHD6xfv77G8+Tl5WHs2LGIiIhAVFQUbrvtNuTk5AAAMjIyEBISgnfffdd7/Pvvv49WrVph//79deovEfnH4oaI6mzLli2YMGECZs2ahf379+OVV17Bm2++iQULFniP0TQNf/jDH5Ceno633noLn332GR555BEAwJAhQ7BkyRKEh4fj6NGjOHr0KB5++OE69eGxxx7DrFmzcODAAaSkpOCdd97BvHnzsGDBAhw4cAALFy7E3Llz8dZbb9Xped966y2EhoZi+/bteOGFF/D00097CxjTNHH77bcjKCgI27dvx8qVK/Hoo4/6PP7MmTNISUlBWFgYtmzZgi+++AKtW7fGjTfeiMrKSiQlJeF3v/sdfvnLXyI3NxeHDx/G/fffj+effx49evSoU1+JyIIkIvJj4sSJUtd1GRoa6v13xx13SCmlvP766+XChQt9jn/77bdlhw4dLJ/vr3/9q4yOjvZ+vWrVKtmmTZsaxwGQa9as8Wlr06aNXLVqlZRSykOHDkkAcsmSJT7HJCQkyHfffden7ZlnnpGDBw++6Bhvu+0279cjRoyQw4YN8zlmwIAB8tFHH5VSSrlu3TrZokUL+cMPP3i//+mnn/r0+e2335aJiYnSNE3vMRUVFbJVq1Zy3bp13rZRo0bJq6++Wl5//fXyhhtu8DmeiC4N19wQkaVrr70WK1as8H4dGhoKAPj666/xxRdf+FypMQwD5eXlOH36NEJCQvCvf/0LixYtQkZGBkpKSnD27Fmf71+q/v37e/+7rKwMWVlZmDJlCqZNm+ZtP3v2LNq0aVOn5+3Vq5fP1x06dEBBQQEA4MCBA4iLi0PHjh293x88eLDP8V9//TW+++47hIWF+bSXl5cjKyvL+/Ubb7yByy+/HJqmIT09HUKIOvWTiKyxuCEiS6GhoejWrVuN9lOnTiE1NRW33357je+5XC7k5OTglltuwfTp07FgwQJERUVh69atmDJlCiorKy9a3AghIKX0aTtz5ozfvp3fHwB49dVXMWjQIJ/jqtcI1daFC5OFEDBNs9aPP3XqFPr16+ddB3S+8xdjf/311ygrK4OmaTh69Cg6dOhQp34SkTUWN0RUZ3379kVmZqbfwgcAdu/eDdM08eKLL0LTqpb2vf/++z7HBAUFwTCMGo9t164djh496v364MGDOH369EX7ExMTg44dOyI7Oxvjx4+v63Bq7YorrkBeXp5PMfLvf//b55i+ffti9erVaN++PcLDw/0+T2FhISZNmoQnnngCR48exfjx47Fnzx60atWq0fpOFEi4oJiI6mzevHn405/+hNTUVKSnp+PAgQN477338OSTTwIAunXrhjNnzuDll19GdnY23n77baxcudLnOTp37oxTp05hw4YNOH78uLeAue6667B06VJ89dVX2LVrF+6///5abfNOTU3FokWL8Ic//AHffvst0tLSsGrVKixevLjBxj1y5EhcfvnlmDhxIr7++mts2bIFTzzxhM8x48ePR9u2bXHbbbdhy5YtOHToEDZu3IgHHngAhw8fBgDcf//9iIuLw5NPPonFixfDMIw6L6gmImssboiozlJSUvCPf/wD//znPzFgwAD87Gc/w+9//3t06tQJANC7d28sXrwYzz//PHr27Il33nkHixYt8nmOIUOG4P7778e4cePQrl07vPDCCwCAF198EXFxcbj66qtxzz334OGHH67VGp2pU6fitddew6pVq5CcnIwRI0bgzTffRJcuXRps3JqmYc2aNfjPf/6DgQMHYurUqT7rjgAgJCQEmzdvRnx8PG6//XZcccUVmDJlCsrLyxEeHo4//elP+OSTT/D222+jRYsWCA0NxZ///Ge8+uqr+PTTTxusr0SBTMgLb24TERERNWO8ckNERESOwuKGiIiIHIXFDRERETkKixsiIiJyFBY3RERE5CgsboiIiMhRWNwQERGRo7C4ISIiIkdhcUNERESOwuKGiIiIHIXFDRERETkKixsiIiJylP8PBXOi/XfEUEYAAAAASUVORK5CYII=",
johannes bilk
committed
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create bar plot\n",
"plt.bar(np.arange(len(forrest.featureImportance)), forrest.featureImportance, color='steelblue')\n",
"\n",
"# Add labels and title\n",
"plt.xlabel('Feature Index')\n",
"plt.ylabel('Importance')\n",
"plt.title('Feature Importance')\n",
"\n",
"# Add grid\n",
"plt.grid(True, linestyle='--', alpha=0.6)\n",
"\n",
"# Show plot\n",
"plt.show()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluating predictions\n",
"\n",
"Depending on the task at hand we create a confusion matrix (classification) or simple metrics (regression). Since the number of classes is fixed to two, we don't need to change anything here."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━ evaluation ━━━━━━━━━━━━\n",
"————————— confusion matrix —————————\n",
" Class 0 Class 1 \n",
"····································\n",
johannes bilk
committed
" 49% 0% \n",
"····································\n",
johannes bilk
committed
"———————————————————————————————— scores ———————————————————————————————\n",
" accuracy precision sensitivity miss rate \n",
"·······································································\n",
" Class 0 0.996 0.997 0.994 0.006 \n",
" Class 1 0.996 0.994 0.998 0.003 \n",
johannes bilk
committed
"·······································································\n",
johannes bilk
committed
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
]
}
],
"source": [
"if task == 'regressor':\n",
" metrics = RegressionScores(numClasses=2)\n",
" metrics.calcScores(prediction, validTargets, validLabels)\n",
" print(metrics)\n",
"elif task == 'classifier':\n",
" confusion = ConfusionMatrix(numClasses=2)\n",
" confusion.update(prediction, validLabels)\n",
" confusion.percentages()\n",
" confusion.calcScores()\n",
" print(confusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Saving and Loading a Forrest\n",
"\n",
"Forrests can be converted to dictionaries and then saved as a json file. This allows us to load them and re-use them. Also json is a raw text format, which is neat."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: GradientBoosting, bootstrapping: False\n",
johannes bilk
committed
"————————————————————— tree: 01/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 4 <= -0.88, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 02/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.86, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 03/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 3 <= 2.22, samples: 7\n",
" │ │ ├─feat: 4 <= 0.63, samples: 6\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 1 <= 2.37, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
johannes bilk
committed
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 04/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 1.29, samples: 53\n",
" │ └─╴feat: 1 <= 2.37, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 0.09, samples: 29\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 4 <= 0.49, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 3 <= 2.61, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
johannes bilk
committed
" │ │ └─╴value: 0.0\n",
johannes bilk
committed
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 05/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 47\n",
johannes bilk
committed
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 3 <= 2.29, samples: 9600\n",
" ├─feat: 1 <= 2.33, samples: 4747\n",
" │ ├─feat: 3 <= 2.06, samples: 4694\n",
" │ │ └─╴feat: 0 <= 1.14, samples: 7\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.63, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= 0.35, samples: 8\n",
" │ └─╴value: 0.0\n",
johannes bilk
committed
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.34, samples: 4853\n",
" ├─feat: 1 <= 1.46, samples: 267\n",
" │ ├─feat: 0 <= 0.93, samples: 82\n",
" │ │ ├─feat: 3 <= 2.62, samples: 59\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= -0.32, samples: 23\n",
" │ │ ├─feat: 2 <= -0.30, samples: 4\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= 2.46, samples: 19\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 0 <= -0.64, samples: 185\n",
" │ ├─feat: 3 <= 2.48, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.87, samples: 180\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.86, samples: 4\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 3 <= 3.57, samples: 4586\n",
" ├─feat: 1 <= 0.83, samples: 152\n",
" │ ├─feat: 0 <= -0.07, samples: 9\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"\n"
]
}
],
"source": [
johannes bilk
committed
"ModelIO.save(forrest, 'forrest-test')\n",
"newForrest = ModelIO.load('forrest-test')\n",
"print(newForrest)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━ evaluation ━━━━━━━━━━━━\n",
"————————— confusion matrix —————————\n",
" Class 0 Class 1 \n",
"····································\n",
" 49% 0% \n",
"····································\n",
"\n",
"———————————————————————————————— scores ———————————————————————————————\n",
" accuracy precision sensitivity miss rate \n",
"·······································································\n",
" Class 0 0.996 0.997 0.994 0.006 \n",
" Class 1 0.996 0.994 0.998 0.003 \n",
"·······································································\n",
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
]
}
],
"source": [
"prediction = newForrest.eval(validData)\n",
"\n",
"if task == 'regressor':\n",
" newMetrics = RegressionScores(numClasses=2)\n",
" newMetrics.calcScores(prediction, validTargets, validLabels)\n",
" print(newMetrics)\n",
"elif task == 'classifier':\n",
" newConfusion = ConfusionMatrix(numClasses=2)\n",
" newConfusion.update(prediction, validLabels)\n",
" newConfusion.percentages()\n",
" newConfusion.calcScores()\n",
" print(newConfusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Comment\n",
"\n",
"The forrest works as well as the tree code, because it completely builds on it, thus inhereting all it's problems. The progress bar for training trees is very erratic, because it's always set to the current level and because of the recurvise learning process it's jumping between levels. Also if you are using boosting, every tree will be trained twice."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
},
"vscode": {
"interpreter": {
"hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}