{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Testing the Forrest\n",
"\n",
"## Importing the Basics"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from matplotlib import pyplot as plt\n",
"from machineLearning.metric import ConfusionMatrix, RegressionScores\n",
"from machineLearning.utility import ModelIO\n",
"from machineLearning.rf import (\n",
" RandomForest, DecisionTree,\n",
" Gini, Entropy, MAE, MSE, ODD,\n",
" Mode, Mean, Confidence, Probabilities, AnomalyDetection,\n",
" CART, ID3, C45, RSA,\n",
" AdaBoosting, GradientBoosting,\n",
" Majority, Confidence, Average, Median\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generating Test Data\n",
"\n",
"Here I generate random test data. It's two blocks shifted very slightly in some dimensions. For classifier tasks each block gets a label, for regressor tasks each block gets the average coordinates plus some random value as a traget. It's a very simple dummy data set meant for testing the code.\n",
"\n",
"Here one can change the dimensionallity and amount of the data."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def dataShift(dims):\n",
" offSet = [5, 1.5, 2.5]\n",
" diffLen = abs(len(offSet) - dims)\n",
" offSet.extend([0] * diffLen)\n",
" random.shuffle(offSet)\n",
" return offSet[:dims]\n",
"\n",
"# Initialize some parameters\n",
"totalAmount = 6400\n",
"dims = 5\n",
"evalAmount = totalAmount // 4\n",
"trainAmount = totalAmount - evalAmount\n",
"offSet = dataShift(dims)\n",
"\n",
"# Create covariance matrix\n",
"cov = np.eye(dims) # This creates a covariance matrix with variances 1 and covariances 0\n",
"\n",
"# Generate random multivariate data\n",
"oneData = np.random.multivariate_normal(np.zeros(dims), cov, totalAmount)\n",
"twoData = np.random.multivariate_normal(offSet, cov, totalAmount)\n",
"\n",
"# Split the data into training and evaluation sets\n",
"trainData = np.vstack((oneData[:trainAmount], twoData[:trainAmount]))\n",
"validData = np.vstack((oneData[trainAmount:], twoData[trainAmount:]))\n",
"\n",
"# Labels for classification tasks\n",
"trainLabels = np.hstack((np.zeros(trainAmount), np.ones(trainAmount)))\n",
"validLabels = np.hstack((np.zeros(evalAmount), np.ones(evalAmount)))\n",
"\n",
"# Targets for regression tasks\n",
"trainTargets = np.sum(trainData, axis=1) + np.random.normal(0, 0.1, 2*trainAmount)\n",
"validTargets = np.sum(validData, axis=1) + np.random.normal(0, 0.1, 2*evalAmount)\n",
"\n",
"# Shuffle the training data\n",
"trainIndex = np.random.permutation(len(trainData))\n",
"trainData = trainData[trainIndex]\n",
"trainLabels = trainLabels[trainIndex]\n",
"trainTargets = trainTargets[trainIndex]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating the Forrest\n",
"\n",
"Here the forrest is created. One can set the number of trees and set the maximum depth. Depending on the task, we add a different impurity function and a different leaf function. Finally we add the split algorithm and set the feature percentile. Higher numbers look at more possible splits, but decreases speed. Lower numbers look at less possible splits, speeding up the algorithm. Depending on the data set this can have a strong impact on the performance.\n",
"\n",
"One can set a different depth, leaf function, splitting algorithm and impurity function for each tree. Here in this simple case we create all trees with same parameters."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"task = 'outlier' # 'classifier'/'regressor'/'outlier'\n",
"forrest = RandomForest(bootstrapping=True, retrainFirst=False)\n",
"#forrest.setComponent(AdaBoosting())\n",
"forrest.setComponent(Majority())\n",
"for i in range(5):\n",
" tree = DecisionTree(maxDepth=7, minSamplesSplit=2)\n",
" if task == 'regressor':\n",
" tree.setComponent(MSE())\n",
" tree.setComponent(Mean())\n",
" tree.setComponent(CART(featurePercentile=90))\n",
" elif task == 'classifier':\n",
" tree.setComponent(Entropy())\n",
" tree.setComponent(Mode())\n",
" tree.setComponent(CART(featurePercentile=90))\n",
" elif task == 'outlier':\n",
" tree.setComponent(RSA())\n",
" tree.setComponent(ODD())\n",
" tree.setComponent(AnomalyDetection())\n",
" forrest.append(tree)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Trainining the tree\n",
"\n",
"Again, depending on the task we train the forrest with targets or labels. Then we make a prediction and plot the tree."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tree 1 |\u001b[0m\u001b[31m⣿⣿⣿⣿⣿\u001b[0m\u001b[0m\u001b[31m⡇\u001b[0m | 10%\r"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"tree 1 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 21%\n",
"tree 2 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 10%\n",
"tree 3 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 15%\n",
"tree 4 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 21%\n",
"tree 5 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 07%\n",
"━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: None, bootstrapping: True\n",
"\n",
"—————————————————————————— tree: 1/5 ——————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 57\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 4 <= 1.03, samples: 9600\n",
" ├─feat: 2 <= 3.54, samples: 5667\n",
" │ ├─feat: 4 <= -0.76, samples: 5666\n",
" │ │ ├─feat: 1 <= 4.21, samples: 1184\n",
" │ │ │ ├─feat: 2 <= 1.95, samples: 1182\n",
" │ │ │ │ ├─feat: 1 <= 0.01, samples: 1157\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴feat: 1 <= -0.18, samples: 25\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= -0.93, samples: 4482\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 6.85, samples: 3985\n",
" │ │ ├─feat: 1 <= -2.57, samples: 3931\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 0.0\n",
" └─╴feat: 0 <= -1.60, samples: 3933\n",
" ├─feat: 2 <= 1.64, samples: 210\n",
" │ ├─feat: 1 <= -1.12, samples: 202\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 4.79, samples: 200\n",
" │ │ ├─feat: 1 <= 3.72, samples: 103\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.72, samples: 8\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 2.69, samples: 7\n",
" │ ├─feat: 3 <= -0.72, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 0 <= 1.57, samples: 3723\n",
" ├─feat: 0 <= 0.64, samples: 3491\n",
" │ ├─feat: 3 <= 1.35, samples: 2615\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 6.57, samples: 2212\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.72, samples: 876\n",
" │ ├─feat: 0 <= 1.44, samples: 428\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 2 <= 0.62, samples: 448\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= -2.06, samples: 232\n",
" └─╴value: 0.0\n",
" └─╴feat: 3 <= 6.65, samples: 230\n",
" ├─feat: 3 <= 3.02, samples: 218\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————————— tree: 2/5 ——————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 27\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 4 <= 4.59, samples: 9600\n",
" ├─feat: 2 <= 3.05, samples: 9597\n",
" │ ├─feat: 3 <= 5.44, samples: 9588\n",
" │ │ ├─feat: 2 <= -2.60, samples: 8022\n",
" │ │ │ ├─feat: 4 <= 0.81, samples: 20\n",
" │ │ │ │ ├─feat: 1 <= -0.94, samples: 12\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴feat: 1 <= 0.60, samples: 8\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴feat: 3 <= 5.05, samples: 8002\n",
" │ │ │ ├─feat: 1 <= 4.19, samples: 7280\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 2.78, samples: 9\n",
" │ ├─feat: 4 <= 1.35, samples: 8\n",
" │ │ ├─feat: 0 <= -0.50, samples: 6\n",
" │ │ │ ├─feat: 4 <= 0.72, samples: 4\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————————— tree: 3/5 ——————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 41\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 0 <= 3.53, samples: 9600\n",
" ├─feat: 4 <= 0.63, samples: 9599\n",
" │ ├─feat: 2 <= -1.44, samples: 4509\n",
" │ │ ├─feat: 0 <= 1.21, samples: 322\n",
" │ │ │ ├─feat: 0 <= -2.28, samples: 303\n",
" │ │ │ │ ├─feat: 0 <= -2.46, samples: 5\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴feat: 4 <= 0.61, samples: 298\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 0 <= 2.61, samples: 19\n",
" │ │ │ ├─feat: 4 <= -1.10, samples: 18\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= -0.42, samples: 4187\n",
" │ │ ├─feat: 0 <= -0.58, samples: 1674\n",
" │ │ │ ├─feat: 2 <= -0.64, samples: 441\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 1.48, samples: 1233\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.64, samples: 2513\n",
" │ │ ├─feat: 1 <= 4.31, samples: 643\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 2 <= 0.78, samples: 1870\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 4.42, samples: 5090\n",
" │ ├─feat: 2 <= -3.63, samples: 5077\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= -1.36, samples: 5076\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -2.25, samples: 4962\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 0.0\n",
"\n",
"—————————————————————————— tree: 4/5 ——————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 55\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 2 <= -2.67, samples: 9600\n",
" ├─feat: 3 <= 5.96, samples: 26\n",
" │ ├─feat: 0 <= 1.52, samples: 20\n",
" │ │ ├─feat: 0 <= -0.61, samples: 18\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 3 <= 1.56, samples: 17\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 0 <= -2.68, samples: 9574\n",
" ├─feat: 2 <= 0.19, samples: 34\n",
" │ ├─feat: 0 <= -3.51, samples: 17\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 4.30, samples: 16\n",
" │ │ ├─feat: 4 <= 1.69, samples: 9\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 2 <= 0.64, samples: 17\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 1.20, samples: 13\n",
" │ ├─feat: 4 <= 0.61, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 4 <= 2.02, samples: 9540\n",
" ├─feat: 4 <= -0.28, samples: 8065\n",
" │ ├─feat: 2 <= 0.03, samples: 2045\n",
" │ │ ├─feat: 2 <= -0.38, samples: 1054\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= -0.66, samples: 991\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 2 <= 0.34, samples: 6020\n",
" │ ├─feat: 1 <= 0.99, samples: 3815\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.66, samples: 2205\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 2 <= -0.15, samples: 1475\n",
" ├─feat: 2 <= -0.44, samples: 654\n",
" │ ├─feat: 1 <= 2.78, samples: 504\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 3.39, samples: 150\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 4 <= 3.66, samples: 821\n",
" ├─feat: 1 <= 2.69, samples: 783\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————————— tree: 5/5 ——————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 19\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 3 <= 3.85, samples: 9600\n",
" ├─feat: 4 <= 3.46, samples: 5378\n",
" │ ├─feat: 1 <= -1.00, samples: 5359\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 2.45, samples: 4667\n",
" │ │ ├─feat: 1 <= 0.49, samples: 4108\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 3 <= -0.44, samples: 1560\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -1.19, samples: 559\n",
" │ │ ├─feat: 3 <= 3.28, samples: 87\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.59, samples: 472\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"\n"
]
}
],
"source": [
"if task == 'regressor':\n",
" forrest.train(trainData, trainTargets)\n",
"elif task == 'classifier':\n",
" forrest.train(trainData,trainLabels)\n",
"elif task == 'outlier':\n",
" forrest.train(trainData,trainLabels)\n",
"forrest.bake()\n",
"prediction = forrest.eval(validData)\n",
"print(forrest)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Accuracy(name='tree: 0', accuracy=0.0),\n",
" Accuracy(name='tree: 1', accuracy=0.0),\n",
" Accuracy(name='tree: 2', accuracy=0.0),\n",
" Accuracy(name='tree: 3', accuracy=0.0),\n",
" Accuracy(name='tree: 4', accuracy=0.0)]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"forrest.accuracy(validData, validLabels)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVTElEQVR4nO3de3wU1d0/8M+ZgWTJjVyQLImBEMBEMIgEpCiKF9qIeOHxqWBLH0Kq9KeVqk9qrdQKRgtRailWLtqL2FKt2Av2aatQGwVEEBBElmBSBCJBSAIkJCFpEpg5vz9ilmx2B0lI5mxmP+/Xy9dLTiab73xmMnyZOWdXSCkliIiIiBxCU10AERERUVdic0NERESOwuaGiIiIHIXNDRERETkKmxsiIiJyFDY3RERE5ChsboiIiMhR2NwQERGRo7C5ISIiIkdhc0NERESOwuaGKMi8/PLLEEIE/O/RRx/tlp+5efNmPPHEEzh58mS3vP6FaM3jww8/VF1Kpy1fvhwvv/yy6jKIQkYv1QUQUWBPPvkkBg8e7DN22WWXdcvP2rx5M/Lz8zFr1izExsZ2y88IZcuXL0e/fv0wa9Ys1aUQhQQ2N0RBavLkyRgzZozqMi5IfX09IiMjVZehTENDAyIiIlSXQRRy+FiKqId66623cM011yAyMhLR0dGYMmUKioqKfLbZvXs3Zs2ahbS0NLhcLrjdbnz729/GiRMnvNs88cQT+MEPfgAAGDx4sPcRWGlpKUpLSyGECPhIRQiBJ554wud1hBDYu3cvvvnNbyIuLg4TJkzwfv33v/89srKy0KdPH8THx+Ouu+5CWVlZp/Z91qxZiIqKwqFDh3DLLbcgKioKycnJWLZsGQDA4/HghhtuQGRkJAYNGoRXX33V5/tbH3Vt3LgR/+///T8kJCQgJiYGM2fORHV1td/PW758OUaMGIHw8HAkJSXh/vvv93uEd9111+Gyyy7Djh07cO211yIiIgI/+tGPkJqaiqKiImzYsMGb7XXXXQcAqKqqwsMPP4zMzExERUUhJiYGkydPxscff+zz2uvXr4cQAq+//joWLFiAiy++GC6XCzfeeCM+/fRTv3q3bt2Km2++GXFxcYiMjMTIkSPx3HPP+WxTXFyMr3/964iPj4fL5cKYMWPwf//3fx09FERBiXduiIJUTU0Njh8/7jPWr18/AMCqVauQk5OD7OxsPPPMM2hoaMCKFSswYcIEfPTRR0hNTQUAvP322zhw4AByc3PhdrtRVFSEX/7ylygqKsIHH3wAIQTuuOMO/Pvf/8Yf/vAH/PznP/f+jIsuugjHjh3rcN133nknhg0bhoULF0JKCQBYsGABHn/8cUybNg333HMPjh07hueffx7XXnstPvroo049CjMMA5MnT8a1116LRYsW4ZVXXsGcOXMQGRmJxx57DDNmzMAdd9yBF154ATNnzsT48eP9HvPNmTMHsbGxeOKJJ1BSUoIVK1bgs88+8zYTQEvTlp+fj0mTJuG+++7zbrd9+3a8//776N27t/f1Tpw4gcmTJ+Ouu+7Ct771LSQmJuK6667D9773PURFReGxxx4DACQmJgIADhw4gDfeeAN33nknBg8ejIqKCrz44ouYOHEi9u7di6SkJJ96n376aWiahocffhg1NTVYtGgRZsyYga1bt3q3efvtt3HLLbdgwIABePDBB+F2u/HJJ5/g73//Ox588EEAQFFREa6++mokJyfj0UcfRWRkJF5//XVMnToVf/7zn/Ff//VfHT4eREFFElFQWblypQQQ8D8ppayrq5OxsbFy9uzZPt9XXl4u+/bt6zPe0NDg9/p/+MMfJAC5ceNG79hPf/pTCUAePHjQZ9uDBw9KAHLlypV+rwNAzp8/3/vn+fPnSwDyG9/4hs92paWlUtd1uWDBAp9xj8cje/Xq5Tdulcf27du9Yzk5ORKAXLhwoXesurpa9unTRwoh5GuvveYdLy4u9qu19TWzsrJkc3Ozd3zRokUSgPzrX/8qpZSysrJShoWFya997WvSMAzvdkuXLpUA5EsvveQdmzhxogQgX3jhBb99GDFihJw4caLfeGNjo8/rStmSeXh4uHzyySe9Y++++64EIC+99FLZ1NTkHX/uueckAOnxeKSUUp45c0YOHjxYDho0SFZXV/u8rmma3v+/8cYbZWZmpmxsbPT5+lVXXSWHDRvmVydRT8PHUkRBatmyZXj77bd9/gNa/mV+8uRJfOMb38Dx48e9/+m6jnHjxuHdd9/1vkafPn28/9/Y2Ijjx4/jK1/5CgBg586d3VL3vffe6/Pnv/zlLzBNE9OmTfOp1+12Y9iwYT71dtQ999zj/f/Y2Fikp6cjMjIS06ZN846np6cjNjYWBw4c8Pv+73znOz53Xu677z706tULb775JgDgX//6F5qbm/HQQw9B085eLmfPno2YmBj84x//8Hm98PBw5Obmnnf94eHh3tc1DAMnTpxAVFQU0tPTAx6f3NxchIWFef98zTXXAIB33z766CMcPHgQDz30kN/dsNY7UVVVVXjnnXcwbdo01NXVeY/HiRMnkJ2djX379uHzzz8/730gCkZ8LEUUpK688sqAE4r37dsHALjhhhsCfl9MTIz3/6uqqpCfn4/XXnsNlZWVPtvV1NR0YbVntX/0s2/fPkgpMWzYsIDbt20uOsLlcuGiiy7yGevbty8uvvhi71/kbccDzaVpX1NUVBQGDBiA0tJSAMBnn30GoKVBaissLAxpaWner7dKTk72aT6+jGmaeO6557B8+XIcPHgQhmF4v5aQkOC3/cCBA33+HBcXBwDefdu/fz+Ac6+q+/TTTyGlxOOPP47HH3884DaVlZVITk4+7/0gCjZsboh6GNM0AbTMu3G73X5f79Xr7K/1tGnTsHnzZvzgBz/AqFGjEBUVBdM0cdNNN3lf51zaNwmt2v4l3F7bu0Wt9Qoh8NZbb0HXdb/to6KivrSOQAK91rnG5Rfzf7pT+33/MgsXLsTjjz+Ob3/723jqqacQHx8PTdPw0EMPBTw+XbFvra/78MMPIzs7O+A2Q4cOPe/XIwpGbG6IepghQ4YAAPr3749JkyZZblddXY3CwkLk5+dj3rx53vHWOz9tWTUxrXcG2q8Man/H4svqlVJi8ODBuOSSS877++ywb98+XH/99d4/nzp1CkePHsXNN98MABg0aBAAoKSkBGlpad7tmpubcfDgwXPm35ZVvn/6059w/fXX4ze/+Y3P+MmTJ70Tuzui9dzYs2ePZW2t+9G7d+/zrp+op+GcG6IeJjs7GzExMVi4cCFOnz7t9/XWFU6t/8pv/6/6JUuW+H1P63vRtG9iYmJi0K9fP2zcuNFnfPny5edd7x133AFd15Gfn+9Xi5TSZ1m63X75y1/6ZLhixQqcOXMGkydPBgBMmjQJYWFh+MUvfuFT+29+8xvU1NRgypQp5/VzIiMjA777s67rfpn88Y9/7PScl9GjR2Pw4MFYsmSJ389r/Tn9+/fHddddhxdffBFHjx71e43OrJAjCja8c0PUw8TExGDFihX4n//5H4wePRp33XUXLrroIhw6dAj/+Mc/cPXVV2Pp0qWIiYnxLpM+ffo0kpOT8c9//hMHDx70e82srCwAwGOPPYa77roLvXv3xq233orIyEjcc889ePrpp3HPPfdgzJgx2LhxI/7973+fd71DhgzBT37yE8ydOxelpaWYOnUqoqOjcfDgQaxZswbf+c538PDDD3dZPh3R3NyMG2+8EdOmTUNJSQmWL1+OCRMm4LbbbgPQshx+7ty5yM/Px0033YTbbrvNu93YsWPxrW9967x+TlZWFlasWIGf/OQnGDp0KPr3748bbrgBt9xyC5588knk5ubiqquugsfjwSuvvOJzl6gjNE3DihUrcOutt2LUqFHIzc3FgAEDUFxcjKKiIqxbtw5Ay2T1CRMmIDMzE7Nnz0ZaWhoqKiqwZcsWHD582O99doh6HEWrtIjIQqClz4G8++67Mjs7W/bt21e6XC45ZMgQOWvWLPnhhx96tzl8+LD8r//6LxkbGyv79u0r77zzTnnkyBG/pdFSSvnUU0/J5ORkqWmaz7LwhoYGeffdd8u+ffvK6OhoOW3aNFlZWWm5FPzYsWMB6/3zn/8sJ0yYICMjI2VkZKTMyMiQ999/vywpKelwHjk5OTIyMtJv24kTJ8oRI0b4jQ8aNEhOmTLF7zU3bNggv/Od78i4uDgZFRUlZ8yYIU+cOOH3/UuXLpUZGRmyd+/eMjExUd53331+S62tfraULcv0p0yZIqOjoyUA77LwxsZG+f3vf18OGDBA9unTR1599dVyy5YtcuLEiT5Lx1uXgv/xj3/0eV2rpfqbNm2SX/3qV2V0dLSMjIyUI0eOlM8//7zPNvv375czZ86Ubrdb9u7dWyYnJ8tbbrlF/ulPfwq4D0Q9iZDShll2RERB5OWXX0Zubi62b9/e4z/igoj8cc4NEREROQqbGyIiInIUNjdERETkKJxzQ0RERI7COzdERETkKGxuiIiIyFFC7k38TNPEkSNHEB0dbfmW6ERERBRcpJSoq6tDUlISNO3c92ZCrrk5cuQIUlJSVJdBREREnVBWVoaLL774nNuEXHMTHR0NoCWcmJgYxdV0P8Mw8Nlnn2HQoEGWnyhMXY+5q8Hc1WDuaoRa7rW1tUhJSfH+PX4uIdfctD6KiomJCZnmBmjZ31A4+YMFc1eDuavB3NUI1dzPZ0oJJxQTERGRo7C5ISIiIkdhc+NwQgikpKRwZZjNmLsazF0N5q4Gc7cWcnNuQo2maUhISFBdRshh7mowdzWYuxrM3Rrv3DicYRgoLi72TjwjezB3NZi7GsxdDeZujc1NCGhsbFRdQkhi7mowdzWYuxrMPTA2N0REROQobG6IiIjIUdjcOJymaUhLS/vSz+GgrsXc1WDuajB3NZi7Na6WcjghREi8E3OwYe5qMHc1mLsazN0a2z2HMwwDHo+Hs+ltxtzVYO5qMHc1mLs1NjchgCe+GsxdDeauBnNXg7kHxuaGiIiIHIXNDRERETmKkFJK1UXYqba2Fn379kVNTU1ITMSSUqKxsREul4ufP2Ij5q4Gc1eDuasRarl35O9vrpYKAWFhYapLCEnMXQ3mrkYo5J791D9Ul9COhC4AQwJAcDU36x6fovTn87GUw5mmCY/HA9M0VZcSUpi7GsxdDeauhi6ACUkm9ODqa4ICmxsiIiJyFDY3RERE5ChsboiIiMhR2Nw4nKZpyMzM5GeP2Iy5q8Hc1WDuahgS2HRE+2JCMbXFMzEENDc3qy4hJDF3NZi7GsxdjXBddQXBic2Nw5mmiZKSEq5isBlzV4O5q8Hc1dAFMDaRq6UCYXNDREREjsLmhoiIiByFzU0I0HU+lFWBuavB3NVg7mpwMnFg/PgFh9N1HZmZmarLCDnMXQ3mrgZzV8OQApuOsKkMhHduHE5KidraWoTY56Mqx9zVYO5qMHdVJOLCJQDm3h6bG4czTRMHDhzgKgabMXc1mLsazF0NXQAj+3G1VCBsboiIiMhR2NwQERGRo7C5CQEul0t1CSGJuavB3NVg7vaTABrOcMZNIFwt5XC6riMjI0N1GSGHuavB3NVg7mqYUmB7BVdLBcI7Nw5nmiZOnDjBiX42Y+5qMHc1mLsaAhLuCAnBezd+2Nw4nJQSZWVlXKJpM+auBnNXg7mroQkgPc6ExtVSftjcEBERkaOwuSEiIiJHCYrmZtmyZUhNTYXL5cK4ceOwbdu28/q+1157DUIITJ06tXsL7OGio6NVlxCSmLsazF0N5m4/CaC6kTNuAlHe3KxevRp5eXmYP38+du7cicsvvxzZ2dmorKw85/eVlpbi4YcfxjXXXGNTpT2TrusYMmQIP9TOZsxdDeauBnNXw5QCu09oMCUn3bSnvLlZvHgxZs+ejdzcXAwfPhwvvPACIiIi8NJLL1l+j2EYmDFjBvLz85GWlmZjtT2PaZooLy/nKgabMXc1mLsazF0NAYlB0SZXSwWg9H1umpubsWPHDsydO9c7pmkaJk2ahC1btlh+35NPPon+/fvj7rvvxnvvvXfOn9HU1ISmpibvn2trawG0NEiGYQAAhBDQNA2mafrM9m8db93uy8Y1TYMQIuA4AL9ffKtxXdchpQw43r5Gq/G2NR49ehTx8fHQdd0R+9QTjlP73J2wTz3hOJmmGTD3nrxPgcaDbZ+klH659/R9CnScWt8ur/1nORlf/Cj/cQFA+oxLtNxxEZA+q5wsxyVgQkCDhGgzbsqW1VKDY0wcrde8NZgSkBDQhETbclq+LqAL3xyta7+wfWrNrjuP07kobW6OHz8OwzCQmJjoM56YmIji4uKA37Np0yb85je/wa5du87rZxQUFCA/P99vvKioCFFRUQCA+Ph4DBw4EIcPH0ZVVZV3G7fbDbfbjdLSUtTV1XnHU1JSkJCQgH379qGxsdE7npaWhpiYGOzdu9fnIKSnpyMsLAwej8enhszMTDQ3N6OkpMQ7pus6MjMzUVdXhwMHDnjHXS4XMjIyUF1djbKyMu94dHQ0hgwZgsrKSpSXl3vHW/fpyJEjqKqqQlFREYQQjtinnnCcTp486ZO7E/apJxynpqYmn9ydsE894ThFRkaiurram7sT9inQcYroBTQZwIQk3wZs0xEN4TowNvHsuCGBTUd0xIW3fLhlq4YzwPYKHYkRLcu4W1U3Cuw+ITAwWiI15uxf+OX1AiUnBYbFSrgjz46X1gocPiUQ5wKuGiAhv2i8Sqo1lDcAWf1NRLT5G373cQ3VTcD4Ab4ftLm9QuuWffJ4PF1+nPbv34/zJaTCNyY4cuQIkpOTsXnzZowfP947/sgjj2DDhg3YunWrz/Z1dXUYOXIkli9fjsmTJwMAZs2ahZMnT+KNN94I+DMC3blJSUlBVVUVYmJiADj7X2WnT5/Gnj17MGLECN65sXGf2ufuhH3qCcfJMAx4PB6/3HvyPgUaD7Z9Mk0Tu3fv9sm9p+9ToON088K3Wn52EN25uSbJwOaj2hc/K3ju3Pxtbsvf0V15nKqrqxEfH4+amhrv399WlN656devH3RdR0VFhc94RUUF3G633/b79+9HaWkpbr31Vu9Y6y9Cr169UFJSgiFDhvh8T3h4OMLDw/1eS9d1v8lvrb90gba1e7z1X53tWdV4rtoTEhLQq1cvn2168j71hONklXtP3qeecJw0TQuYe0/eJ6vxYNonIUTA3DvzOsGyT4HHW/7mNqT/ttbjIuC47OC4CeH3IVJSShyt13DGFC1fb7u9xSRjw3I84HCn96ltdt19nAK+9nlv2Q3CwsKQlZWFwsJC75hpmigsLPS5k9MqIyMDHo8Hu3bt8v5322234frrr8euXbuQkpJiZ/k9gqZpGDhwoOVJRN2DuavB3NVg7mqYECg5qfk1NhQEH5yZl5eHnJwcjBkzBldeeSWWLFmC+vp65ObmAgBmzpyJ5ORkFBQUwOVy4bLLLvP5/tjYWADwG6cWpmni8OHDuPjii3nhsRFzV4O5q8Hc1dAgMSxWYt9J/zs3oU55czN9+nQcO3YM8+bNQ3l5OUaNGoW1a9d6JxkfOnSIvywXQEqJqqoqJCcnqy4lpDB3NZi7GsxdDSEAd6TEpzX+j6xCnfLmBgDmzJmDOXPmBPza+vXrz/m9L7/8ctcXRERERD0Wb4kQERGRo7C5cbjW97YRgs9j7cTc1WDuajB3NUzZ8n43Jh9J+QmKx1LUfTRNC7isnroXc1eDuavB3NWQEPisjg1lILxz43CGYWD//v0dettqunDMXQ3mrgZzV0MTEiMTTGiCt27aY3MTAtq+jTXZh7mrwdzVYO72EwDiXJKLwANgc0NERESOwuaGiIiIHIXNjcMJIZCSksJVDDZj7mowdzWYuxqmbPkUcK6W8sfVUg7X+kGCZC/mrgZzV4O5qyEhUN6guorgxDs3DmcYBoqLi7mKwWbMXQ3mrgZzV0MTEmMTDa6WCoDNTQhobGxUXUJIYu5qMHc1mLv9BICIXuBqqQDY3BAREZGjsLkhIiIiR2Fz43CapiEtLQ2axkNtJ+auBnNXg7mrYUhg93ENBqfc+OFqKYcTQiAmJkZ1GSGHuavB3NVg7qoIVDepriE4sc12OMMw4PF4uIrBZsxdDeauBnNXQxcSE5IM6Fwt5YfNTQjgBUcN5q4Gc1eDuauhc6lUQGxuiIiIyFHY3BAREZGjsLlxOE3TkJ6ezlUMNmPuajB3NZi7GoYEtldwtVQgPBNDQFhYmOoSQhJzV4O5q8Hc1WjiVKeA2Nw4nGma8Hg8ME1TdSkhhbmrwdzVYO5q6AKYkGRyUnEAbG6IiIjIUdjcEBERkaOwuSEiIiJHYXPjcJqmITMzk6sYbMbc1WDuajB3NQwJbDrC1VKB8EwMAc3NzapLCEnMXQ3mrgZzVyNcV11BcGJz43CmaaKkpISrGGzG3NVg7mowdzV0AYxN5GqpQNjcEBERkaOwuSEiIiJHYXMTAnSdD2VVYO5qMHc1mLsanEwcWC/VBVD30nUdmZmZqssIOcxdDeauBnNXw5ACm46wqQyEd24cTkqJ2tpaSMn23k7MXQ3mrgZzV0UiLlwCYO7tsblxONM0ceDAAa5isBlzV4O5q8Hc1dAFMLIfV0sFwuaGiIiIHIXNDRERETkKm5sQ4HK5VJcQkpi7GsxdDeZuPwmg4Qxn3ATC1VIOp+s6MjIyVJcRcpi7GsxdDeauhikFtldwtVQgvHPjcKZp4sSJE5zoZzPmrgZzV4O5qyEg4Y6QELx344fNjcNJKVFWVsYlmjZj7mowdzWYuxqaANLjTGhcLeWHzQ0RERE5CpsbIiIichQ2NyEgOjpadQkhibmrwdzVYO72kwCqGznjJhCulnI4XdcxZMgQ1WWEHOauBnNXg7mrYUqB3Sc44SYQ3rlxONM0UV5ezlUMNmPuajB3NZi7GgISg6JNrpYKgM2Nw0kpUV5ezlUMNmPuajB3NZi7GpoAUmMkV0sFwOaGiIiIHIXNDRERETkKmxuHE0IgPj4eQvC+pZ2YuxrMXQ3mroaUQHm9AJ8G+uNqKYfTNA0DBw5UXUbIYe5qMHc1mLsaJgRKTrKhDIR3bhzONE0cOnSIqxhsxtzVYO5qMHc1NEikx5rQuFrKD5sbh5NSoqqqiqsYbMbc1WDuajB3NYQA3JESfBroj80NEREROQqbGyIiInIUNjcOJ4SA2+3mKgabMXc1mLsazF0NUwKltQImnwb64Woph9M0DW63W3UZIYe5q8Hc1WDuakgIfFbHhjIQ3rlxOMMwsH//fhiGobqUkMLc1WDuajB3NTQhMTLBhCZ466Y9NjchoK6uTnUJIYm5q8Hc1WDu9hMA4lwSvHfjj80NEREROQqbGyIiInIUNjcOJ4RASkoKVzHYjLmrwdzVYO5qmBIoqda4WioArpZyOE3TkJCQoLqMkMPc1WDuajB3NSQEyhtUVxGceOfG4QzDQHFxMVcx2Iy5q8Hc1WDuamhCYmyiwdVSAbC5CQGNjY2qSwhJzF0N5q4Gc7efABDRC1wtFQCbGyIiInIUNjdERETkKGxuHE7TNKSlpUHTeKjtxNzVYO5qMHc1DAnsPq7B4JQbP1wt5XBCCMTExKguI+QwdzWYuxrMXRWB6ibVNQQnttkOZxgGPB4PVzHYjLmrwdzVYO5q6EJiQpIBnaul/LC5CQG84KjB3NVg7mowdzV0LpUKKCiam2XLliE1NRUulwvjxo3Dtm3bLLf9y1/+gjFjxiA2NhaRkZEYNWoUVq1aZWO1REREFMyUNzerV69GXl4e5s+fj507d+Lyyy9HdnY2KisrA24fHx+Pxx57DFu2bMHu3buRm5uL3NxcrFu3zubKiYiIKBgJKaXSh3Xjxo3D2LFjsXTpUgCAaZpISUnB9773PTz66KPn9RqjR4/GlClT8NRTT33ptrW1tejbty9qampCYgKclBKNjY1wuVz83BcbMXc1mLsaoZJ79lP/UF1COxIRvYCGM0CwvZXfusendPlrduTvb6WrpZqbm7Fjxw7MnTvXO6ZpGiZNmoQtW7Z86fdLKfHOO++gpKQEzzzzTMBtmpqa0NR0djp5bW0tgJbnw63PiIUQ0DQNpmmiba/XOt7+WbLVuKZpEEIEHAdaGrfzGdd1HVLKgOPta7Qab7tPuq7DMAwIIRyzTz3hOLXN3Sn7FOzHSUoZMPeevE+BxoNtn1p/Rtvce/o+BTpOQMtrtp/n0roU239cAJA+4xKAKQUEJLTzGZeACQENEm37RlO2fM9p0/f1W8YFNCF92p2WGoXf5GPr2i9sn1qz687jdC5Km5vjx4/DMAwkJib6jCcmJqK4uNjy+2pqapCcnIympibouo7ly5fjq1/9asBtCwoKkJ+f7zdeVFSEqKgoAC2PugYOHIjDhw+jqqrKu43b7Ybb7UZpaSnq6uq84ykpKUhISMC+fft83nI8LS0NMTEx2Lt3r89BSE9PR1hYGDwej08NmZmZaG5uRklJiXdM13VkZmairq4OBw4c8I67XC5kZGSguroaZWVl3vHo6GgMGTIElZWVKC8v94633adPP/0U8fHxEEI4Zp+C/ThVVVVh9+7d3tydsE894Tg1Njbigw8+8ObuhH3qCccpMjISmzZtQlxcnLe56en7FOg4RfQCmgxgQpJvA7bpiIZwHRibeHbckMCmIzriwoGR/c6ON5wBtlfoSIwA0uPOjlc3Cuw+ITAwWiI15uxf+OX1AiUnBYbFSrgjz46X1gocPiVwy2AT1Y0C8ovGq6RaQ3kDkNXfRESbv+F3H9dQ3QSMH2D6NCbbK7Ru2SePx9Plx2n//v04X0ofSx05cgTJycnYvHkzxo8f7x1/5JFHsGHDBmzdujXg95mmiQMHDuDUqVMoLCzEU089hTfeeAPXXXed37aB7tykpKSgqqrKe1vLyf8qO336NPbs2YMRI0ZA13VH7FNPOE7tc3fCPvWE49S6JLl97j15nwKNB9s+maaJ3bt3++Te0/cp0HG6eeFbLT87SO7caAK4JsnA5qPaFz8reO7c/G3u5Jbv78LjVF1djfj4+OB/LNWvXz/ouo6Kigqf8YqKCrjdbsvv0zQNQ4cOBQCMGjUKn3zyCQoKCgI2N+Hh4QgPD/cb13Xd+0vY9nUDab+dHeOt/+psz6rGc423vlbb1+vp+9RdNXZ0/Fz7FCj3nr5PgQTTPrU+em2fe0/eJ6vxYNunQLl39HWCbZ/8x1v+5rZ6R+DA4yLguOzguAnR+lTMZ+uW7YW3ufFu3+7PZ2u0Gg843Ol9svO6F/C1z3vLbhAWFoasrCwUFhZ6x0zTRGFhoc+dnC9jmqbP3RkiIiIKXco/fiEvLw85OTkYM2YMrrzySixZsgT19fXIzc0FAMycORPJyckoKCgA0DKHZsyYMRgyZAiamprw5ptvYtWqVVixYoXK3QhamqYhMzOTn/liM+auBnNXg7mr0TIHhp8tFYjy5mb69Ok4duwY5s2bh/LycowaNQpr1671TjI+dOiQzy9MfX09vvvd7+Lw4cPo06cPMjIy8Pvf/x7Tp09XtQtBr7m5GS6XS3UZIYe5q8Hc1WDuaoTrrUvBqS3l73Njt1B7n5vWCZaZmZkdel5JF4a5q8Hc1QiV3IPtfW5aPlvK/OLuDd/npi3eQyQiIiJHYXNDREREjsLmJgQ4+TZxMGPuajB3NZi7GpxMHJjyCcXUvVrf+ZPsxdzVYO5qMHc1DCmw6QibykB458bhpJSora31e3dP6l7MXQ3mrgZzV0UiLlwiwLv7hTzeuelinE1//rpjNn2waP2IEKevHgk2zF0N5q6GLlo+44nvdeOPd26IiIjIUdjcEBERkaOwuXE4iZZ3r+QdS/vx3VrVYO5qMHf78fpujXNuHM6UAtsr+AzcbrquIyMjQ3UZIYe5q8Hc1eD13Rrv3DicgIQ7QkKwt7eVaZo4ceIETNNUXUpIYe5qMHc1eH23xubG4TQBpMeZ0IJroZTjSSlRVlbGpbE2Y+5qMHc1eH23xuaGiIiIHIXNDRERETkKmxuHkwCqG/lEVoXo6GjVJYQk5q4Gc7cfr+/WuFrK4UwpsPsEH8jaTdd1DBkyRHUZIYe5q8Hc1eD13Rrv3DicgMSgaJOz6W1mmibKy8u5esRmzF0N5q4Gr+/WOt3crFq1CldffTWSkpLw2WefAQCWLFmCv/71r11WHF04TQCpMZKz6W0mpUR5eTlXj9iMuavB3NXg9d1ap5qbFStWIC8vDzfffDNOnjwJwzAAALGxsViyZElX1kdERETUIZ1qbp5//nn86le/wmOPPebzCbBjxoyBx+PpsuKIiIiIOqpTzc3BgwdxxRVX+I2Hh4ejvr7+gouiriMlUF4vwLvF9hJCID4+HkLwfrGdmLsazF0NXt+tdaq5GTx4MHbt2uU3vnbtWlx66aUXWhN1IRMCJSc1mOBFx06apmHgwIHQNM7ZtxNzV4O5q8Hru7VOLQXPy8vD/fffj8bGRkgpsW3bNvzhD39AQUEBfv3rX3d1jXQBNEgMi5XYd1LwF8BGpmni8OHDuPjii3nBtxFzV4O5q8Hru7VONTf33HMP+vTpgx//+MdoaGjAN7/5TSQlJeG5557DXXfd1dU10gUQAnBHSnxaI8DVgvaRUqKqqgrJycmqSwkpzF0N5q4Gr+/WOv0mfjNmzMCMGTPQ0NCAU6dOoX///l1ZFxEREVGndKq5OXjwIM6cOYNhw4YhIiICERERAIB9+/ahd+/eSE1N7coaiYiIiM5bpx6Ozpo1C5s3b/Yb37p1K2bNmnWhNVEXMiVQWitg8palrYQQcLvdXD1iM+auBnNXg9d3a51qbj766CNcffXVfuNf+cpXAq6iInUkBD6r0yA52cxWmqbB7XZzcqXNmLsazF0NXt+tdepMFEKgrq7Ob7ympsb7bsUUHDQhMTLBhCbY2tvJMAzs37+fvw82Y+5qMHc1eH231qnm5tprr0VBQYHPiWwYBgoKCjBhwoQuK44unAAQ52Jfr0KgfwBQ92PuajB3+/H6bq1TE4qfeeYZXHvttUhPT8c111wDAHjvvfdQW1uLd955p0sLJCIiIuqITt25GT58OHbv3o1p06ahsrISdXV1mDlzJoqLi3HZZZd1dY1ERERE563T73OTlJSEhQsXdmUt1A1MCZRUa5xNbzMhBFJSUrh6xGbMXQ3mrgav79Y63dycPHkS27ZtQ2VlJUzT9PnazJkzL7gw6hoSAuUNqqsIPZqmISEhQXUZIYe5q8Hc1eD13Vqnmpu//e1vmDFjBk6dOoWYmBifbl0IweYmiGhCIqu/iR2VGkzJf1XZxTAM7Nu3D8OGDYOu66rLCRnMXQ3mrgav79Y6Nefm+9//Pr797W/j1KlTOHnyJKqrq73/VVVVdXWNdAEEgIhe4Gx6BRobG1WXEJKYuxrM3X68vlvrVHPz+eef44EHHvB+7AIRERFRsOhUc5OdnY0PP/ywq2shIiIiumCdmnMzZcoU/OAHP8DevXuRmZmJ3r17+3z9tttu65Li6MIZEth9XIPB2fS20jQNaWlpfDt6mzF3NZi7Gry+W+tUczN79mwAwJNPPun3NSEE34I7qAhUN6muIfQIIRATE6O6jJDD3NVg7qrw+m6lU222aZqW/7GxCS66kJiQZEDnZ4/YyjAMeDwe/j7YjLmrwdzV4PXdGu8hhgCdU+mV4IVeDeauBnNXg9f3wDr9Jn719fXYsGEDDh06hObmZp+vPfDAAxdcGBEREVFndKq5+eijj3DzzTejoaEB9fX1iI+Px/HjxxEREYH+/fuzuSEiIiJlOvVY6n//939x6623orq6Gn369MEHH3yAzz77DFlZWXj22We7uka6AIYEtldwNr3dNE1Deno6V4/YjLmrwdzV4PXdWqfOxF27duH73/8+NE2DrutoampCSkoKFi1ahB/96EddXSNdoCY+ClciLCxMdQkhibmrwdzV4PU9sE41N7179/Z26P3798ehQ4cAAH379kVZWVnXVUcXTBfAhCSTk85sZpomPB6P34fKUvdi7mowdzV4fbfWqTk3V1xxBbZv345hw4Zh4sSJmDdvHo4fP45Vq1bhsssu6+oaiYiIiM5bp+7cLFy4EAMGDAAALFiwAHFxcbjvvvtw7NgxvPjii11aIBEREVFHdOrOzZgxY7z/379/f6xdu7bLCiIiIiK6EJ26c3PDDTfg5MmTfuO1tbW44YYbLrQm6kKGBDYd4Wx6u2mahszMTK4esRlzV4O5q8Hru7VOnYnr16/3e+M+AGhsbMR77713wUVR1wrXVVcQmgL9jlD3Y+5qMHc1eH0PrEOPpXbv3u39/71796K8vNz7Z8MwsHbtWiQnJ3dddXTBdAGMTTTZ3dvMNE2UlJQgMzMTus6rj12YuxrMXQ1e3611qLkZNWoUhBAQQgR8/NSnTx88//zzXVYcERERUUd1qLk5ePAgpJRIS0vDtm3bcNFFF3m/FhYWhv79+7NrJyIiIqU61NwMGjQIp0+fRk5ODhISEjBo0KDuqou6EG9XqsFGXw3mrgZzV4PX98A6PKG4d+/eWLNmTXfUQt3AkAKbjugwJN/C0k66rnP+gQLMXQ3mrgav79Y6tVrq9ttvxxtvvNHFpVD3kIgLlwDY3ttJSona2lpIydztxNzVYO6q8PpupVNv4jds2DA8+eSTeP/995GVlYXIyEifrz/wwANdUhxdOF0AI/txNr3dTNPEgQMH+K9ZmzF3NZi7Gry+W+tUc/Ob3/wGsbGx2LFjB3bs2OHzNSEEmxsiIiJSplPNzcGDB7u6DiIiIqIuccHvlS2l5HPWICYBNJzhE1kVXC6X6hJCEnNXg7nbj9d3a51ubn73u98hMzMTffr0QZ8+fTBy5EisWrWqK2ujLmBKge0VOkzOpreVruvIyMjg/AObMXc1mLsavL5b61Rzs3jxYtx33324+eab8frrr+P111/HTTfdhHvvvRc///nPu7pGugACEu4ICcHe3lamaeLEiRMwTVN1KSGFuavB3NXg9d1ap+bcPP/881ixYgVmzpzpHbvtttswYsQIPPHEE/jf//3fLiuQLowmgPQ4E8f+w9n0dpJSoqysDLGxsapLCSnMXQ3mrgav79Y6defm6NGjuOqqq/zGr7rqKhw9evSCiyIiIiLqrE41N0OHDsXrr7/uN7569WoMGzbsgosiIiIi6qxOPZbKz8/H9OnTsXHjRlx99dUAgPfffx+FhYUBmx5SRwKobuQTWRWio6NVlxCSmLsazN1+vL5b61Rz89///d/YunUrfv7zn3s/huHSSy/Ftm3bcMUVV3RlfXSBTCmw+wRn0ttN13UMGTJEdRkhh7mrwdzV4PXdWqeaGwDIysrC73//+66shbqBgMTAaIlDdQIS/CWwi2maqKysRP/+/aFpF/x2UnSemLsazF0NXt+tdbq5MQwDa9aswSeffAIAGD58OG6//Xb06tXpl6RuoAkgNUbi8CnB2fQ2klKivLwcF110kepSQgpzV4O5q8Hru7VOtdhFRUW45JJLkJOTgzVr1mDNmjXIycnBsGHDsGfPng6/3rJly5CamgqXy4Vx48Zh27Ztltv+6le/wjXXXIO4uDjExcVh0qRJ59yeiIiIQkunmpt77rkHI0aMwOHDh7Fz507s3LkTZWVlGDlyJL7zne906LVWr16NvLw8zJ8/Hzt37sTll1+O7OxsVFZWBtx+/fr1+MY3voF3330XW7ZsQUpKCr72ta/h888/78yuEBERkcN0qrnZtWsXCgoKEBcX5x2Li4vDggUL8NFHH3XotRYvXozZs2cjNzcXw4cPxwsvvICIiAi89NJLAbd/5ZVX8N3vfhejRo1CRkYGfv3rX8M0TRQWFnZmVxxPSqC8XoAf/2UvIQTi4+MhBJ+D24m5q8Hc1eD13VqnJshccsklqKiowIgRI3zGKysrMXTo0PN+nebmZuzYsQNz5871jmmahkmTJmHLli3n9RoNDQ04ffo04uPjA369qakJTU1N3j/X1tYCaJkzZBgGgJZfTE3TYJqmz4eAto63bvdl460T6XThe6a1PgvV2/3eW48LANJnXKJlZryAhHY+4xIwv5hg9mmNgBCADglTAhICmvCdfmY13lKj6JZ9MgwDuq775Q4g4HhXHichhOXxa/8W8lbjuq5DShlwHACSk5MhpfT+nJ6+Tz3hOAkhAubek/cp0Hiw7ZOmaX659/R9CnScWj+iUsW1XINE297R/GJ8Xw281/fWcbuv5YFqt+O6dy6dam4KCgrwwAMP4IknnsBXvvIVAMAHH3yAJ598Es8884y3gQCAmJgYy9c5fvw4DMNAYmKiz3hiYiKKi4vPq5Yf/vCHSEpKwqRJkyxrzc/P9xsvKipCVFQUACA+Ph4DBw7E4cOHUVVV5d3G7XbD7XajtLQUdXV13vGUlBQkJCRg3759aGxs9I6npaUBAMYPMH0O/PYKDU0GMCHJ95d20xEN4TowNvHsuCGBTUd0xIUDI/udHW84A2yv0JEY0fJ2262qG1uWAg6MlkiNOXuSlNcLlJwUuCTWRHqcRE1zy3shlNYKfFYncFm8RJzr7PYl1RrKG4Cs/iYi2pwVu49rqG7qnn3at28fMjIyUF1djbKyMu94dHQ0hgwZgsrKSpSXl3vHu/I4xcTEYO/evT6/LOnp6QgLC4PH4/HZp8zMTDQ3N6OkpMQ7pus6MjMzUVdXhwMHDnjHXS4XMjIycOLECZSUlCAyMhJCCEfsU084Tk1NTdi5c6c3dyfsU084TlFRUdi6dSv69OnjvXvT0/cp0HGK6AVl1/JhsRLuyLPjpbUCZXXADRebOGOe/WRwFdfyQPvk8Xi6/Djt378f50vI9u3weWi71K/1RG59mbZ/Dtz5nnXkyBEkJydj8+bNGD9+vHf8kUcewYYNG7B169Zz1vH0009j0aJFWL9+PUaOHBlwm0B3blJSUlBVVeVtvLqy27/pJ28G1Z2b3sLE1UkmNh/VYEgRVHdu/jZ3clD9q6wr/6V5+vRp7NmzByNGjPDeyenp+9QTjpNhGPB4PH659+R9CjQebPtkmiZ2797tk3tP36dAx+nmhW+1/OwguXOjCeCaJMN7fW8dD4Y7N3+bO7nl+7vwOFVXVyM+Ph41NTXnvHECdPLOzbvvvtuZb/PTr18/6LqOiooKn/GKigq43e5zfu+zzz6Lp59+Gv/6178sGxsACA8PR3h4uN+4ruveX8JWVu/P0H67Lxs3LD5+3mqpXuDxwEv7ZAfHTYgvviZ86jItarQa7459an8RbK+j4x09Tl0x3np3IFCNrV9r+/Wevk+BBNM+CSEC5t6T98lqPNj2KVDuHX2dYNsn//GW66Cqa7n/WxHLgNd3wN5r+dlqzo7bed0LpFPNzcSJEzvzbX7CwsKQlZWFwsJCTJ06FQC8k4PnzJlj+X2LFi3CggULsG7dOowZM6ZLaiEiIiJn6PQ77jU2NmL37t2orKz0u4142223nffr5OXlIScnB2PGjMGVV16JJUuWoL6+Hrm5uQCAmTNnIjk5GQUFBQCAZ555BvPmzcOrr76K1NRU73O8qKgo7xwaOsuULc9mTc6mt5UQAm63m6tHbMbc1WDuavD6bq1Tzc3atWsxc+ZMHD9+3O9rXzbPpr3p06fj2LFjmDdvHsrLyzFq1CisXbvWO8n40KFDPreuVqxYgebmZnz961/3eZ358+fjiSee6MzuOJpEywRispemaV/6aJW6HnNXg7mrweu7tU41N9/73vdw5513Yt68eX4rnTpjzpw5lo+h1q9f7/Pn0tLSC/55oUQTEpfFS+ypEpbPYKnrGYaB0tJSpKamdug5MV0Y5q4Gc1eD13drnXoTv4qKCuTl5XVJY0PdSwCIc/Ej1VRou5SR7MPc1WDu9uP13Vqnmpuvf/3rfndUiIiIiIJBpx5LLV26FHfeeSfee+89ZGZmonfv3j5ff+CBB7qkOCIiIqKO6lRz84c//AH//Oc/4XK5sH79ep8Z8kIINjdBxJQt71jJ2fT2EkIgJSWFq0dsxtzVYO5q8PpurVPNzWOPPYb8/Hw8+uijlm/CQ8FBQqC8QXUVoUfTNCQkJKguI+QwdzWYuxq8vlvrVGfS3NyM6dOns7HpATQhMTbRgCbY2tvJMAwUFxd36G0R6MIxdzWYuxq8vlvrVHeSk5OD1atXd3Ut1A0EWj7sjTeL7df2g/jIPsxdDeZuP17frXXqsZRhGFi0aBHWrVuHkSNH+k0oXrx4cZcUR0RERNRRnWpuPB4PrrjiCgDAnj17urQgIiIioguh9FPBqfsZEth9XLP8xFfqHpqmIS0tjfPSbMbc1WDuavD6bq1Dzc0dd9zxpdsIIfDnP/+50wVRVxOoblJdQ+gRQiAmJkZ1GSGHuavB3FXh9d1Kh9rsvn37ful/PMGDiy4kJiQZ0Dmb3laGYcDj8XD1iM2YuxrMXQ1e36116M7NypUru6sO6kY6p9IrwQu9GsxdDeauBq/vgfEBKRERETkKmxsiIiJyFDY3DmdIYHsFZ9PbTdM0pKenc/WIzZi7GsxdDV7frfFMDAFNfBSuRFhYmOoSQhJzV4O5q8Hre2BsbhxOF8CEJJOTzmxmmiY8Hg9M01RdSkhh7mowdzV4fbfG5oaIiIgchc0NEREROQqbGyIiInIUNjcOZ0hg0xHOprebpmnIzMzk6hGbMXc1mLsavL5b45kYAsJ11RWEpubmZtUlhCTmrgZzV4PX98DY3DicLoCxiZxNbzfTNFFSUsLVIzZj7mowdzV4fbfG5oaIiIgchc0NEREROQqbmxDAyWZq6DofhqvA3NVg7mrw+h5YL9UFUPcypMCmI7zo2E3XdWRmZqouI+QwdzWYuxq8vlvjnRvHk4gLlwDY3ttJSona2lpIydztxNzVYO6q8Ppuhc2Nw+kCGNmPs+ntZpomDhw4wNUjNmPuajB3NXh9t8bmhoiIiByFzQ0RERE5Cpsbh5MAGs7wiawKLpdLdQkhibmrwdztx+u7Na6WcjhTCmyv4Gx6u+m6joyMDNVlhBzmrgZzV4PXd2u8c+NwAhLuCAnB3t5WpmnixIkTnGBpM+auBnNXg9d3a2xuHE4TQHqcCY2z6W0lpURZWRmXxtqMuavB3NXg9d0amxsiIiJyFDY3RERE5ChsbhxOAqhu5BNZFaKjo1WXEJKYuxrM3X68vlvjaimHM6XA7hPOfyCb/dQ/VJdgoVh1AX7WPT5FdQndRtd1DBkyRHUZIYe5qxEq1/fO4J0bhxOQGBRtcja9zZi7GqZpory8nKt2bMbc1eB1xhqbG4fTBJAaIzmb3mbMXQ0pJcrLy7lqx2bMXQ1eZ6yxuSEiIiJHYXNDREREjsLmxuGkBMrrBXi32F7MXQ0hBOLj4yEE79PbibmrweuMNa6WcjgTAiUnecGxG3NXQ9M0DBw4UHUZIYe5q8HrjDXeuXE4DRLpsSY0zqa3FXNXwzRNHDp0iKt2bMbc1eB1xhqbG4cTAnBHSvBusb2YuxpSSlRVVXHVjs2Yuxq8zlhjc0NERESOwuaGiIiIHIXNjcOZEiitFTB5t9hWzF0NIQTcbjdX7diMuavB64w1rpZyOAmBz+p4wbEbc1dD0zS43W7VZYQc5q4GrzPWeOfG4TQhMTLBhCbY2tuJuathGAb2798PwzBUlxJSmLsavM5YY3PjcAJAnEuCvb29mLs6dXV1qksISczdfrzOWGNzQ0RERI7C5oaIiIgchc2Nw5kSKKnWOJveZsxdDSEEUlJSuGrHZsxdDV5nrHG1lMNJCJQ3qK4i9DB3NTRNQ0JCguoyQg5zV4PXGWu8c+NwmpAYm2hwNr3NmLsahmGguLiYq3ZsxtzV4HXGGpsbhxMAInqBs+ltxtzVaWxsVF1CSGLu9uN1xhqbGyIiInIUNjdERETkKJxQ7HCGBHYf12DwkaytQiX37Kf+obqEdiTiwoHqvx1GsN2sX/f4FNUldBtN05CWlgZN47+X7RQq15nOYHPjeALVTaprCEXMXQ3mroIQAjExMarLCEE8362wzXY4XUhMSDKgcza9rZi7GsxdDcMw4PF4uFrKZjzfrbG5CQF6cN2dDxnMXQ3mrgYbGzV4vgfG5oaIiIgchc0NEREROQqbG4czJLC9grPp7cbc1WDuamiahvT0dK6WshnPd2s8E0NAEx+FK8Hc1WDuaoSFhakuISTxfA9MeXOzbNkypKamwuVyYdy4cdi2bZvltkVFRfjv//5vpKamQgiBJUuW2FdoD6ULYEKSyUlnNmPuajB3NUzThMfjgWmaqksJKTzfrSltblavXo28vDzMnz8fO3fuxOWXX47s7GxUVlYG3L6hoQFpaWl4+umn4Xa7ba6WiIiIegKlzc3ixYsxe/Zs5ObmYvjw4XjhhRcQERGBl156KeD2Y8eOxU9/+lPcddddCA8Pt7laIiIi6gmUvUNxc3MzduzYgblz53rHNE3DpEmTsGXLli77OU1NTWhqOvsWjrW1tQBa3pOh9X0ZhBDQNA2maULKszOzWsfbv3+D1XjrZLr2b6jUOtmr/a1D63EBQPqMSwCmFBCQ0M5nXAImBDRICEhvTaYEJAQ0IX3enN5qvKVG0S37ZBgGdF33yx1AwPFzHScAQbFPbY9H29zP6zi1Ge/O42QYhvdcbf8YQdd1SCkDjgc6Ti0/Vf0+nR33zb1lvIt/nzq5T625CyEsrx3tc++q49TR36fOXPeklD5f6+n7FOg4tZwhCq/l7c49AH7nu+rrXmvtrdl153E6F2XNzfHjx2EYBhITE33GExMTUVxc3GU/p6CgAPn5+X7jRUVFiIqKAgDEx8dj4MCBOHz4MKqqqrzbuN1uuN1ulJaWoq6uzjuekpKChIQE7Nu3D42Njd7xtLQ0AMD4Ab7PQLdXaGgyWp6NtrXpiIZwHRibeHbckMCmIzriwoGR/c6ON5wBtlfoSIwA0uPOjlc3Cuw+ITAwWiI15uxJUl4vUHJSIK2vBCAwfoAEIFFaK/BZncBl8RJxrrPbl1RrKG8AsvqbiGhzVuw+rqG6qXv2ad++fcjIyEB1dTXKysq849HR0RgyZAgqKytRXl7uHT/XcQIQFPvUepz69YFP7l92nIbFSrgjz45353HyeDzIzMxEc3MzSkpKvOO6riMzMxN1dXU4cOCAd9zlclkeJwBBsU/A2ePUNveu/n26kH3yeDxIS0tDTEwM9u7d63OhTk9PR1hYGDwej88+ddVx6ujvU0eve9HR0dA0DXv37nXMPgU6ThG9oOxabnXuVTedPd8Btde9tvvk8Xi6/Djt378f50vIQP8Us8GRI0eQnJyMzZs3Y/z48d7xRx55BBs2bMDWrVvP+f2pqal46KGH8NBDD51zu0B3blJSUlBVVeX9LJSu7PZv+smbQXbnxkRk75aTDxBBdefmb3Mnd9m/yiYveCso9uns8TAR1Sb3YLpz87e5k7vsX883L1wbFPt0dlwiuk3uLePBceemNXen3rmpr6+Hy+U6eye1h+9ToON088K3Wn52kNy5kQCiekv8p835Hix3bv42d3LL93fhcaqurkZ8fDxqamq+9LPMlN256devH3RdR0VFhc94RUVFl04WDg8PDzg/R9d16LruM2b1Hg3tt/uy8ZaDH2g8cI2Bx0XAcdnBcSEERvc3semI5lOXaVGj1Xh37FNrfla5d3Q8GPaplWaRu9VxMiFa/+HlO94N+9T2vA10DgshAo5b5R4M+9RKFwiYe1f9Pl3IPn1Z7h0d7+hx6uh4R2oxDAOffvopMjMz/b7eU/cp8Hhrwxxw8269lgc693QhkdVfBjjf1Vz32tbeNrvuPk4BX/u8t+xiYWFhyMrKQmFhoXfMNE0UFhb63MkhIiIi6ghld24AIC8vDzk5ORgzZgyuvPJKLFmyBPX19cjNzQUAzJw5E8nJySgoKADQMgm59Zluc3MzPv/8c+zatQtRUVEYOnSosv0gIiKi4KG0uZk+fTqOHTuGefPmoby8HKNGjcLatWu9k4wPHTrkc9vqyJEjuOKKK7x/fvbZZ/Hss89i4sSJWL9+vd3l9xh8a241mLsazF2NjjwyoK7D8z0wpc0NAMyZMwdz5swJ+LX2DUtqamrApahkzZACm47womM35q4Gc1ejdbUT2YvnuzXlH79A3U0iLvzsMkGyC3NXg7mrIKVEbW0t//FpO57vVtjcOJwuWt6PgJ89Yi/mrgZzV8M0TRw4cICfLWUznu/W2NwQERGRo7C5ISIiIkdhc+NwEi3v1sonsvZi7mowd3VcLpfqEkIOz3dryldLUfcypcD2Cs6mtxtzV4O5q6HrOjIyMlSXEXJ4vlvjnRuHE5BwR8gvPt2I7MLc1WDuapimiRMnTnBCsc14vltjc+Nwmmj5lFaNs+ltxdzVYO5qSClRVlbGpeA24/lujc0NEREROQqbGyIiInIUNjcOJwFUN/KJrN2YuxrMXZ3o6GjVJYQcnu/WuFrK4UwpsPsEH8jajbmrwdzV0HUdQ4YMUV1GyOH5bo13bhxOQGJQtMnZ9DZj7mowdzVM00R5eTlXS9mM57s1NjcOpwkgNUZyNr3NmLsazF0NKSXKy8u5WspmPN+tsbkhIiIiR2FzQ0RERI7C5sbhpATK6wV4t9hezF0N5q6GEALx8fEQgs9H7MTz3RpXSzmcCYGSk7zg2I25q8Hc1dA0DQMHDlRdRsjh+W6Nd24cToNEeqwJjbPpbcXc1WDuapimiUOHDnG1lM14vltjc+NwQgDuSAneLbYXc1eDuashpURVVRVXS9mM57s1NjdERETkKGxuiIiIyFHY3DicKYHSWgGTd4ttxdzVYO5qCCHgdru5WspmPN+tcbWUw0kIfFbHC47dmLsazF0NTdPgdrtVlxFyeL5b450bh9OExMgEE5pga28n5q4Gc1fDMAzs378fhmGoLiWk8Hy3xubG4QSAOJcEe3t7MXc1mLs6dXV1qksIOTzfrbG5ISIiIkdhc0NERESOwubG4UwJlFRrnE1vM+auBnNXQwiBlJQUrpayGc93a1wt5XASAuUNqqsIPcxdDeauhqZpSEhIUF1GyOH5bo13bhxOExJjEw3OprcZc1eDuathGAaKi4u5WspmPN+tsblxOAEgohc4m95mzF0N5q5OY2Oj6hJCDs93a2xuiIiIyFHY3BAREZGjsLlxOEMCu49rMPhI1lbMXQ3mroamaUhLS4Om8a8UO/F8t8bVUo4nUN2kuoZQxNzVYO4qCCEQExOjuowQxPPdCttsh9OFxIQkAzpn09uKuavB3NUwDAMej4erpWzG890am5sQoHMqvRLMXQ3mrgYbGzV4vgfG5oaIiIgchc0NEREROQqbG4czJLC9grPp7cbc1WDuamiahvT0dK6WshnPd2s8E0NAEx+FK8Hc1WDuaoSFhakuISTxfA+MzY3D6QKYkGRy0pnNmLsazF0N0zTh8XhgmqbqUkIKz3drbG6IiIjIUdjcEBERkaOwuSEiIiJHYXPjcIYENh3hbHq7MXc1mLsamqYhMzOTq6VsxvPdGs/EEBCuq64gNDF3NZi7Gs3NzapLCEk83wNjc+NwugDGJnI2vd2YuxrMXQ3TNFFSUsLVUjbj+W6NzQ0RERE5CpsbIiIichQ2NyGAk83UYO5qMHc1dJ2TP1Tg+R5YL9UFUPcypMCmI7zo2I25q8Hc1dB1HZmZmarLCDk8363xzo3jScSFSwBs7+3F3NVg7ipIKVFbWwspmbu9eL5bYXPjcLoARvbjbHq7MXc1mLsapmniwIEDXC1lM57v1tjcEBERkaNwzg0RUQ+T/dQ/VJfgQxcSE5JM/Ojvh2HI4LqNsO7xKapLIAV458bhJICGM3wiazfmrgZzV4O5q8HcrfHOjcOZUmB7BWfT2425q8Hc1WDuajB3a7xz43ACEu4ICcHe3lbMXQ3mrgZzV4O5W2Nz43CaANLjTGjB9Rjc8Zi7GsxdDeauBnO3xuaGiIiIHIXNDRERETkKmxuHkwCqG/lE1m7MXQ3mrgZzV4O5W+NqKYczpcDuE3wgazfmrgZzV4O5q8HcrfHOjcMJSAyKNjmb3mbMXQ3mrgZzV4O5W2Nz43CaAFJjJGfT24y5q8Hc1WDuajB3a2xuiIiIyFHY3BAREZGjsLlxOCmB8noByUeytmLuajB3NZi7GszdWlA0N8uWLUNqaipcLhfGjRuHbdu2nXP7P/7xj8jIyIDL5UJmZibefPNNmyrteUwIlJzUYIIPZe3E3NVg7mowdzWYuzXlzc3q1auRl5eH+fPnY+fOnbj88suRnZ2NysrKgNtv3rwZ3/jGN3D33Xfjo48+wtSpUzF16lTs2bPH5sp7Bg0S6bEmNM6mtxVzV4O5q8Hc1WDu1pQ3N4sXL8bs2bORm5uL4cOH44UXXkBERAReeumlgNs/99xzuOmmm/CDH/wAl156KZ566imMHj0aS5cutbnynkEIwB0pIdjY24q5q8Hc1WDuajB3a0qbm+bmZuzYsQOTJk3yjmmahkmTJmHLli0Bv2fLli0+2wNAdna25fZEREQUWpS+Q/Hx48dhGAYSExN9xhMTE1FcXBzwe8rLywNuX15eHnD7pqYmNDU1ef9cU1MDAKiuroZhGAAAIQQ0TYNpmpBtZma1jrdu92XjmqbhTGMDNOF7i9D84o/t34vAelwA8H/vAlMKCPh36QHHZcvzWAgTTf8xYTZpMGXLxDMJ4Vej1XhLjVbjF7ZP1dXV0HXdL3cAAcfPdZwC5a5in1rHZbvcW8etjpMGibaPzbvzOFVXV0PTWv5dY5qmz/a6rkNKGXA80HE609gQFPvUOm4KGTD3rvp9upB9as1dCBHw2gH4Hw+r8TONDUGxTy2vDZgCaPqP0S73L/ZBwXWv7T5VV1d36loe6DidaawPin0CWo5HoNxVXvfa1l5dXQ2g49fycx2n1tdsfx0KxPEfv1BQUID8/Hy/8dTUVPuLUeSfqguwEL9AdQXd623VBVhweu7/Ul2ABeauBnNXoztzr6urQ9++fc+5jdLmpl+/ftB1HRUVFT7jFRUVcLvdAb/H7XZ3aPu5c+ciLy/P+2fTNFFVVYWEhASIEHhQWVtbi5SUFJSVlSEmJkZ1OSGDuavB3NVg7mqEWu5SStTV1SEpKelLt1Xa3ISFhSErKwuFhYWYOnUqgJbmo7CwEHPmzAn4PePHj0dhYSEeeugh79jbb7+N8ePHB9w+PDwc4eHhPmOxsbFdUX6PEhMTExInf7Bh7mowdzWYuxqhlPuX3bFppfyxVF5eHnJycjBmzBhceeWVWLJkCerr65GbmwsAmDlzJpKTk1FQUAAAePDBBzFx4kT87Gc/w5QpU/Daa6/hww8/xC9/+UuVu0FERERBQnlzM336dBw7dgzz5s1DeXk5Ro0ahbVr13onDR86dMg7qQ4ArrrqKrz66qv48Y9/jB/96EcYNmwY3njjDVx22WWqdoGIiIiCiPLmBgDmzJlj+Rhq/fr1fmN33nkn7rzzzm6uyhnCw8Mxf/58v0dz1L2YuxrMXQ3mrgZztybk+aypIiIiIuohlL9DMREREVFXYnNDREREjsLmhoiIiByFzQ0RERE5Cpsbh1u2bBlSU1Phcrkwbtw4bNu2TXVJjrZx40bceuutSEpKghACb7zxhuqSQkJBQQHGjh2L6Oho9O/fH1OnTkVJSYnqshxvxYoVGDlypPdN5MaPH4+33npLdVkh5emnn4YQwueNbYnNjaOtXr0aeXl5mD9/Pnbu3InLL78c2dnZqKysVF2aY9XX1+Pyyy/HsmXLVJcSUjZs2ID7778fH3zwAd5++22cPn0aX/va11BfX6+6NEe7+OKL8fTTT2PHjh348MMPccMNN+D2229HUVGR6tJCwvbt2/Hiiy9i5MiRqksJOlwK7mDjxo3D2LFjsXTpUgAtH22RkpKC733ve3j00UcVV+d8QgisWbPG+9EiZJ9jx46hf//+2LBhA6699lrV5YSU+Ph4/PSnP8Xdd9+tuhRHO3XqFEaPHo3ly5fjJz/5CUaNGoUlS5aoLito8M6NQzU3N2PHjh2YNGmSd0zTNEyaNAlbtmxRWBlR96upqQHQ8hct2cMwDLz22muor6+3/Kw/6jr3338/pkyZ4nONp7OC4h2KqesdP34chmF4P8aiVWJiIoqLixVVRdT9TNPEQw89hKuvvpofy2IDj8eD8ePHo7GxEVFRUVizZg2GDx+uuixHe+2117Bz505s375ddSlBi80NETnK/fffjz179mDTpk2qSwkJ6enp2LVrF2pqavCnP/0JOTk52LBhAxucblJWVoYHH3wQb7/9Nlwul+pyghabG4fq168fdF1HRUWFz3hFRQXcbreiqoi615w5c/D3v/8dGzduxMUXX6y6nJAQFhaGoUOHAgCysrKwfft2PPfcc3jxxRcVV+ZMO3bsQGVlJUaPHu0dMwwDGzduxNKlS9HU1ARd1xVWGBw458ahwsLCkJWVhcLCQu+YaZooLCzk83ByHCkl5syZgzVr1uCdd97B4MGDVZcUskzTRFNTk+oyHOvGG2+Ex+PBrl27vP+NGTMGM2bMwK5du9jYfIF3bhwsLy8POTk5GDNmDK688kosWbIE9fX1yM3NVV2aY506dQqffvqp988HDx7Erl27EB8fj4EDByqszNnuv/9+vPrqq/jrX/+K6OholJeXAwD69u2LPn36KK7OuebOnYvJkydj4MCBqKurw6uvvor169dj3bp1qktzrOjoaL+5ZJGRkUhISOAcszbY3DjY9OnTcezYMcybNw/l5eUYNWoU1q5d6zfJmLrOhx9+iOuvv97757y8PABATk4OXn75ZUVVOd+KFSsAANddd53P+MqVKzFr1iz7CwoRlZWVmDlzJo4ePYq+ffti5MiRWLduHb761a+qLo1CHN/nhoiIiByFc26IiIjIUdjcEBERkaOwuSEiIiJHYXNDREREjsLmhoiIiByFzQ0RERE5CpsbIiIichQ2N0REF0gIgTfeeEN1GUT0BTY3RBTQrFmzIITw+6/tx0tciJdffhmxsbFd8lqdNWvWLEydOlVpDUTU9fjxC0Rk6aabbsLKlSt9xi666CJF1Vg7ffo0evfurboMIgoSvHNDRJbCw8Phdrt9/mv91OG//vWvGD16NFwuF9LS0pCfn48zZ854v3fx4sXIzMxEZGQkUlJS8N3vfhenTp0CAKxfvx65ubmoqanx3hF64oknAAR+xBMbG+v9bK7S0lIIIbB69WpMnDgRLpcLr7zyCgDg17/+NS699FK4XC5kZGRg+fLlHdrf6667Dg888AAeeeQRxMfHw+12e+tqtW/fPlx77bVwuVwYPnw43n77bb/XKSsrw7Rp0xAbG4v4+HjcfvvtKC0tBQAUFxcjIiICr776qnf7119/HX369MHevXs7VC8RBcbmhog67L333sPMmTPx4IMPYu/evXjxxRfx8ssvY8GCBd5tNE3DL37xCxQVFeG3v/0t3nnnHTzyyCMAgKuuugpLlixBTEwMjh49iqNHj+Lhhx/uUA2PPvooHnzwQXzyySfIzs7GK6+8gnnz5mHBggX45JNPsHDhQjz++OP47W9/26HX/e1vf4vIyEhs3boVixYtwpNPPultYEzTxB133IGwsDBs3boVL7zwAn74wx/6fP/p06eRnZ2N6OhovPfee3j//fcRFRWFm266Cc3NzcjIyMCzzz6L7373uzh06BAOHz6Me++9F8888wyGDx/eoVqJyIIkIgogJydH6rouIyMjvf99/etfl1JKeeONN8qFCxf6bL9q1So5YMAAy9f74x//KBMSErx/Xrlypezbt6/fdgDkmjVrfMb69u0rV65cKaWU8uDBgxKAXLJkic82Q4YMka+++qrP2FNPPSXHjx9/zn28/fbbvX+eOHGinDBhgs82Y8eOlT/84Q+llFKuW7dO9urVS37++efer7/11ls+Na9atUqmp6dL0zS92zQ1Nck+ffrIdevWecemTJkir7nmGnnjjTfKr33taz7bE9GF4ZwbIrJ0/fXXY8WKFd4/R0ZGAgA+/vhjvP/++z53agzDQGNjIxoaGhAREYF//etfKCgoQHFxMWpra3HmzBmfr1+oMWPGeP+/vr4e+/fvx913343Zs2d7x8+cOYO+fft26HVHjhzp8+cBAwagsrISAPDJJ58gJSUFSUlJ3q+PHz/eZ/uPP/4Yn376KaKjo33GGxsbsX//fu+fX3rpJVxyySXQNA1FRUUQQnSoTiKyxuaGiCxFRkZi6NChfuOnTp1Cfn4+7rjjDr+vuVwulJaW4pZbbsF9992HBQsWID4+Hps2bcLdd9+N5ubmczY3QghIKX3GTp8+HbC2tvUAwK9+9SuMGzfOZ7vWOULnq/3EZCEETNM87+8/deoUsrKyvPOA2mo7Gfvjjz9GfX09NE3D0aNHMWDAgA7VSUTW2NwQUYeNHj0aJSUlARsfANixYwdM08TPfvYzaFrL1L7XX3/dZ5uwsDAYhuH3vRdddBGOHj3q/fO+ffvQ0NBwznoSExORlJSEAwcOYMaMGR3dnfN26aWXoqyszKcZ+eCDD3y2GT16NFavXo3+/fsjJiYm4OtUVVVh1qxZeOyxx3D06FHMmDEDO3fuRJ8+fbqtdqJQwgnFRNRh8+bNw+9+9zvk5+ejqKgIn3zyCV577TX8+Mc/BgAMHToUp0+fxvPPP48DBw5g1apVeOGFF3xeIzU1FadOnUJhYSGOHz/ubWBuuOEGLF26FB999BE+/PBD3Hvvvee1zDs/Px8FBQX4xS9+gX//+9/weDxYuXIlFi9e3GX7PWnSJFxyySXIycnBxx9/jPfeew+PPfaYzzYzZsxAv379cPvtt+O9997DwYMHsX79ejzwwAM4fPgwAODee+9FSkoKfvzjH2Px4sUwDKPDE6qJyBqbGyLqsOzsbPz973/HP//5T4wdOxZf+cpX8POf/xyDBg0CAFx++eVYvHgxnnnmGVx22WV45ZVXUFBQ4PMaV111Fe69915Mnz4dF110ERYtWgQA+NnPfoaUlBRcc801+OY3v4mHH374vObo3HPPPfj1r3+NlStXIjMzExMnTsTLL7+MwYMHd9l+a5qGNWvW4D//+Q+uvPJK3HPPPT7zjgAgIiICGzduxMCBA3HHHXfg0ksvxd13343GxkbExMTgd7/7Hd58802sWrUKvXr1QmRkJH7/+9/jV7/6Fd56660uq5UolAnZ/uE2ERERUQ/GOzdERETkKGxuiIiIyFHY3BAREZGjsLkhIiIiR2FzQ0RERI7C5oaIiIgchc0NEREROQqbGyIiInIUNjdERETkKGxuiIiIyFHY3BAREZGjsLkhIiIiR/n/dSWgfrENCcwAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create bar plot\n",
"plt.bar(np.arange(len(forrest.featureImportance)), forrest.featureImportance, color='steelblue')\n",
"\n",
"# Add labels and title\n",
"plt.xlabel('Feature Index')\n",
"plt.ylabel('Importance')\n",
"plt.title('Feature Importance')\n",
"\n",
"# Add grid\n",
"plt.grid(True, linestyle='--', alpha=0.6)\n",
"\n",
"# Show plot\n",
"plt.show()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluating predictions\n",
"\n",
"Depending on the task at hand we create a confusion matrix (classification) or simple metrics (regression). Since the number of classes is fixed to two, we don't need to change anything here."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"if task == 'regressor':\n",
" metrics = RegressionScores(numClasses=2)\n",
" metrics.calcScores(prediction, validTargets, validLabels)\n",
" print(metrics)\n",
"elif task == 'classifier':\n",
" confusion = ConfusionMatrix(numClasses=2)\n",
" confusion.update(prediction, validLabels)\n",
" confusion.percentages()\n",
" confusion.calcScores()\n",
" print(confusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Saving and Loading a Forrest\n",
"\n",
"Forrests can be converted to dictionaries and then saved as a json file. This allows us to load them and re-use them. Also json is a raw text format, which is neat."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: None, bootstrapping: True\n",
"\n",
"————————————————————————— tree: 01/15 —————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 57\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 4 <= 1.03, samples: 9600\n",
" ├─feat: 2 <= 3.54, samples: 5667\n",
" │ ├─feat: 4 <= -0.76, samples: 5666\n",
" │ │ ├─feat: 1 <= 4.21, samples: 1184\n",
" │ │ │ ├─feat: 2 <= 1.95, samples: 1182\n",
" │ │ │ │ ├─feat: 1 <= 0.01, samples: 1157\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴feat: 1 <= -0.18, samples: 25\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 3 <= -0.93, samples: 4482\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 6.85, samples: 3985\n",
" │ │ ├─feat: 1 <= -2.57, samples: 3931\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 0.0\n",
" └─╴feat: 0 <= -1.60, samples: 3933\n",
" ├─feat: 2 <= 1.64, samples: 210\n",
" │ ├─feat: 1 <= -1.12, samples: 202\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 4.79, samples: 200\n",
" │ │ ├─feat: 1 <= 3.72, samples: 103\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.72, samples: 8\n",
" │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 2.69, samples: 7\n",
" │ ├─feat: 3 <= -0.72, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 0 <= 1.57, samples: 3723\n",
" ├─feat: 0 <= 0.64, samples: 3491\n",
" │ ├─feat: 3 <= 1.35, samples: 2615\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 6.57, samples: 2212\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 1.72, samples: 876\n",
" │ ├─feat: 0 <= 1.44, samples: 428\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 2 <= 0.62, samples: 448\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= -2.06, samples: 232\n",
" └─╴value: 0.0\n",
" └─╴feat: 3 <= 6.65, samples: 230\n",
" ├─feat: 3 <= 3.02, samples: 218\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————————— tree: 02/15 —————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 27\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 4 <= 4.59, samples: 9600\n",
" ├─feat: 2 <= 3.05, samples: 9597\n",
" │ ├─feat: 3 <= 5.44, samples: 9588\n",
" │ │ ├─feat: 2 <= -2.60, samples: 8022\n",
" │ │ │ ├─feat: 4 <= 0.81, samples: 20\n",
" │ │ │ │ ├─feat: 1 <= -0.94, samples: 12\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴feat: 1 <= 0.60, samples: 8\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴feat: 3 <= 5.05, samples: 8002\n",
" │ │ │ ├─feat: 1 <= 4.19, samples: 7280\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 2.78, samples: 9\n",
" │ ├─feat: 4 <= 1.35, samples: 8\n",
" │ │ ├─feat: 0 <= -0.50, samples: 6\n",
" │ │ │ ├─feat: 4 <= 0.72, samples: 4\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————————— tree: 03/15 —————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 41\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 0 <= 3.53, samples: 9600\n",
" ├─feat: 4 <= 0.63, samples: 9599\n",
" │ ├─feat: 2 <= -1.44, samples: 4509\n",
" │ │ ├─feat: 0 <= 1.21, samples: 322\n",
" │ │ │ ├─feat: 0 <= -2.28, samples: 303\n",
" │ │ │ │ ├─feat: 0 <= -2.46, samples: 5\n",
" │ │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴feat: 4 <= 0.61, samples: 298\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 0 <= 2.61, samples: 19\n",
" │ │ │ ├─feat: 4 <= -1.10, samples: 18\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= -0.42, samples: 4187\n",
" │ │ ├─feat: 0 <= -0.58, samples: 1674\n",
" │ │ │ ├─feat: 2 <= -0.64, samples: 441\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 1 <= 1.48, samples: 1233\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.64, samples: 2513\n",
" │ │ ├─feat: 1 <= 4.31, samples: 643\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 2 <= 0.78, samples: 1870\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 4.42, samples: 5090\n",
" │ ├─feat: 2 <= -3.63, samples: 5077\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= -1.36, samples: 5076\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -2.25, samples: 4962\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 0.0\n",
"\n",
"————————————————————————— tree: 04/15 —————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 55\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 2 <= -2.67, samples: 9600\n",
" ├─feat: 3 <= 5.96, samples: 26\n",
" │ ├─feat: 0 <= 1.52, samples: 20\n",
" │ │ ├─feat: 0 <= -0.61, samples: 18\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 3 <= 1.56, samples: 17\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 0 <= -2.68, samples: 9574\n",
" ├─feat: 2 <= 0.19, samples: 34\n",
" │ ├─feat: 0 <= -3.51, samples: 17\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 4.30, samples: 16\n",
" │ │ ├─feat: 4 <= 1.69, samples: 9\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 2 <= 0.64, samples: 17\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 1.20, samples: 13\n",
" │ ├─feat: 4 <= 0.61, samples: 6\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 4 <= 2.02, samples: 9540\n",
" ├─feat: 4 <= -0.28, samples: 8065\n",
" │ ├─feat: 2 <= 0.03, samples: 2045\n",
" │ │ ├─feat: 2 <= -0.38, samples: 1054\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= -0.66, samples: 991\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 2 <= 0.34, samples: 6020\n",
" │ ├─feat: 1 <= 0.99, samples: 3815\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 1.66, samples: 2205\n",
" │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 2 <= -0.15, samples: 1475\n",
" ├─feat: 2 <= -0.44, samples: 654\n",
" │ ├─feat: 1 <= 2.78, samples: 504\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 1 <= 3.39, samples: 150\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 4 <= 3.66, samples: 821\n",
" ├─feat: 1 <= 2.69, samples: 783\n",
" │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————————— tree: 05/15 —————————————————————————\n",
"split: RSA, impurity: ODD, leaf: AnomalyDetection, nodes: 19\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"·······························································\n",
"╴feat: 3 <= 3.85, samples: 9600\n",
" ├─feat: 4 <= 3.46, samples: 5378\n",
" │ ├─feat: 1 <= -1.00, samples: 5359\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 2.45, samples: 4667\n",
" │ │ ├─feat: 1 <= 0.49, samples: 4108\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴feat: 3 <= -0.44, samples: 1560\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -1.19, samples: 559\n",
" │ │ ├─feat: 3 <= 3.28, samples: 87\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴feat: 4 <= 0.59, samples: 472\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"\n"
]
}
],
"source": [
"ModelIO.save(forrest, 'forrest-test')\n",
"newForrest = ModelIO.load('forrest-test')\n",
"print(newForrest)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"prediction = newForrest.eval(validData)\n",
"\n",
"if task == 'regressor':\n",
" newMetrics = RegressionScores(numClasses=2)\n",
" newMetrics.calcScores(prediction, validTargets, validLabels)\n",
" print(newMetrics)\n",
"elif task == 'classifier':\n",
" newConfusion = ConfusionMatrix(numClasses=2)\n",
" newConfusion.update(prediction, validLabels)\n",
" newConfusion.percentages()\n",
" newConfusion.calcScores()\n",
" print(newConfusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Comment\n",
"\n",
"The forrest works as well as the tree code, because it completely builds on it, thus inhereting all it's problems. The progress bar for training trees is very erratic, because it's always set to the current level and because of the recurvise learning process it's jumping between levels. Also if you are using boosting, every tree will be trained twice."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.2"
},
"vscode": {
"interpreter": {
"hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}