{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Testing the Forrest\n",
"\n",
"## Importing the Basics"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from matplotlib import pyplot as plt\n",
"from machineLearning.metric import ConfusionMatrix, RegressionScores\n",
"from machineLearning.utility import ModelIO\n",
"from machineLearning.rf import (\n",
" RandomForest, DecisionTree,\n",
" Gini, Entropy, MAE, MSE,\n",
" Mode, Mean, Confidence, Probabilities,\n",
" CART, ID3, C45,\n",
" AdaBoosting, GradientBoosting,\n",
" Majority, Confidence, Average, Median\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generating Test Data\n",
"\n",
"Here I generate random test data. It's two blocks shifted very slightly in some dimensions. For classifier tasks each block gets a label, for regressor tasks each block gets the average coordinates plus some random value as a traget. It's a very simple dummy data set meant for testing the code.\n",
"\n",
"Here one can change the dimensionallity and amount of the data."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def dataShift(dims):\n",
" offSet = [5, 1.5, 2.5]\n",
" diffLen = abs(len(offSet) - dims)\n",
" offSet.extend([0] * diffLen)\n",
" random.shuffle(offSet)\n",
" return offSet[:dims]\n",
"\n",
"# Initialize some parameters\n",
"totalAmount = 6400\n",
"dims = 5\n",
"evalAmount = totalAmount // 4\n",
"trainAmount = totalAmount - evalAmount\n",
"offSet = dataShift(dims)\n",
"\n",
"# Create covariance matrix\n",
"cov = np.eye(dims) # This creates a covariance matrix with variances 1 and covariances 0\n",
"\n",
"# Generate random multivariate data\n",
"oneData = np.random.multivariate_normal(np.zeros(dims), cov, totalAmount)\n",
"twoData = np.random.multivariate_normal(offSet, cov, totalAmount)\n",
"\n",
"# Split the data into training and evaluation sets\n",
"trainData = np.vstack((oneData[:trainAmount], twoData[:trainAmount]))\n",
"validData = np.vstack((oneData[trainAmount:], twoData[trainAmount:]))\n",
"\n",
"# Labels for classification tasks\n",
"trainLabels = np.hstack((np.zeros(trainAmount), np.ones(trainAmount)))\n",
"validLabels = np.hstack((np.zeros(evalAmount), np.ones(evalAmount)))\n",
"\n",
"# Targets for regression tasks\n",
"trainTargets = np.sum(trainData, axis=1) + np.random.normal(0, 0.1, 2*trainAmount)\n",
"validTargets = np.sum(validData, axis=1) + np.random.normal(0, 0.1, 2*evalAmount)\n",
"\n",
"# Shuffle the training data\n",
"trainIndex = np.random.permutation(len(trainData))\n",
"trainData = trainData[trainIndex]\n",
"trainLabels = trainLabels[trainIndex]\n",
"trainTargets = trainTargets[trainIndex]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creating the Forrest\n",
"\n",
"Here the forrest is created. One can set the number of trees and set the maximum depth. Depending on the task, we add a different impurity function and a different leaf function. Finally we add the split algorithm and set the feature percentile. Higher numbers look at more possible splits, but decreases speed. Lower numbers look at less possible splits, speeding up the algorithm. Depending on the data set this can have a strong impact on the performance.\n",
"\n",
"One can set a different depth, leaf function, splitting algorithm and impurity function for each tree. Here in this simple case we create all trees with same parameters."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"task = 'classifier' # 'classifier'/'regressor'\n",
"forrest = RandomForest(bootstrapping=False, retrainFirst=False)\n",
"forrest.setComponent(AdaBoosting())\n",
"forrest.setComponent(Majority())\n",
"for i in range(5):\n",
" tree = DecisionTree(maxDepth=7, minSamplesSplit=2)\n",
" if task == 'regressor':\n",
" tree.setComponent(MSE())\n",
" tree.setComponent(Mean())\n",
" elif task == 'classifier':\n",
" tree.setComponent(Entropy())\n",
" tree.setComponent(Mode())\n",
" tree.setComponent(CART(featurePercentile=90))\n",
" forrest.append(tree)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Trainining the tree\n",
"\n",
"Again, depending on the task we train the forrest with targets or labels. Then we make a prediction and plot the tree."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tree 1 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 17%\n",
"tree 2 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 17%\n",
"tree 3 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 17%\n",
"tree 4 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 17%\n",
"tree 5 |⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿| done ✔ | 17%\n",
"━━━━━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: AdaBoosting, bootstrapping: False\n",
"\n",
"—————————————————————— tree: 1/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= 2.18, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 3 <= 0.04, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.52, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 3 <= 0.61, samples: 5\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 2/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 0.81, samples: 4\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 3 <= 0.04, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 3/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= 2.18, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 3 <= 0.05, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 1.96, samples: 2\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.52, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 3 <= 0.61, samples: 5\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 4/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 0.81, samples: 4\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 0 <= 2.28, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"—————————————————————— tree: 5/5 ———————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 2 <= -0.82, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 0 <= 2.45, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 2 <= -1.00, samples: 2\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 0 <= 0.17, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"\n"
]
}
],
"source": [
"if task == 'regressor':\n",
" forrest.train(trainData, trainTargets)\n",
"elif task == 'classifier':\n",
" forrest.train(trainData,trainLabels)\n",
"prediction = forrest.eval(validData)\n",
"print(forrest)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Accuracy(name='tree: 0', accuracy=0.9959375),\n",
" Accuracy(name='tree: 1', accuracy=0.995625),\n",
" Accuracy(name='tree: 2', accuracy=0.99625),\n",
" Accuracy(name='tree: 3', accuracy=0.9953125),\n",
" Accuracy(name='tree: 4', accuracy=0.995625)]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"forrest.accuracy(validData, validLabels)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABYSElEQVR4nO3de3wU1d0/8M+ZgWTJjVyALMRwxwQlgFykIIoXakREeXwqqDzlUsGfFCqaWoUqYFRAbUWscrEqYK1WrH3Qp1WhFOUmyB2JgUQkRIKwCSFXQpPAzPn9kWZhyQ4mIZmTzH7erxevlzmZ3XznM8fwZfacXSGllCAiIiJyCE11AUREREQNic0NEREROQqbGyIiInIUNjdERETkKGxuiIiIyFHY3BAREZGjsLkhIiIiR2FzQ0RERI7C5oaIiIgchc0NEREROQqbG6ImZuXKlRBC+P0zc+bMRvmZW7duxdNPP42ioqJGef7LUZ3Hrl27VJdSb0uWLMHKlStVl0EUMFqoLoCI/HvmmWfQpUsXn7FevXo1ys/aunUrUlNTMXHiRERGRjbKzwhkS5YsQZs2bTBx4kTVpRAFBDY3RE3UiBEjMGDAANVlXJaysjKEhoaqLkOZM2fOICQkRHUZRAGHL0sRNVOfffYZrr/+eoSGhiI8PBwjR45Eenq6zzH79+/HxIkT0bVrV7hcLrjdbvziF7/AqVOnvMc8/fTT+M1vfgMA6NKli/clsOzsbGRnZ0MI4fclFSEEnn76aZ/nEULgwIEDuP/++xEVFYWhQ4d6v//nP/8Z/fv3R6tWrRAdHY17770XOTk59Tr3iRMnIiwsDEePHsUdd9yBsLAwxMXFYfHixQCAtLQ03HzzzQgNDUWnTp3w3nvv+Ty++qWuTZs24f/9v/+HmJgYREREYPz48SgsLKzx85YsWYKrr74awcHB6NChA6ZNm1bjJbwbb7wRvXr1wu7du3HDDTcgJCQEv/3tb9G5c2ekp6dj48aN3mxvvPFGAEBBQQEee+wxJCUlISwsDBERERgxYgS+/vprn+fesGEDhBD44IMPMG/ePFxxxRVwuVy45ZZb8N1339Wod/v27bj99tsRFRWF0NBQ9O7dG6+88orPMRkZGfjZz36G6OhouFwuDBgwAP/3f/9X10tB1CTxzg1RE1VcXIz8/HyfsTZt2gAA3nnnHUyYMAHJycl44YUXcObMGSxduhRDhw7F3r170blzZwDAunXrkJWVhUmTJsHtdiM9PR1//OMfkZ6ejq+++gpCCNx999349ttv8Ze//AUvv/yy92e0bdsWJ0+erHPd99xzD3r06IH58+dDSgkAmDdvHmbPno0xY8Zg8uTJOHnyJF599VXccMMN2Lt3b71eCjMMAyNGjMANN9yAF198Ee+++y6mT5+O0NBQPPnkkxg3bhzuvvtuLFu2DOPHj8fgwYNrvMw3ffp0REZG4umnn0ZmZiaWLl2K77//3ttMAFVNW2pqKoYPH46pU6d6j9u5cye+/PJLtGzZ0vt8p06dwogRI3Dvvffif/7nfxAbG4sbb7wRv/rVrxAWFoYnn3wSABAbGwsAyMrKwkcffYR77rkHXbp0QW5uLl5//XUMGzYMBw4cQIcOHXzqff7556FpGh577DEUFxfjxRdfxLhx47B9+3bvMevWrcMdd9yB9u3bY8aMGXC73Th48CD+8Y9/YMaMGQCA9PR0XHfddYiLi8PMmTMRGhqKDz74AKNHj8bf/vY3/Nd//VedrwdRkyKJqElZsWKFBOD3j5RSlpaWysjISDllyhSfx3k8Htm6dWuf8TNnztR4/r/85S8SgNy0aZN37He/+50EII8cOeJz7JEjRyQAuWLFihrPA0DOnTvX+/XcuXMlAHnffff5HJednS11XZfz5s3zGU9LS5MtWrSoMW6Vx86dO71jEyZMkADk/PnzvWOFhYWyVatWUggh33//fe94RkZGjVqrn7N///6ysrLSO/7iiy9KAPLjjz+WUkqZl5cng4KC5K233ioNw/Ae99prr0kAcvny5d6xYcOGSQBy2bJlNc7h6quvlsOGDasxXl5e7vO8UlZlHhwcLJ955hnv2BdffCEByJ49e8qKigrv+CuvvCIByLS0NCmllOfOnZNdunSRnTp1koWFhT7Pa5qm979vueUWmZSUJMvLy32+P2TIENmjR48adRI1N3xZiqiJWrx4MdatW+fzB6j6l3lRURHuu+8+5Ofne//ouo5Bgwbhiy++8D5Hq1atvP9dXl6O/Px8/OQnPwEA7Nmzp1Hqfuihh3y+/t///V+YpokxY8b41Ot2u9GjRw+feutq8uTJ3v+OjIxEQkICQkNDMWbMGO94QkICIiMjkZWVVePxDz74oM+dl6lTp6JFixb49NNPAQD/+te/UFlZiUceeQSadv7X5ZQpUxAREYFPPvnE5/mCg4MxadKkWtcfHBzsfV7DMHDq1CmEhYUhISHB7/WZNGkSgoKCvF9ff/31AOA9t7179+LIkSN45JFHatwNq74TVVBQgM8//xxjxoxBaWmp93qcOnUKycnJOHToEH744YdanwNRU8SXpYiaqGuvvdbvguJDhw4BAG6++Wa/j4uIiPD+d0FBAVJTU/H+++8jLy/P57ji4uIGrPa8i1/6OXToEKSU6NGjh9/jL2wu6sLlcqFt27Y+Y61bt8YVV1zh/Yv8wnF/a2kuriksLAzt27dHdnY2AOD7778HUNUgXSgoKAhdu3b1fr9aXFycT/PxY0zTxCuvvIIlS5bgyJEjMAzD+72YmJgax3fs2NHn66ioKADwntvhw4cBXHpX3XfffQcpJWbPno3Zs2f7PSYvLw9xcXG1Pg+ipobNDVEzY5omgKp1N263u8b3W7Q4/7/1mDFjsHXrVvzmN79B3759ERYWBtM0cdttt3mf51IubhKqXfiX8MUuvFtUXa8QAp999hl0Xa9xfFhY2I/W4Y+/57rUuPzP+p/GdPG5/5j58+dj9uzZ+MUvfoFnn30W0dHR0DQNjzzyiN/r0xDnVv28jz32GJKTk/0e071791o/H1FTxOaGqJnp1q0bAKBdu3YYPny45XGFhYVYv349UlNTMWfOHO949Z2fC1k1MdV3Bi7eGXTxHYsfq1dKiS5duuDKK6+s9ePscOjQIdx0003er0+fPo0TJ07g9ttvBwB06tQJAJCZmYmuXbt6j6usrMSRI0cumf+FrPL98MMPcdNNN+Gtt97yGS8qKvIu7K6L6rnxzTffWNZWfR4tW7asdf1EzQ3X3BA1M8nJyYiIiMD8+fNx9uzZGt+v3uFU/a/8i/9Vv2jRohqPqX4vmoubmIiICLRp0wabNm3yGV+yZEmt67377ruh6zpSU1Nr1CKl9NmWbrc//vGPPhkuXboU586dw4gRIwAAw4cPR1BQEP7whz/41P7WW2+huLgYI0eOrNXPCQ0N9fvuz7qu18jkr3/9a73XvPTr1w9dunTBokWLavy86p/Trl073HjjjXj99ddx4sSJGs9Rnx1yRE0N79wQNTMRERFYunQpfv7zn6Nfv36499570bZtWxw9ehSffPIJrrvuOrz22muIiIjwbpM+e/Ys4uLi8M9//hNHjhyp8Zz9+/cHADz55JO499570bJlS4waNQqhoaGYPHkynn/+eUyePBkDBgzApk2b8O2339a63m7duuG5557DrFmzkJ2djdGjRyM8PBxHjhzB6tWr8eCDD+Kxxx5rsHzqorKyErfccgvGjBmDzMxMLFmyBEOHDsWdd94JoGo7/KxZs5CamorbbrsNd955p/e4gQMH4n/+539q9XP69++PpUuX4rnnnkP37t3Rrl073HzzzbjjjjvwzDPPYNKkSRgyZAjS0tLw7rvv+twlqgtN07B06VKMGjUKffv2xaRJk9C+fXtkZGQgPT0da9euBVC1WH3o0KFISkrClClT0LVrV+Tm5mLbtm04duxYjffZIWp2FO3SIiIL/rY++/PFF1/I5ORk2bp1a+lyuWS3bt3kxIkT5a5du7zHHDt2TP7Xf/2XjIyMlK1bt5b33HOPPH78eI2t0VJK+eyzz8q4uDipaZrPtvAzZ87IBx54QLZu3VqGh4fLMWPGyLy8PMut4CdPnvRb79/+9jc5dOhQGRoaKkNDQ2ViYqKcNm2azMzMrHMeEyZMkKGhoTWOHTZsmLz66qtrjHfq1EmOHDmyxnNu3LhRPvjggzIqKkqGhYXJcePGyVOnTtV4/GuvvSYTExNly5YtZWxsrJw6dWqNrdZWP1vKqm36I0eOlOHh4RKAd1t4eXm5/PWvfy3bt28vW7VqJa+77jq5bds2OWzYMJ+t49Vbwf/617/6PK/VVv0tW7bIn/70pzI8PFyGhobK3r17y1dffdXnmMOHD8vx48dLt9stW7ZsKePi4uQdd9whP/zwQ7/nQNScCCltWGVHRNSErFy5EpMmTcLOnTub/UdcEFFNXHNDREREjsLmhoiIiByFzQ0RERE5CtfcEBERkaPwzg0RERE5CpsbIiIicpSAexM/0zRx/PhxhIeHW74lOhERETUtUkqUlpaiQ4cO0LRL35sJuObm+PHjiI+PV10GERER1UNOTg6uuOKKSx4TcM1NeHg4gKpwIiIiFFfT+AzDwPfff49OnTpZfqIwNTzmrgZzV4O5qxFouZeUlCA+Pt779/ilBFxzU/1SVERERMA0N0DV+QbC5G8qmLsazF0N5q5GoOZemyUlXFBMREREjsLmhoiIiByFzY3DCSEQHx/PnWE2Y+5qMHc1mLsazN2a0uZm06ZNGDVqFDp06AAhBD766KMffcyGDRvQr18/BAcHo3v37li5cmWj19mcaZqGmJiYH902Rw2LuavB3NVg7mowd2tKEykrK0OfPn2wePHiWh1/5MgRjBw5EjfddBP27duHRx55BJMnT8batWsbudLmyzAMZGRkeBeekT2YuxrMXQ3mrgZzt6Z0t9SIESMwYsSIWh+/bNkydOnSBS+99BIAoGfPntiyZQtefvllJCcnN1aZzV55ebnqEgISc1eDuavB3NVg7v41q63g27Ztw/Dhw33GkpOT8cgjj1g+pqKiAhUVFd6vS0pKAFR1vNXdrhACmqbBNE1c+Dmi1eMXd8VW45qmQQjhdxyoenfk2ozrug4ppd/xi2u0Gr/4nC4+Vyec04/V3hTO6cKf4ZRz+rFxleckpfSbe3M+J3/jTe2cANTIvbmfU3O4TkDN3Jv7OdXlOl1Ks2puPB4PYmNjfcZiY2NRUlKCf//732jVqlWNxyxYsACpqak1xtPT0xEWFgYAiI6ORseOHXHs2DEUFBR4j3G73XC73cjOzkZpaal3PD4+HjExMTh06JBP19y1a1dERETgwIEDPhchISEBQUFBSEtL86khKSkJlZWVyMzM9I7puo6kpCSUlpYiKyvLO+5yuZCYmIjCwkLk5OR4x8PDw9GtWzfk5eXB4/F4x6vP6fjx4ygoKEB6ejqEEI44p+ZwnYqKinxyd8I5NYfrVFFR4ZO7E86pOVyn0NBQFBYWenN3wjk1h+vUtm1blJaW+uTe3M/pUtfp8OHDqC0hL26zFBFCYPXq1Rg9erTlMVdeeSUmTZqEWbNmecc+/fRTjBw5EmfOnPHb3Pi7cxMfH4+CggLvm/g5uds3DAMlJSXez9Jywjk1h+t0ce5OOKfmcJ2klCguLq6Re3M+J3/jTe2cAKCoqMgn9+Z+Ts3hOgkhUFxcjLCwMG/uzf2cLnWdCgsLER0djeLi4h99E95mdefG7XYjNzfXZyw3NxcRERF+GxsACA4ORnBwcI1xXddrvKOj1Ypzq3d+bMzx6n91XsyqxkvVHhUV1Sg11nW8oc6pOVwnq9yb8zk1h+skhPCbe3M+J6vxpnZO/nKv6/M0tXNqDtcpMjLysmus63hTu05+n7vWRzYBgwcPxvr1633G1q1bh8GDByuqqOkzDANpaWlcTW8z5q4Gc1eDuavB3K0pbW5Onz6Nffv2Yd++fQCqtnrv27cPR48eBQDMmjUL48eP9x7/0EMPISsrC48//jgyMjKwZMkSfPDBB3j00UdVlN9scOKrwdzVYO5qMHc1mLt/SpubXbt24ZprrsE111wDAEhJScE111yDOXPmAABOnDjhbXQAoEuXLvjkk0+wbt069OnTBy+99BLefPNNbgMnIiIiL6Vrbm688cYai48u5O/dh2+88Ubs3bu3EasiIiKi5qzJ7JayS0lJCVq3bl2r1dZOIKVEeXk5XC4XP3/ERsxdDeauBnNXI9Byr8vf381qtxTVT1BQkOoSGl3ys5+oLuEiEroADAkATeuXztrZI1WX0KgCYb43RcxdDebuX7PaLUV1Z5om0tLSarwnATUuXQBDO5jQm1Zf43ic72owdzWYuzU2N0REROQobG6IiIjIUdjcEBERkaOwuXE4TdOQlJRk+TbX1DgMCWw5rv1nQTHZhfNdDeauBnO3xkQCQGVlpeoSAlJw7T8GhRoQ57sazF0N5u4fmxuHM00TmZmZXE1vM10AA2O5W8punO9qMHc1mLs1NjdERETkKGxuiIiIyFHY3AQAXefiDxW4mFgNznc1mLsazN0/fvyCw+m6jqSkJNVlBBxDCmw5zl86duN8V4O5q8HcrfHOjcNJKVFSUnLJT1+nxiARFSwBMHc7cb6rwdzVYO7W2Nw4nGmayMrK4mp6m+kC6N2Gu6XsxvmuBnNXg7lbY3NDREREjsLmhoiIiByFzU0AcLlcqksIOBLAmXNccaMC57sazF0N5u4fd0s5nK7rSExMVF1GwDGlwM5c7payG+e7GsxdDeZujXduHM40TZw6dYoLzmwmIOEOkRC8d2Mrznc1mLsazN0amxuHk1IiJyeHWwVtpgkgIcqExt1StuJ8V4O5q8HcrbG5ISIiIkdhc0NERESOwuYmAISHh6suIeBIAIXlXHGjAue7GsxdDebuH3dLOZyu6+jWrZvqMgKOKQX2n+KCG7txvqvB3NVg7tZ458bhTNOEx+PhanqbCUh0Cje5W8pmnO9qMHc1mLs1NjcOJ6WEx+PhanqbaQLoHCG5W8pmnO9qMHc1mLs1NjdERETkKGxuiIiIyFHY3DicEALR0dEQgq+P2ElKwFMmwLvF9uJ8V4O5q8HcrXG3lMNpmoaOHTuqLiPgmBDILOIvHLtxvqvB3NVg7tZ458bhTNPE0aNHuZreZhokEiJNaNwtZSvOdzWYuxrM3RqbG4eTUqKgoICr6W0mBOAOleDdYntxvqvB3NVg7tbY3BAREZGjsLkhIiIiR2Fz43BCCLjdbq6mt5kpgewSAZN3i23F+a4Gc1eDuVvjbimH0zQNbrdbdRkBR0Lg+1L+wrEb57sazF0N5m6Nd24czjAMHD58GIZhqC4loGhConeMCU3w1o2dON/VYO5qMHdrbG4CQGlpqeoSAo4AEOWS4L0b+3G+q8Hc1WDu/rG5ISIiIkdhc0NERESOwubG4YQQiI+P52p6m5kSyCzUuFvKZpzvajB3NZi7Ne6WcjhN0xATE6O6jIAjIeA5o7qKwMP5rgZzV4O5W+OdG4czDAMZGRlcTW8zTUgMjDW4W8pmnO9qMHc1mLs1NjcBoLy8XHUJAUcACGkB7pZSgPNdDeauBnP3j80NEREROQqbGyIiInIUNjcOp2kaunbtCk3jpbaTIYH9+RoMLrmxFee7GsxdDeZujbulHE4IgYiICNVlBCCBwgrVNQQeznc1mLsazN0a2z2HMwwDaWlpXE1vM11IDO1gQOduKVtxvqvB3NVg7tbY3AQATnw1dG6VUoLzXQ3mrgZz94/NDRERETkKmxsiIiJyFDY3DqdpGhISEria3maGBHbmcreU3Tjf1WDuajB3a0wkAAQFBakuISBV8KVwJTjf1WDuajB3/9jcOJxpmkhLS4NpmqpLCSi6AIZ2MLmo2Gac72owdzWYuzU2N0REROQobG6IiIjIUdjcEBERkaOwuXE4TdOQlJTE1fQ2MySw5Th3S9mN810N5q4Gc7fGRAJAZWWl6hICUrCuuoLAxPmuBnNXg7n7p7y5Wbx4MTp37gyXy4VBgwZhx44dlzx+0aJFSEhIQKtWrRAfH49HH30U5eXlNlXb/JimiczMTK6mt5kugIGx3C1lN853NZi7GszdmtLmZtWqVUhJScHcuXOxZ88e9OnTB8nJycjLy/N7/HvvvYeZM2di7ty5OHjwIN566y2sWrUKv/3tb22unIiIiJoqpc3NwoULMWXKFEyaNAlXXXUVli1bhpCQECxfvtzv8Vu3bsV1112H+++/H507d8att96K++6770fv9hAREVHgaKHqB1dWVmL37t2YNWuWd0zTNAwfPhzbtm3z+5ghQ4bgz3/+M3bs2IFrr70WWVlZ+PTTT/Hzn//c8udUVFSgoqLC+3VJSQmAqk9Srf40VSEENE2DaZqQ8vwK0Orxiz911Wpc0zQIIfyOA6hx69BqXNd1SCn9jl9co9X4hed0YU1OOaeLxwFAExIXvgpkSkBC1BivWuQroAvfn1m9+Pfil5KsxwUA6TMuAZhSQEDClNL7My4c1y48XgImBDRIiAvGrWpviHMyDEPZdWrsuSelrPGY5n5O/sab2jlVf//C7zX3c2oO16n6Z19YZ3M/p7pcp0tR1tzk5+fDMAzExsb6jMfGxiIjI8PvY+6//37k5+dj6NChkFLi3LlzeOihhy75stSCBQuQmppaYzw9PR1hYWEAgOjoaHTs2BHHjh1DQUGB9xi32w23243s7GyUlpZ6x+Pj4xETE4NDhw75rPfp2rUrIiIicODAAZ+LkJCQgKCgIKSlpfnUkJSUhMrKSmRmZnrHdF1HUlISSktLkZWV5R13uVxITExEYWEhcnJyvOPh4eHo1q0b8vLy4PF4vOPV53TixAlIKXHgwAHHnJO/6wQAvaIlolzn/0fJLNTgOQP0b2ci5IKZvj9fQ2EFMLi975qYnbkaKoyqdxa+0JbjGoL1qjU01ap2Q+mICgZ6tzk/fuYcsDNXR5tWApoQGNxeApAoLBfYf0qgY7hE54jzNXrKBDKLBHpESrhDz49nlwh8Xyoa5ZzS0tKUXafGnntnz571me9OOKfmcp00TfPm7pRzag7XKTQ01Cd3J5yT1XU6fPgwakvIi9ssmxw/fhxxcXHYunUrBg8e7B1//PHHsXHjRmzfvr3GYzZs2IB7770Xzz33HAYNGoTvvvsOM2bMwJQpUzB79my/P8ffnZv4+HgUFBQgIiICgLO7fcMwUFJSgvDwcAghHHFO/sZHzPusid25MRHtAooqqn5WU7pz8/dZI5rFv8rqe+emuLjYO9+dcE7+xpvaOQFAUVGRT+7N/Zyaw3USQqC4uBhhYWHe3Jv7OV3qOhUWFiI6OhrFxcXev7+tKLtz06ZNG+i6jtzcXJ/x3Nxc77/ELzZ79mz8/Oc/x+TJkwFUdZZlZWV48MEH8eSTT/rd6x8cHIzg4OAa47quQ9d99+pavVfAxcfZMS6E8DtuVeOl3ufg+++/R1JSks/zNedzsho3pf+tSVbjhuW432GLceF3XBMCvWLM/7zXzfmfIy2ONyGqOqOLxxvhnH5sHjT2dWrMOWaapt/53pzPyWq8KZ2TYRh+c6/r8zSlc2qo8cY8J8MwkJ2dfdm513W8qV0nv89d6yMbWFBQEPr374/169d7x0zTxPr1633u5FzozJkzNcKoPllFN6CIiIioiVF25wYAUlJSMGHCBAwYMADXXnstFi1ahLKyMkyaNAkAMH78eMTFxWHBggUAgFGjRmHhwoW45pprvC9LzZ49G6NGjapTR0dERETOpbS5GTt2LE6ePIk5c+bA4/Ggb9++WLNmjXeR8dGjR33u1Dz11FMQQuCpp57CDz/8gLZt22LUqFGYN2+eqlNoFlwul+oSAo5E1eJi3k+0H+e7GsxdDebun7IFxaqUlJSgdevWtVqQRM1H8rOfqC6h2Vg7e6TqEoiI6qwuf38r//gFalymaeLUqVN8e26bCUi4Q+R/9kmRXTjf1WDuajB3a2xuHE5KiZycHC64tpkmgIQo02fbNzU+znc1mLsazN0amxsiIiJyFDY3RERE5ChsbgJAeHi46hICjgRQWM4VNypwvqvB3NVg7v4p3QpOjU/XdXTr1k11GQHHlFWfJUX24nxXg7mrwdyt8c6Nw5mmCY/Hw9X0NhOQ6BRucreUzTjf1WDuajB3a2xuHE5KCY/Hw9X0NtME0DlCcreUzTjf1WDuajB3a2xuiIiIyFHY3BAREZGjsLlxOCEEoqOjIQRfH7GTlICnTIB3i+3F+a4Gc1eDuVvjbimH0zQNHTt2VF1GwDEhkFnEXzh243xXg7mrwdyt8c6Nw5mmiaNHj3I1vc00SCREmtC4W8pWnO9qMHc1mLs1NjcOJ6VEQUEBV9PbTAjAHSrBu8X24nxXg7mrwdytsbkhIiIiR2FzQ0RERI7C5sbhhBBwu91cTW8zUwLZJQIm7xbbivNdDeauBnO3xt1SDqdpGtxut+oyAo6EwPel/IVjN853NZi7GszdGu/cOJxhGDh8+DAMw1BdSkDRhETvGBOa4K0bO3G+q8Hc1WDu1tjcBIDS0lLVJQQcASDKJcF7N/bjfFeDuavB3P1jc0NERESOwuaGiIiIHIXNjcMJIRAfH8/V9DYzJZBZqHG3lM0439Vg7mowd2vcLeVwmqYhJiZGdRkBR0LAc0Z1FYGH810N5q4Gc7fGOzcOZxgGMjIyuJreZpqQGBhrcLeUzTjf1WDuajB3a2xuAkB5ebnqEgKOABDSAtwtpQDnuxrMXQ3m7h+bGyIiInIUNjdERETkKGxuHE7TNHTt2hWaxkttJ0MC+/M1GFxyYyvOdzWYuxrM3Rp3SzmcEAIRERGqywhAAoUVqmsIPJzvajB3NZi7NbZ7DmcYBtLS0ria3ma6kBjawYDO3VK24nxXg7mrwdytsbkJAJz4aujcKqUE57sazF0N5u4fmxsiIiJyFDY3RERE5ChsbhxO0zQkJCRwNb3NDAnszOVuKbtxvqvB3NVg7taYSAAICgpSXUJAquBL4UpwvqvB3NVg7v6xuXE40zSRlpYG0zRVlxJQdAEM7WByUbHNON/VYO5qMHdrbG6IiIjIUdjcEBERkaOwuSEiIiJHYXPjcJqmISkpiavpbWZIYMtx7payG+e7GsxdDeZujYkEgMrKStUlBKRgXXUFgYnzXQ3mrgZz94/NjcOZponMzEyupreZLoCBsdwtZTfOdzWYuxrM3RqbGyIiInIUNjdERETkKGxuAoCuc/GHClxMrAbnuxrMXQ3m7l8L1QVQ49J1HUlJSarLCDiGFNhynL907Mb5rgZzV4O5W+OdG4eTUqKkpARS8jaCvSSigiUA5m4nznc1mLsazN0amxuHM00TWVlZXE1vM10Avdtwt5TdON/VYO5qMHdrbG6IiIjIUdjcEBERkaOwuQkALpdLdQkBRwI4c44rblTgfFeDuavB3P3jbimH03UdiYmJqssIOKYU2JnL3VJ243xXg7mrwdyt8c6Nw5mmiVOnTnHBmc0EJNwhEoL3bmzF+a4Gc1eDuVtjc+NwUkrk5ORwq6DNNAEkRJnQuFvKVpzvajB3NZi7NTY3RERE5ChsboiIiMhR2NwEgPDwcNUlBBwJoLCcK25U4HxXg7mrwdz9424ph9N1Hd26dVNdRsAxpcD+U1xwYzfOdzWYuxrM3Rrv3DicaZrweDxcTW8zAYlO4SZ3S9mM810N5q4Gc7dW7+bmnXfewXXXXYcOHTrg+++/BwAsWrQIH3/8cYMVR5dPSgmPx8PV9DbTBNA5QnK3lM0439Vg7mowd2v1am6WLl2KlJQU3H777SgqKoJhGACAyMhILFq0qE7PtXjxYnTu3BkulwuDBg3Cjh07Lnl8UVERpk2bhvbt2yM4OBhXXnklPv300/qcBhERETlQvZqbV199FW+88QaefPJJ6Pr5d2EdMGAA0tLSav08q1atQkpKCubOnYs9e/agT58+SE5ORl5ent/jKysr8dOf/hTZ2dn48MMPkZmZiTfeeANxcXH1OQ0iIiJyoHotKD5y5AiuueaaGuPBwcEoKyur9fMsXLgQU6ZMwaRJkwAAy5YtwyeffILly5dj5syZNY5fvnw5CgoKsHXrVrRs2RIA0Llz5/qcQsAQQiA6OhpC8PURO0kJeMoEeLfYXpzvajB3NZi7tXo1N126dMG+ffvQqVMnn/E1a9agZ8+etXqOyspK7N69G7NmzfKOaZqG4cOHY9u2bX4f83//938YPHgwpk2bho8//hht27bF/fffjyeeeMLnDtKFKioqUFFR4f26pKQEAGAYhvflNCEENE2DaZo+r11Wj1cf92PjmqZBCOF3HECNRV9W47quQ0rpd/ziGq3Gq2sEgLi4OEgpYRiGI87J3zgAaELiwv/FTQlIiBrjhgQAAV34/kzjP1/qF/2esB4XAKTPuETVTikJ4LtiASEAHdI7LuC7DkdKwISABokLfz9Z1d4Q52QYhrLr1NhzTwjhM9+dcE7+xpvaOWmaViP35n5OzeU6XZy7E86pttfpUurV3KSkpGDatGkoLy+HlBI7duzAX/7yFyxYsABvvvlmrZ4jPz8fhmEgNjbWZzw2NhYZGRl+H5OVlYXPP/8c48aNw6efforvvvsOv/zlL3H27FnMnTvX72MWLFiA1NTUGuPp6ekICwsDAERHR6Njx444duwYCgoKvMe43W643W5kZ2ejtLTUOx4fH4+YmBgcOnQI5eXl3vGuXbsiIiICBw4c8LkICQkJCAoKqvGSXVJSEiorK5GZmekd03UdSUlJKC0tRVZWlnfc5XIhMTERhYWFyMnJ8Y6Hh4ejW7duyMvLg8fj8Y5Xn1NOTg5ycnIQGhoKIYQjzsnfdQKAXtESUa7z/6NkFmrwnAH6tzMRcsFM35+vobACGNze9PlLf2euhgoDGNrB93/aLcc1BOvAwNjz44YEthzXERUM9G5zfvzMOWBnro72IRIDY00UVwrve97sPyXQMVyic8T5Gj1lAplFAj0iJdyh58ezSwS+LxWNck5paWnKrlNjz72Kigrs2bPHO9+dcE7N4TqFhYVh+/btaNWqlfcfG839nJrDdWrXrh327t0LXde9uTf3c7rUdTp8+DBqS8h6LrN+99138fTTT3t/WIcOHZCamooHHnigVo8/fvw44uLisHXrVgwePNg7/vjjj2Pjxo3Yvn17jcdceeWVKC8vx5EjR7x3ahYuXIjf/e53OHHihN+f4+/OTXx8PAoKChAREQHA2d3+2bNn8c033+Dqq6/2/g/Q3M/J3/iIeZ81qTs3LYSJoR1MbD2hwfjPnZymcufm77NGNIt/ldVn7hmGgbS0NO98d8I5+Rtvaudkmib279/vk3tzP6fmcJ2klDVyb+7ndKnrVFhYiOjoaBQXF3v//rZS7zfxGzduHMaNG4czZ87g9OnTaNeuXZ0e36ZNG+i6jtzcXJ/x3Nxc77/EL9a+fXu0bNnS5yWonj17wuPxoLKyEkFBQTUeExwcjODg4Brjuq7XeCmr+mL6O9bu8ep/dV7MqsZLjVc/14XP19zPyR9T+n/d2WrcsBz3O2wxLvyOV30euIAhhc/PkRbHmxDw95Y4jXFOPzYPGvs6NfYc8zffm/M5WY03tXPyl3tdn6epnVNTv07VSw0uN/e6jje16+T3uWt95AWOHDmCQ4cOAQBCQkK8jc2hQ4eQnZ1dq+cICgpC//79sX79eu+YaZpYv369z52cC1133XX47rvvfDrGb7/9Fu3bt/fb2BAREVHgqVdzM3HiRGzdurXG+Pbt2zFx4sRaP09KSgreeOMNvP322zh48CCmTp2KsrIy7+6p8ePH+yw4njp1KgoKCjBjxgx8++23+OSTTzB//nxMmzatPqcREKrX2XA1vb1MWbVuxuRuKVtxvqvB3NVg7tbq9bLU3r17cd1119UY/8lPfoLp06fX+nnGjh2LkydPYs6cOfB4POjbty/WrFnjXWR89OhRn9tW8fHxWLt2LR599FH07t0bcXFxmDFjBp544on6nEZA0DTN8mU+ajwSVQuCyV6c72owdzWYu7V6NTdCCJ+VzNWKi4vrtFULAKZPn27ZEG3YsKHG2ODBg/HVV1/V6WcEMsMwkJ2djc6dO9fp9Uq6PJqQ6BUt8U2BsFwfQw2P810N5q4Gc7dWr5elbrjhBixYsMCnkTEMAwsWLMDQoUMbrDhqGP4aUWpcAkCUy3dHE9mD810N5q4Gc/evXnduXnjhBdxwww1ISEjA9ddfDwDYvHkzSkpK8PnnnzdogURERER1Ua87N1dddRX279+PMWPGIC8vD6WlpRg/fjwyMjLQq1evhq6RiIiIqNbq/T43HTp0wPz58xuyFmoEQgjEx8dzNb3NTFn1bsLcLWUvznc1mLsazN1avZuboqIi7NixA3l5eTXeqXD8+PGXXRg1DE3TEBMTo7qMgCMh4DmjuorAw/muBnNXg7lbq1dz8/e//x3jxo3D6dOnERER4dM1CiHY3DQhhmHg0KFD6NGjB1fT20gTEv3bmdidp3G3lI0439Vg7mowd2v1WnPz61//Gr/4xS9w+vRpFBUVobCw0Pvnwg/Boqbhwg9PI3sIACEtwN1SCnC+q8Hc1WDu/tWrufnhhx/w8MMPIyQkpKHrISIiIros9WpukpOTsWvXroauhYiIiOiy1WvNzciRI/Gb3/wGBw4cQFJSElq2bOnz/TvvvLNBiqPLp2kaunbtavnpq9Q4DAnsz9csP2GcGgfnuxrMXQ3mbq1ezc2UKVMAAM8880yN7wkh6vwRDNR4hBCIiIhQXUYAEiisUF1D4OF8V4O5q8HcrdWr3TNN0/IPG5umxTAMpKWl8brYTBcSQzsY0AVv3diJ810N5q4Gc7fGe1kBgBNfDZ1bpZTgfFeDuavB3P2r95v4lZWVYePGjTh69CgqKyt9vvfwww9fdmFERERE9VGv5mbv3r24/fbbcebMGZSVlSE6Ohr5+fkICQlBu3bt2NwQERGRMvV6WerRRx/FqFGjUFhYiFatWuGrr77C999/j/79++P3v/99Q9dIl0HTNCQkJHA1vc0MCezM5W4pu3G+q8Hc1WDu1uqVyL59+/DrX/8amqZB13VUVFQgPj4eL774In772982dI10mYKCglSXEJAq+FK4EpzvajB3NZi7f/Vqblq2bOntFNu1a4ejR48CAFq3bo2cnJyGq44um2maSEtLq/HhptS4dAEM7WByUbHNON/VYO5qMHdr9Vpzc80112Dnzp3o0aMHhg0bhjlz5iA/Px/vvPMOevXq1dA1EhEREdVave7czJ8/H+3btwcAzJs3D1FRUZg6dSpOnjyJ119/vUELJCIiIqqLet25GTBggPe/27VrhzVr1jRYQURERESXo153bm6++WYUFRXVGC8pKcHNN998uTVRA9I0DUlJSVxNbzNDAluOc7eU3Tjf1WDuajB3a/VKZMOGDTXeuA8AysvLsXnz5ssuihqWv2tFjS9YV11BYOJ8V4O5q8Hc/avTy1L79+/3/veBAwfg8Xi8XxuGgTVr1iAuLq7hqqPLZpomMjMzkZSUBF3n37Z20QUwMNbk3Rubcb6rwdzVYO7W6tTc9O3bF0IICCH8vvzUqlUrvPrqqw1WHBEREVFd1am5OXLkCKSU6Nq1K3bs2IG2bdt6vxcUFIR27dqxeyQiIiKl6tTcdOrUCWfPnsWECRMQExODTp06NVZd1IDYcKrBl6PU4HxXg7mrwdz9q/OC4pYtW2L16tWNUQs1Al3X+XqsAoYU2HJchyH5FsV24nxXg7mrwdyt1Wu31F133YWPPvqogUuhxiClRElJCaTkbQR7SUQFSwDM3U6c72owdzWYu7V6vYlfjx498Mwzz+DLL79E//79ERoa6vP9hx9+uEGKo8tnmiaysrLY3dtMF0DvNtwtZTfOdzWYuxrM3Vq9mpu33noLkZGR2L17N3bv3u3zPSEEmxsiIiJSpl7NzZEjRxq6DiIiIqIGcdnv2Syl5Ot9TZzL5VJdQsCRAM6c44obFTjf1WDuajB3/+rd3PzpT39CUlISWrVqhVatWqF379545513GrI2agC6riMxMZGvx9rMlAI7c3WY3C1lK853NZi7GszdWr2am4ULF2Lq1Km4/fbb8cEHH+CDDz7Abbfdhoceeggvv/xyQ9dIl8E0TZw6dQqmaaouJaAISLhDJATv3diK810N5q4Gc7dWrzU3r776KpYuXYrx48d7x+68805cffXVePrpp/Hoo482WIF0eaSUyMnJQWRkpOpSAoomgIQoEyf/zd1SduJ8V4O5q8HcrdXrzs2JEycwZMiQGuNDhgzBiRMnLrsoIiIiovqqV3PTvXt3fPDBBzXGV61ahR49elx2UURERET1Va+XpVJTUzF27Fhs2rQJ1113HQDgyy+/xPr16/02PaRWeHi46hICjgRQWM4VNypwvqvB3NVg7v7Vq7n57//+b2zfvh0vv/yy92MYevbsiR07duCaa65pyProMum6jm7duqkuI+CYUmD/Ke6UshvnuxrMXQ3mbq1ezQ0A9O/fH3/+858bshZqBKZpIi8vD+3atYOmXfbbGlEtCUh0DJc4WiogwSbHLpzvajB3NZi7tXo3N4ZhYPXq1Th48CAA4KqrrsJdd92FFi3q/ZTUCKSU8Hg8aNu2repSAoomgM4REsdOC+6WshHnuxrMXQ3mbq1enUh6ejruvPNOeDweJCQkAABeeOEFtG3bFn//+9/Rq1evBi2SiIiIqLbqdR9r8uTJuPrqq3Hs2DHs2bMHe/bsQU5ODnr37o0HH3ywoWskIiIiqrV63bnZt28fdu3ahaioKO9YVFQU5s2bh4EDBzZYcXT5hBCIjo6GEFz3YScpAU+ZAD92zV6c72owdzWYu7V63bm58sorkZubW2M8Ly8P3bt3v+yiqOFomoaOHTtysZnNTAhkFmkwuZjYVpzvajB3NZi7tXolsmDBAjz88MP48MMPcezYMRw7dgwffvghHnnkEbzwwgsoKSnx/iG1TNPE0aNH+dkjNtMgkRBpQuM73diK810N5q4Gc7dWr5el7rjjDgDAmDFjvLfD5H/uv48aNcr7tRAChmE0RJ1UT1JKFBQUIC4uTnUpAUUIwB0q8V2xAPsb+3C+q8Hc1WDu1urV3HzxxRcNXQcRERFRg6hXczNs2LCGroOIiIioQdT7HffKy8uxf/9+5OXl1Xi9784777zswqhhCCHgdru5mt5mpgSySwRMviRlK853NZi7GszdWr2amzVr1mD8+PHIz8+v8T2us2laNE2D2+1WXUbAkRD4vpS/cOzG+a4Gc1eDuVur126pX/3qV7jnnntw4sQJmKbp84eNTdNiGAYOHz7M62IzTUj0jjGhCd66sRPnuxrMXQ3mbq1ezU1ubi5SUlIQGxvb0PVQIygtLVVdQsARAKJc/MhMFTjf1WDuajB3/+rV3PzsZz/Dhg0bGrgUIiIiostXrzU3r732Gu655x5s3rwZSUlJaNmypc/3H3744QYpjoiIiKiu6tXc/OUvf8E///lPuFwubNiwwWelthCCzU0TIoRAfHw8V9PbzJRAZqHG3VI243xXg7mrwdyt1au5efLJJ5GamoqZM2fyMy2aOE3TEBMTo7qMgCMh4DmjuorAw/muBnNXg7lbq1dnUllZibFjx7KxaQYMw0BGRgZX09tMExIDYw3ulrIZ57sazF0N5m6tXt3JhAkTsGrVqoauhRpJeXm56hICjgAQ0gLcLaUA57sazF0N5u5fvV6WMgwDL774ItauXYvevXvXWFC8cOHCBimOiIiIqK7qdecmLS0N11xzDTRNwzfffIO9e/f6/KmrxYsXo3PnznC5XBg0aBB27NhRq8e9//77EEJg9OjRdf6ZRERE5EzKPxV81apVSElJwbJlyzBo0CAsWrQIycnJyMzMRLt27Swfl52djcceewzXX399g9XiRJqmoWvXrlwfZTNDAvvzNRhccmMrznc1mLsazN1anZqbu++++0ePEULgb3/7W62fc+HChZgyZQomTZoEAFi2bBk++eQTLF++HDNnzvT7GMMwMG7cOKSmpmLz5s0oKiqq9c8LNEIIREREqC4jAAkUVqiuIfBwvqvB3NVg7tbq1Ny0bt26QX94ZWUldu/ejVmzZnnHNE3D8OHDsW3bNsvHPfPMM2jXrh0eeOABbN68+ZI/o6KiAhUV5/+WKSkpAVDVIFWvMBdCQNM0mKYJKc//U7t6/OKV6Fbjmqb5/eDQ6q764k9PtxrXdR1SSr/jF9doNV5d49mzZ3HgwAH07NkTuq474pz8jQNVO5QuXMBryqot2RePV91NEdAv2slUfZdFv2gVsPW4ACB9xiUAUwq0ECYGtzex3aPBkMI7LiChXXi8BEwIaJC48K0qrGpviHMyDEPZdWrsuWcYBtLT073z3Qnn5G+8qZ2TaZr45ptvfHJv7ufUHK6TlBLp6elITEz05t7cz6ku1+lS6tTcrFixoi6H/6j8/HwYhlHjM6piY2ORkZHh9zFbtmzBW2+9hX379tXqZyxYsACpqak1xtPT0xEWFgYAiI6ORseOHXHs2DEUFBR4j3G73XC73cjOzvb5/I74+HjExMTg0KFDPivVu3btioiICBw4cMDnIiQkJCAoKAhpaWk+NSQlJaGyshKZmZneMV3XkZSUhNLSUmRlZXnHXS4XEhMTUVhYiJycHO94eHg4unXrhry8PHg8Hu949TkdP34c+fn5SE9PhxDCEefk7zoBQK9oiSjX+f9RMgs1eM4A/duZCLlgpu/P11BYAQxub/r8pb8zV0OFAQzt4Ps/7ZbjGoJ1YGDs+XFDAluO64gKBnq3OT9+5hywM1dHbAgQFwYMaS8hIVFYLrD/lEDHcInOEedr9JQJZBYJ9IiUcIeeH88uqfpU8cY4p7S0NGXXqbHnXkVFhc98d8I5NYfrFBoailOnTnlzd8I5NYfr1LZtWxQVFfnk3tzP6VLX6fDhw6gtIS9us2x0/PhxxMXFYevWrRg8eLB3/PHHH8fGjRuxfft2n+NLS0vRu3dvLFmyBCNGjAAATJw4EUVFRfjoo4/8/gx/d27i4+NRUFDgvZ3n5G7/7Nmz+Oabb3D11Vc7+s7NiHmfNbk7N0M7mNh6oundufn7rBHN4l9l9b1zk5aW5p3vTjgnf+NN7ZxM08T+/ft9cm/u59QcrpOUskbuzf2cLnWdCgsLER0djeLi4h99Oa5eC4obSps2baDrOnJzc33Gc3Nzvf8av9Dhw4eRnZ2NUaNGeceqA27RogUyMzPRrVs3n8cEBwcjODi4xnPpuu6dDNWsFmVdfJwd49X/6ryYVY2XGq9+rgufr7mfkz+m9P+uMlbjhuW432GLceF3XEJAQsCQwufnSIvjTYiqzuji8UY4px+bB419nRp7jvmb7835nKzGm9o5+cu9rs/T1M6pqV8nwzAaJPe6jje16+T3uWt9ZCMICgpC//79sX79eu+YaZpYv369z52caomJiUhLS8O+ffu8f+68807cdNNN2LdvH+Lj4+0sv1nQNA0JCQlcTW8zQ1a9JMTdUvbifFeDuavB3K0pvXMDACkpKZgwYQIGDBiAa6+9FosWLUJZWZl399T48eMRFxeHBQsWwOVyoVevXj6Pj4yMBIAa43ReUFCQ6hICUgXfEV0Jznc1mLsazN0/5e3e2LFj8fvf/x5z5sxB3759sW/fPqxZs8a7yPjo0aM4ceKE4iqbL9M0kZaWVuP1UWpcuqhaxHvxOh1qXJzvajB3NZi7NeV3bgBg+vTpmD59ut/vbdiw4ZKPXblyZcMXRERERM2W8js3RERERA2JzQ0RERE5Cpsbh9M0DUlJSVxNb7OqN/njbim7cb6rwdzVYO7WmEgAqKysVF1CQAqu/VsyUAPifFeDuavB3P1jc+NwpmkiMzOTq+ltpouqj2vgbil7cb6rwdzVYO7W2NwQERGRo7C5ISIiIkdhcxMA6vJ5HNRwuJhYDc53NZi7GszdvybxJn7UeKo/op7sZUiBLcf5S8dunO9qMHc1mLs13rlxOCklSkpKanwMPTU2iahgCb8f9U2NhvNdDeauBnO3xubG4UzTRFZWFlfT20wXQO823C1lN853NZi7GszdGpsbIiIichQ2N0REROQobG4CgMvlUl1CwJEAzpzjihsVON/VYO5qMHf/uFvK4XRdR2JiouoyAo4pBXbmcreU3Tjf1WDuajB3a7xz43CmaeLUqVNccGYzAQl3iITgvRtbcb6rwdzVYO7W2Nw4nJQSOTk53CpoM00ACVEmNO6WshXnuxrMXQ3mbo3NDRERETkKmxsiIiJyFDY3ASA8PFx1CQFHAigs54obFTjf1WDuajB3/7hbyuF0XUe3bt1UlxFwTCmw/xQX3NiN810N5q4Gc7fGOzcOZ5omPB4PV9PbTECiU7jJ3VI243xXg7mrwdytsblxOCklPB4PV9PbTBNA5wjJ3VI243xXg7mrwdytsbkhIiIiR2FzQ0RERI7C5sbhhBCIjo6GEHx9xE5SAp4yAd4tthfnuxrMXQ3mbo27pRxO0zR07NhRdRkBx4RAZhF/4diN810N5q4Gc7fGOzcOZ5omjh49ytX0NtMgkRBpQuNuKVtxvqvB3NVg7tbY3DiclBIFBQVcTW8zIQB3qATvFtuL810N5q4Gc7fG5oaIiIgchc0NEREROQqbG4cTQsDtdnM1vc1MCWSXCJi8W2wrznc1mLsazN0ad0s5nKZpcLvdqssIOBIC35fyF47dON/VYO5qMHdrvHPjcIZh4PDhwzAMQ3UpAUUTEr1jTGiCt27sxPmuBnNXg7lbY3MTAEpLS1WXEHAEgCiXBO/d2I/zXQ3mrgZz94/NDRERETkKmxsiIiJyFDY3DieEQHx8PFfT28yUQGahxt1SNuN8V4O5q8HcrXG3lMNpmoaYmBjVZQQcCQHPGdVVBB7OdzWYuxrM3Rrv3DicYRjIyMjganqbaUJiYKzB3VI243xXg7mrwdytsbkJAOXl5apLCDgCQEgLcLeUApzvajB3NZi7f2xuiIiIyFHY3BAREZGjsLlxOE3T0LVrV2gaL7WdDAnsz9dgcMmNrTjf1WDuajB3a9wt5XBCCERERKguIwAJFFaoriHwcL6rwdzVYO7W2O45nGEYSEtL42p6m+lCYmgHAzp3S9mK810N5q4Gc7fG5iYAcOKroXOrlBKc72owdzWYu39sboiIiMhR2NwQERGRo7C5cThN05CQkMDV9DYzJLAzl7ul7Mb5rgZzV4O5W2MiASAoKEh1CQGpgi+FK8H5rgZzV4O5+8fmxuFM00RaWhpM01RdSkDRBTC0g8lFxTbjfFeDuavB3K2xuSEiIiJHYXNDREREjsLmhoiIiByFzY3DaZqGpKQkrqa3mSGBLce5W8punO9qMHc1mLs1JhIAKisrVZcQkIJ11RUEJs53NZi7GszdPzY3DmeaJjIzM7ma3ma6AAbGcreU3Tjf1WDuajB3a2xuiIiIyFHY3BAREZGjNInmZvHixejcuTNcLhcGDRqEHTt2WB77xhtv4Prrr0dUVBSioqIwfPjwSx5PgK5z8YcKXEysBue7GsxdDebun/LmZtWqVUhJScHcuXOxZ88e9OnTB8nJycjLy/N7/IYNG3Dffffhiy++wLZt2xAfH49bb70VP/zwg82VNw+6riMpKYn/A9jMkAJbjuswJBfd2InzXQ3mrgZzt6a8uVm4cCGmTJmCSZMm4aqrrsKyZcsQEhKC5cuX+z3+3XffxS9/+Uv07dsXiYmJePPNN2GaJtavX29z5c2DlBIlJSWQkrcR7CURFSwBMHc7cb6rwdzVYO7WlDY3lZWV2L17N4YPH+4d0zQNw4cPx7Zt22r1HGfOnMHZs2cRHR3dWGU2a6ZpIisri6vpbaYLoHcb7payG+e7GsxdDeZurYXKH56fnw/DMBAbG+szHhsbi4yMjFo9xxNPPIEOHTr4NEgXqqioQEVFhffrkpISAIBhGDCMqo9tFkJA0zSYpunTAVePVx/3Y+OapkEI4XccQI0JaDWu6zqklH7HL67Ravzic7r4XJ1wTheOA4AmJC7sJUwJSIga41VrYQR04fszq9fIXNyQWI8LANJnXAIwpYCAhID0/owLx7ULj5eACQENEuKCcavaG+KcDMNQdp0ae+5JKX3muxPOyd94UzsnADVyb+7n1ByuE1Az9+Z+TnW5TpeitLm5XM8//zzef/99bNiwAS6Xy+8xCxYsQGpqao3x9PR0hIWFAQCio6PRsWNHHDt2DAUFBd5j3G433G43srOzUVpa6h2Pj49HTEwMDh06hPLycu94165dERERgQMHDvhchISEBAQFBSEtLc2nhqSkJFRWViIzM9M7Vv0aamlpKbKysrzjLpcLiYmJKCwsRE5Ojnc8PDwc3bp1Q15eHjwej3e8+pyOHz+OgoICpKenQwjhiHPyd50AoFe0RJTr/P8omYUaPGeA/u1MhFww0/fnayisAAa3972zsjNXQ4VR9WneF9pyXEOwXvW+NdWq3oFYR1Rw1R2aamfOATtzdcSGAO5QYEh7CQmJwnKB/acEOoZLdI44X6OnTCCzSKBHpIQ79Px4donA96WiUc4pLS1N2XVq7LlXUVHhM9+dcE7N4TqFhoaisLDQm7sTzqk5XKe2bduitLTUJ/fmfk6Xuk6HDx9GbQmp8MW6yspKhISE4MMPP8To0aO94xMmTEBRURE+/vhjy8f+/ve/x3PPPYd//etfGDBggOVx/u7cxMfHo6CgABEREQCc3e2fPXsWhw4dQvfu3aHruiPOyd/4iHmfNak7N7ow0b+dib0nNZhSNKk7N3+fNaJZ/KusPnPPMAx8++233vnuhHPyN97Uzqn6zeQuzL25n1NzuE5Syhq5N/dzutR1KiwsRHR0NIqLi71/f1tR2twAwKBBg3Dttdfi1VdfBVAVZMeOHTF9+nTMnDnT72NefPFFzJs3D2vXrsVPfvKTOv28kpIStG7dulbhUPOR/OwnqktoNtbOHqm6BCKiOqvL39/Kd0ulpKTgjTfewNtvv42DBw9i6tSpKCsrw6RJkwAA48ePx6xZs7zHv/DCC5g9ezaWL1+Ozp07w+PxwOPx4PTp06pOoUkzTROnTp3igjObCUi4Q6rW3ZB9ON/VYO5qMHdrytfcjB07FidPnsScOXPg8XjQt29frFmzxrvI+OjRoz6feLp06VJUVlbiZz/7mc/zzJ07F08//bSdpTcLUkrk5OQgMjJSdSkBRRNAQpSJk//mJ4PbifNdDeauBnO3pry5AYDp06dj+vTpfr+3YcMGn6+zs7MbvyAiIiJqtpS/LEVERETUkNjcBIDw8HDVJQQcCaCwnCtuVOB8V4O5q8Hc/WsSL0tR49F1Hd26dVNdRsAxZdX72pC9ON/VYO5qMHdrvHPjcKZpwuPxcDW9zQQkOoWb3C1lM853NZi7GszdGpsbh5NSwuPx8IPVbKYJoHOE7xv2UePjfFeDuavB3K2xuSEiIiJHYXNDREREjsLmxuGEEIiOjvZ+qBrZQ8qqD8Xk3WJ7cb6rwdzVYO7WuFvK4TRNQ8eOHVWXEXBMVH3aN9mL810N5q4Gc7fGOzcOZ5omjh49ytX0NtMgkRBpQuNuKVtxvqvB3NVg7tbY3DiclBIFBQVcTW8zIQB3qATvFtuL810N5q4Gc7fG5oaIiIgchc0NEREROQqbG4cTQsDtdnM1vc1MCWSXCJi8W2wrznc1mLsazN0ad0s5nKZpcLvdqssIOBIC35fyF47dON/VYO5qMHdrvHPjcIZh4PDhwzAMQ3UpAUUTEr1jTGiCt27sxPmuBnNXg7lbY3MTAEpLS1WXEHAEgCiXBO/d2I/zXQ3mrgZz94/NDRERETkKmxsiIiJyFDY3DieEQHx8PFfT28yUQGahxt1SNuN8V4O5q8HcrXG3lMNpmoaYmBjVZQQcCQHPGdVVBB7OdzWYuxrM3Rrv3DicYRjIyMjganqbaUJiYKzB3VI243xXg7mrwdytsbkJAOXl5apLCDgCQEgLcLeUApzvajB3NZi7f2xuiIiIyFHY3BAREZGjsLlxOE3T0LVrV2gaL7WdDAnsz9dgcMmNrTjf1WDuajB3a9wt5XBCCERERKguIwAJFFaoriHwcL6rwdzVYO7W2O45nGEYSEtL42p6m+lCYmgHAzp3S9mK810N5q4Gc7fG5iYAcOKroXOrlBKc72owdzWYu39sboiIiMhR2NwQERGRo7C5cThN05CQkMDV9DYzJLAzl7ul7Mb5rgZzV4O5W2MiASAoKEh1CQGpgi+FK8H5rgZzV4O5+8fmxuFM00RaWhpM01RdSkDRBTC0g8lFxTbjfFeDuavB3K2xuSEiIiJHYXNDREREjsLmhoiIiByFzY3DaZqGpKQkrqa3mSGBLce5W8punO9qMHc1mLs1JhIAKisrVZcQkIJ11RUEJs53NZi7GszdPzY3DmeaJjIzM7ma3ma6AAbGcreU3Tjf1WDuajB3a2xuiIiIyFHY3BAREZGjsLkJALrOxR8qcDGxGpzvajB3NZi7fy1UF0CNS9d1JCUlqS4j4BhSYMtx/tKxG+e7GsxdDeZujXduHE5KiZKSEkjJ2wj2kogKlgCYu50439Vg7mowd2tsbhzONE1kZWVxNb3NdAH0bsPdUnbjfFeDuavB3K2xuSEiIiJHYXNDREREjsLmJgC4XC7VJQQcCeDMOa64UYHzXQ3mrgZz94+7pRxO13UkJiaqLiPgmFJgZy53S9mN810N5q4Gc7fGOzcOZ5omTp06xQVnNhOQcIdICN67sRXnuxrMXQ3mbo3NjcNJKZGTk8OtgjbTBJAQZULjbilbcb6rwdzVYO7W2NwQERGRo7C5ISIiIkdhcxMAwsPDVZcQcCSAwnKuuFGB810N5q4Gc/ePu6UcTtd1dOvWTXUZAceUAvtPccGN3Tjf1WDuajB3a7xz43CmacLj8XA1vc0EJDqFm9wtZTPOdzWYuxrM3RqbG4eTUsLj8XA1vc00AXSOkNwtZTPOdzWYuxrM3RqbGyIiInIUNjdERETkKFxQ3MCSn/1EdQk+NEj0iJQ49PdjMNG0XiNZO3uk6hIajZSAp0yAd4vtJYRAdHQ0hGhac93pmLsazN1ak7hzs3jxYnTu3BkulwuDBg3Cjh07Lnn8X//6VyQmJsLlciEpKQmffvqpTZU2PyYEMou0JtfYOB1zV0PTNHTs2BGa1iR+tQUM5q4Gc7emPJFVq1YhJSUFc+fOxZ49e9CnTx8kJycjLy/P7/Fbt27FfffdhwceeAB79+7F6NGjMXr0aHzzzTc2V948aJBIiDShcdeOrZi7GqZp4ujRo9w9YjPmrgZzt6a8uVm4cCGmTJmCSZMm4aqrrsKyZcsQEhKC5cuX+z3+lVdewW233Ybf/OY36NmzJ5599ln069cPr732ms2VNw9CAO5QCd61tBdzV0NKiYKCAu4esRlzV4O5W1O65qayshK7d+/GrFmzvGOapmH48OHYtm2b38ds27YNKSkpPmPJycn46KOPGrNUIiIKcE1tTaUuJIZ2MPHbfxyDIZvWv6RUr6lU2tzk5+fDMAzExsb6jMfGxiIjI8PvYzwej9/jPR6P3+MrKipQUVHh/bq4uBgAUFhYCMMwAFQtytI0DaZp+nTA1ePVx/3YuKZpOFd+Bprw7aLN/3x58XueWI8LADXfI8WUAgI17wb4HZdV6z4gTFT824RZocGUVQtcJUSNGq3Gq2q0Gr+8cyosLISu6zVyB+B3/FLXyV/uKs6pelxelHv1uNV10iBx4fKcxrxOhYWF3tfoL76dres6pJR+xxviOtX1/ychhN9xf7VXP0dpaal3bjnhnPyN/+z3/0JD/Y5oiLmnCWBIewMz/5bp/UtW5e+9C8/pg1//tMGu07nysiZxTkDV9TAFUPFvw+f3jMrfexfWXlhYCKBh/3+qfs7a3Kly/G6pBQsWIDU1tcZ4586d7S9GkX+qLsBC9DzVFTSudaoLsOD03EmNf6kuwILT53sg5l5aWorWrVtf8hilzU2bNm2g6zpyc3N9xnNzc+F2u/0+xu121+n4WbNm+byMZZomCgoKEBMTExDb50pKShAfH4+cnBxERESoLidgMHc1mLsazF2NQMtdSonS0lJ06NDhR49V2twEBQWhf//+WL9+PUaPHg2gqvlYv349pk+f7vcxgwcPxvr16/HII494x9atW4fBgwf7PT44OBjBwcE+Y5GRkQ1RfrMSEREREJO/qWHuajB3NZi7GoGU+4/dsamm/GWplJQUTJgwAQMGDMC1116LRYsWoaysDJMmTQIAjB8/HnFxcViwYAEAYMaMGRg2bBheeukljBw5Eu+//z527dqFP/7xjypPg4iIiJoI5c3N2LFjcfLkScyZMwcejwd9+/bFmjVrvIuGjx496vMGRUOGDMF7772Hp556Cr/97W/Ro0cPfPTRR+jVq5eqUyAiIqImRHlzAwDTp0+3fBlqw4YNNcbuuece3HPPPY1clTMEBwdj7ty5NV6ao8bF3NVg7mowdzWYuzUh+e4/RERE5CDK36GYiIiIqCGxuSEiIiJHYXNDREREjsLmhoiIiByFzY3DLV68GJ07d4bL5cKgQYOwY8cO1SU52qZNmzBq1Ch06NABQgh+oKtNFixYgIEDByI8PBzt2rXD6NGjkZmZqbosx1u6dCl69+7tfRO5wYMH47PPPlNdVkB5/vnnIYTweWNbYnPjaKtWrUJKSgrmzp2LPXv2oE+fPkhOTkZeXp7q0hyrrKwMffr0weLFi1WXElA2btyIadOm4auvvsK6detw9uxZ3HrrrSgrK1NdmqNdccUVeP7557F7927s2rULN998M+666y6kp6erLi0g7Ny5E6+//jp69+6tupQmh1vBHWzQoEEYOHAgXnvtNQBVH20RHx+PX/3qV5g5c6bi6pxPCIHVq1d7P1qE7HPy5Em0a9cOGzduxA033KC6nIASHR2N3/3ud3jggQdUl+Jop0+fRr9+/bBkyRI899xz6Nu3LxYtWqS6rCaDd24cqrKyErt378bw4cO9Y5qmYfjw4di2bZvCyogaX3FxMYCqv2jJHoZh4P3330dZWZnlZ/1Rw5k2bRpGjhzp8zuezmsS71BMDS8/Px+GYXg/xqJabGwsMjIyFFVF1PhM08QjjzyC6667jh/LYoO0tDQMHjwY5eXlCAsLw+rVq3HVVVepLsvR3n//fezZswc7d+5UXUqTxeaGiBxl2rRp+Oabb7BlyxbVpQSEhIQE7Nu3D8XFxfjwww8xYcIEbNy4kQ1OI8nJycGMGTOwbt06uFwu1eU0WWxuHKpNmzbQdR25ubk+47m5uXC73YqqImpc06dPxz/+8Q9s2rQJV1xxhepyAkJQUBC6d+8OAOjfvz927tyJV155Ba+//rriypxp9+7dyMvLQ79+/bxjhmFg06ZNeO2111BRUQFd1xVW2DRwzY1DBQUFoX///li/fr13zDRNrF+/nq+Hk+NIKTF9+nSsXr0an3/+Obp06aK6pIBlmiYqKipUl+FYt9xyC9LS0rBv3z7vnwEDBmDcuHHYt28fG5v/4J0bB0tJScGECRMwYMAAXHvttVi0aBHKysowadIk1aU51unTp/Hdd995vz5y5Aj27duH6OhodOzYUWFlzjZt2jS89957+PjjjxEeHg6PxwMAaN26NVq1aqW4OueaNWsWRowYgY4dO6K0tBTvvfceNmzYgLVr16ouzbHCw8NrrCULDQ1FTEwM15hdgM2Ng40dOxYnT57EnDlz4PF40LdvX6xZs6bGImNqOLt27cJNN93k/TolJQUAMGHCBKxcuVJRVc63dOlSAMCNN97oM75ixQpMnDjR/oICRF5eHsaPH48TJ06gdevW6N27N9auXYuf/vSnqkujAMf3uSEiIiJH4ZobIiIichQ2N0REROQobG6IiIjIUdjcEBERkaOwuSEiIiJHYXNDREREjsLmhoiIiByFzQ0R0WUSQuCjjz5SXQYR/QebGyLya+LEiRBC1Phz4cdLXI6VK1ciMjKyQZ6rviZOnIjRo0crrYGIGh4/foGILN12221YsWKFz1jbtm0VVWPt7NmzaNmypeoyiKiJ4J0bIrIUHBwMt9vt86f6U4c//vhj9OvXDy6XC127dkVqairOnTvnfezChQuRlJSE0NBQxMfH45e//CVOnz4NANiwYQMmTZqE4uJi7x2hp59+GoD/l3giIyO9n82VnZ0NIQRWrVqFYcOGweVy4d133wUAvPnmm+jZsydcLhcSExOxZMmSOp3vjTfeiIcffhiPP/44oqOj4Xa7vXVVO3ToEG644Qa4XC5cddVVWLduXY3nycnJwZgxYxAZGYno6GjcddddyM7OBgBkZGQgJCQE7733nvf4Dz74AK1atcKBAwfqVC8R+cfmhojqbPPmzRg/fjxmzJiBAwcO4PXXX8fKlSsxb9487zGapuEPf/gD0tPT8fbbb+Pzzz/H448/DgAYMmQIFi1ahIiICJw4cQInTpzAY489VqcaZs6ciRkzZuDgwYNITk7Gu+++izlz5mDevHk4ePAg5s+fj9mzZ+Ptt9+u0/O+/fbbCA0Nxfbt2/Hiiy/imWee8TYwpmni7rvvRlBQELZv345ly5bhiSee8Hn82bNnkZycjPDwcGzevBlffvklwsLCcNttt6GyshKJiYn4/e9/j1/+8pc4evQojh07hoceeggvvPACrrrqqjrVSkQWJBGRHxMmTJC6rsvQ0FDvn5/97GdSSilvueUWOX/+fJ/j33nnHdm+fXvL5/vrX/8qY2JivF+vWLFCtm7dusZxAOTq1at9xlq3bi1XrFghpZTyyJEjEoBctGiRzzHdunWT7733ns/Ys88+KwcPHnzJc7zrrru8Xw8bNkwOHTrU55iBAwfKJ554Qkop5dq1a2WLFi3kDz/84P3+Z5995lPzO++8IxMSEqRpmt5jKioqZKtWreTatWu9YyNHjpTXX3+9vOWWW+Stt97qczwRXR6uuSEiSzfddBOWLl3q/To0NBQA8PXXX+PLL7/0uVNjGAbKy8tx5swZhISE4F//+hcWLFiAjIwMlJSU4Ny5cz7fv1wDBgzw/ndZWRkOHz6MBx54AFOmTPGOnzt3Dq1bt67T8/bu3dvn6/bt2yMvLw8AcPDgQcTHx6NDhw7e7w8ePNjn+K+//hrfffcdwsPDfcbLy8tx+PBh79fLly/HlVdeCU3TkJ6eDiFEneokImtsbojIUmhoKLp3715j/PTp00hNTcXdd99d43sulwvZ2dm44447MHXqVMybNw/R0dHYsmULHnjgAVRWVl6yuRFCQErpM3b27Fm/tV1YDwC88cYbGDRokM9x1WuEauvihclCCJimWevHnz59Gv379/euA7rQhYuxv/76a5SVlUHTNJw4cQLt27evU51EZI3NDRHVWb9+/ZCZmem38QGA3bt3wzRNvPTSS9C0qqV9H3zwgc8xQUFBMAyjxmPbtm2LEydOeL8+dOgQzpw5c8l6YmNj0aFDB2RlZWHcuHF1PZ1a69mzJ3Jycnyaka+++srnmH79+mHVqlVo164dIiIi/D5PQUEBJk6ciCeffBInTpzAuHHjsGfPHrRq1arRaicKJFxQTER1NmfOHPzpT39Camoq0tPTcfDgQbz//vt46qmnAADdu3fH2bNn8eqrryIrKwvvvPMOli1b5vMcnTt3xunTp7F+/Xrk5+d7G5ibb74Zr732Gvbu3Ytdu3bhoYceqtU279TUVCxYsAB/+MMf8O233yItLQ0rVqzAwoULG+y8hw8fjiuvvBITJkzA119/jc2bN+PJJ5/0OWbcuHFo06YN7rrrLmzevBlHjhzBhg0b8PDDD+PYsWMAgIceegjx8fF46qmnsHDhQhiGUecF1URkjc0NEdVZcnIy/vGPf+Cf//wnBg4ciJ/85Cd4+eWX0alTJwBAnz59sHDhQrzwwgvo1asX3n33XSxYsMDnOYYMGYKHHnoIY8eORdu2bfHiiy8CAF566SXEx8fj+uuvx/3334/HHnusVmt0Jk+ejDfffBMrVqxAUlIShg0bhpUrV6JLly4Ndt6apmH16tX497//jWuvvRaTJ0/2WXcEACEhIdi0aRM6duyIu+++Gz179sQDDzyA8vJyRERE4E9/+hM+/fRTvPPOO2jRogVCQ0Px5z//GW+88QY+++yzBquVKJAJefGL20RERETNGO/cEBERkaOwuSEiIiJHYXNDREREjsLmhoiIiByFzQ0RERE5CpsbIiIichQ2N0REROQobG6IiIjIUdjcEBERkaOwuSEiIiJHYXNDREREjsLmhoiIiBzl/wOED5GtL1LMMgAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Create bar plot\n",
"plt.bar(np.arange(len(forrest.featureImportance)), forrest.featureImportance, color='steelblue')\n",
"\n",
"# Add labels and title\n",
"plt.xlabel('Feature Index')\n",
"plt.ylabel('Importance')\n",
"plt.title('Feature Importance')\n",
"\n",
"# Add grid\n",
"plt.grid(True, linestyle='--', alpha=0.6)\n",
"\n",
"# Show plot\n",
"plt.show()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluating predictions\n",
"\n",
"Depending on the task at hand we create a confusion matrix (classification) or simple metrics (regression). Since the number of classes is fixed to two, we don't need to change anything here."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━ evaluation ━━━━━━━━━━━━\n",
"————————— confusion matrix —————————\n",
" Class 0 Class 1 \n",
"····································\n",
" Class 0 1593 7 \n",
" 49% 0% \n",
"····································\n",
" Class 1 7 1593 \n",
" 0% 49% \n",
"\n",
"———————————————————————————————— scores ———————————————————————————————\n",
" accuracy precision sensitivity miss rate \n",
"·······································································\n",
" Class 0 0.996 0.996 0.996 0.004 \n",
" Class 1 0.996 0.996 0.996 0.004 \n",
"·······································································\n",
" total 0.996 0.996 0.996 0.004 \n"
]
}
],
"source": [
"if task == 'regressor':\n",
" metrics = RegressionScores(numClasses=2)\n",
" metrics.calcScores(prediction, validTargets, validLabels)\n",
" print(metrics)\n",
"elif task == 'classifier':\n",
" confusion = ConfusionMatrix(numClasses=2)\n",
" confusion.update(prediction, validLabels)\n",
" confusion.percentages()\n",
" confusion.calcScores()\n",
" print(confusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Saving and Loading a Forrest\n",
"\n",
"Forrests can be converted to dictionaries and then saved as a json file. This allows us to load them and re-use them. Also json is a raw text format, which is neat."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━━━━━━━━━━━━━━━ forrest ━━━━━━━━━━━━━━━━━━━━━━━━━━\n",
"voting: Majority, booster: AdaBoosting, bootstrapping: False\n",
"\n",
"————————————————————— tree: 01/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= 2.18, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 3 <= 0.04, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.52, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 3 <= 0.61, samples: 5\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 02/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 0.81, samples: 4\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 3 <= 0.04, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 03/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 1 <= 2.18, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 3 <= 0.05, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 1.96, samples: 2\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= -0.52, samples: 3\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 3 <= 0.61, samples: 5\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 04/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 0 <= 0.81, samples: 4\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 1 <= 1.82, samples: 6\n",
" │ │ ├─feat: 0 <= 2.28, samples: 2\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 2 <= 0.76, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"————————————————————— tree: 05/15 ——————————————————————\n",
"split: CART, impurity: Entropy, leaf: Mode, nodes: 45\n",
"maxDepth: 7, reached depth: 7, minSamplesSplit: 2\n",
"························································\n",
"╴feat: 1 <= 2.19, samples: 9600\n",
" ├─feat: 1 <= 1.70, samples: 4747\n",
" │ └─╴value: 0.0\n",
" │ └─╴feat: 0 <= 1.85, samples: 157\n",
" │ ├─feat: 1 <= 2.18, samples: 143\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 2 <= -0.82, samples: 4\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.82, samples: 14\n",
" │ ├─feat: 0 <= 2.45, samples: 6\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 2 <= -1.00, samples: 2\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 2.81, samples: 4853\n",
" ├─feat: 0 <= 1.30, samples: 107\n",
" │ ├─feat: 4 <= 1.51, samples: 55\n",
" │ │ ├─feat: 3 <= -1.78, samples: 52\n",
" │ │ │ ├─feat: 0 <= 0.17, samples: 2\n",
" │ │ │ │ └─╴value: 1.0\n",
" │ │ │ │ └─╴value: 0.0\n",
" │ │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 3 <= 0.51, samples: 3\n",
" │ │ └─╴value: 1.0\n",
" │ │ └─╴value: 0.0\n",
" │ └─╴feat: 4 <= 0.10, samples: 52\n",
" │ ├─feat: 2 <= -0.57, samples: 5\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴feat: 1 <= 3.64, samples: 4746\n",
" ├─feat: 0 <= 0.38, samples: 366\n",
" │ ├─feat: 0 <= -0.54, samples: 13\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴feat: 4 <= 0.70, samples: 7\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴feat: 4 <= 0.44, samples: 353\n",
" │ ├─feat: 4 <= -0.94, samples: 47\n",
" │ │ └─╴value: 0.0\n",
" │ │ └─╴value: 1.0\n",
" │ └─╴value: 1.0\n",
" └─╴value: 1.0\n",
"\n",
"\n"
]
}
],
"source": [
"ModelIO.save(forrest, 'test')\n",
"newForrest = ModelIO.load('test')\n",
"print(newForrest)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"━━━━━━━━━━━━ evaluation ━━━━━━━━━━━━\n",
"————————— confusion matrix —————————\n",
" Class 0 Class 1 \n",
"····································\n",
" Class 0 1593 7 \n",
" 49% 0% \n",
"····································\n",
" Class 1 7 1593 \n",
" 0% 49% \n",
"\n",
"———————————————————————————————— scores ———————————————————————————————\n",
" accuracy precision sensitivity miss rate \n",
"·······································································\n",
" Class 0 0.996 0.996 0.996 0.004 \n",
" Class 1 0.996 0.996 0.996 0.004 \n",
"·······································································\n",
" total 0.996 0.996 0.996 0.004 \n"
]
}
],
"source": [
"prediction = newForrest.eval(validData)\n",
"\n",
"if task == 'regressor':\n",
" newMetrics = RegressionScores(numClasses=2)\n",
" newMetrics.calcScores(prediction, validTargets, validLabels)\n",
" print(newMetrics)\n",
"elif task == 'classifier':\n",
" newConfusion = ConfusionMatrix(numClasses=2)\n",
" newConfusion.update(prediction, validLabels)\n",
" newConfusion.percentages()\n",
" newConfusion.calcScores()\n",
" print(newConfusion)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Comment\n",
"\n",
"The forrest works as well as the tree code, because it completely builds on it, thus inhereting all it's problems. The progress bar for training trees is very erratic, because it's always set to the current level and because of the recurvise learning process it's jumping between levels. Also if you are using boosting, every tree will be trained twice."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
},
"vscode": {
"interpreter": {
"hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}