@@ -8,80 +8,78 @@ Further reason is the poratbility of the code. If written in pure Numpy it can b
...
@@ -8,80 +8,78 @@ Further reason is the poratbility of the code. If written in pure Numpy it can b
Long term I am planning to extent this Framework with the optional use of something like [Numba](https://numba.pydata.org) or [CuPy](https://cupy.dev) in order to accelerate the code. CuPy allows to run Python Code on GPUs, with the added benefit of running on AMD GPUs.
Long term I am planning to extent this Framework with the optional use of something like [Numba](https://numba.pydata.org) or [CuPy](https://cupy.dev) in order to accelerate the code. CuPy allows to run Python Code on GPUs, with the added benefit of running on AMD GPUs.
## Feature List
## Feature List
The Following list gives an overview of features and their status. Keep in meind, that this was only recently started and that there are many bugs and a lack of optimization.
The Following list gives an overview of features and their status. Keep in meind, that this was only recently started and that there are many bugs and a lack of optimization.
- Neural Network
- Neural Network
- Linear (implemented, works great)
- Linear (implemented, works great)
- Dropout (implemented, works)
- Dropout (implemented, works)
- Bachnorm (1d works, 2d backward is wonky)
- Bachnorm (1d works, 2d backward is wonky)
- Convolution (2D implemented, 1D planned)
- Convolution (2D implemented, 1D planned)
- with 1D convolutions one can analyse time serieses
- with 1D convolutions one can analyse time serieses
- 1D convolutions can be used as simplefied RNN layers
- 1D convolutions can be used as simplefied RNN layers
- Pooling (min/max and avg are implemented)
- Pooling (min/max and avg are implemented)
- Loss Functions (implemented, should work, but not sure)
- Loss Functions (implemented, should work, but not sure)
- Optimizers (implemented)
- Optimizers (implemented)
- sgd, sgd with momentum and nesterov are a bit wonky
- sgd, sgd with momentum and nesterov are a bit wonky
- adagrad, adadelta, rmsprop and adam work properly
- adagrad, adadelta, rmsprop and adam work properly
- Activation (implemented, work)
- Activation (implemented, work)
- Graph (very simple solution implemented)
- Graph (very simple solution implemented)
- Transposed Convolution (implemented, works)
- Transposed Convolution (implemented, works)
- LR Scheduler (implemented)
- LR Scheduler (implemented)
- Tensors with autograd (started)
- Tensors with autograd (implemented, but too slow to use)
- Post Training Quantization (implemented)
- Random Forrest
- Random Forrest
- DecisionTree (Modular)
- DecisionTree (Modular)
- Impurity Measures (Gini, Entropy, MAE, MSE)
- Impurity Measures (Gini, Entropy, MAE, MSE)
- Split Algorithms (CART, ID3, C4.5)
- Split Algorithms (CART, ID3, C4.5)
- Feature Preselection
- Feature Preselection
- Pruning
- Pruning
- Regressor (implemented, untested)
- Regressor (implemented, untested)
- Classifier (implemented)
- Classifier (implemented)
- RandomForret
- RandomForret
- Adding different trees
- Adding different trees
- Voting Algorithms
- Voting Algorithms
- Boosting (ansatz)
- Boosting (ansatz)
- AdaBoosting
- AdaBoosting
- GradientBoosting
- GradientBoosting
- Pruning (ansatz)
- Pruning (ansatz)
- reduce error
- reduce error
- reduce complexity
- reduce complexity
- reduce overfitting
- reduce overfitting
- Self-Organizing Map (minimally implemented)
- Self-Organizing Map (minimally implemented)
- Rectangular and Hexagonal Maps
- Rectangular and Hexagonal Maps
- finding BMUs (implemented)
- finding BMUs (implemented)
- Neighborhood (very simple)
- Neighborhood (very simple)
- Neighborhood Functions (several implemented)
- Neighborhood Functions (several implemented)
- Dataloader from Neural Network (works)
- Dataloader from Neural Network (works)
- learning rate scheduler from Neural Network (works)
- learning rate scheduler from Neural Network (works)
- calculating umatrix (implemented)
- calculating umatrix (implemented)
- evaluating the map (minimally implemented)
- evaluating the map (minimally implemented)
- all applications can be saved/loaded as/from json files
- all applications can be saved/loaded as/from json files
The network code is up and running, forward and backward propagation works with Linear, Convolution and Activation, with Dropout and Pooling should work. I am pretty sure that there a bugs and mistakes. But on simple test data it already works well.
The network code is up and running, forward and backward propagation works with Linear, Convolution and Activation, with Dropout and Pooling should work. I am pretty sure that there a bugs and mistakes. But on simple test data it already works well.
For random forrests and decision trees I found a nearly complete code. I reworked that code, first of all I renamed all variables, functions etc. to more meaningful names.
For random forrests and decision trees I found a nearly complete code. I reworked that code, first of all I renamed all variables, functions etc. to more meaningful names.
Then I refactored to code a lot and made it more Modular, so that users can assamble trees by hand and append them to forrests... makes them less random.
Then I refactored to code a lot and made it more Modular, so that users can assamble trees by hand and append them to forrests... makes them less random.
For the Self-Organizing Map I came up with a solution that can work with batches and reuse some code from the neural network. Next up are validation/prediction methods for the SOM.
For the Self-Organizing Map I came up with a solution that can work with batches and reuse some code from the neural network. Next up are validation/prediction methods for the SOM.
The code is still rather bare bones in general. There is not to much internal checking within the code, thus the user needs to be rather careful.
The code is still rather bare bones in general. There is not to much internal checking within the code, thus the user needs to be rather careful.
For now I am trying to write proper doc-strings and comments. The plan is to have a more coherent naming system and make code more readable to minimize the need for comments.
For now I am trying to write proper doc-strings and comments. The plan is to have a more coherent naming system and make code more readable to minimize the need for comments.
## Running the Code
## Running the Code
There are now different test files, namely:
There are now different test files, namely:
- network-test.py, config-network
- network-test.py, config-network
- som-test.py, config-som
- som-test.py, config-som
- tree-test.py, config-tree
- tree-test.py, config-tree
- forrest-test.py, config-forrest
- forrest-test.py, config-forrest
These can be run using the command line:
These can be run using the command line:
> python network-test.py config-network
> python network-test.py config-network
Keep in mind that you need at least python 3.11.
Keep in mind that you need at least python 3.11.
...
@@ -92,20 +90,17 @@ They rely on dummy data that is generated inside of each notebook.
...
@@ -92,20 +90,17 @@ They rely on dummy data that is generated inside of each notebook.
They have the same names as the python scripts. I recommend starting from here.
They have the same names as the python scripts. I recommend starting from here.
I will write later a proper documentation and some basic tutorials.
I will write later a proper documentation and some basic tutorials.
## Jupyter Notebooks
## Jupyter Notebooks
In this repo are some Jupyter notebooks, which show in a very simple manner how to use the code.
In this repo are some Jupyter notebooks, which show in a very simple manner how to use the code.
Users can easily change application parameters, since they are set in their own boxes.
Users can easily change application parameters, since they are set in their own boxes.
They generate dummy data, where one can set different parameters.
They generate dummy data, where one can set different parameters.
## Getting data files
## Getting data files
There are another tools, one can use to extract data from root files. Firstly I recommend to us [uproot](https://github.com/scikit-hep/uproot5).
There are another tools, one can use to extract data from root files. Firstly I recommend to us [uproot](https://github.com/scikit-hep/uproot5).
Otherwise I wrote a simple class that can extract PXD data from root files. It's based on uproot. This code can be found [here](https://gitlab.ub.uni-giessen.de/gc2052/fromroot).
Otherwise I wrote a simple class that can extract PXD data from root files. It's based on uproot. This code can be found [here](https://gitlab.ub.uni-giessen.de/gc2052/fromroot).
## Installing as a system wide library
## Installing as a system wide library
One has to download the whole repo, navigate there using the terminal and run the following commands:
One has to download the whole repo, navigate there using the terminal and run the following commands:
...
@@ -116,55 +111,55 @@ and
...
@@ -116,55 +111,55 @@ and
> pip3 install .
> pip3 install .
## Sources
## Sources
- Neural Network
- Neural Network
-[ansatz for linear](https://towardsdatascience.com/math-neural-network-from-scratch-in-python-d6da9f29ce65)
-[ansatz for linear](https://towardsdatascience.com/math-neural-network-from-scratch-in-python-d6da9f29ce65)
-[more on linear](https://towardsdatascience.com/creating-neural-networks-from-scratch-in-python-6f02b5dd911)
-[more on linear](https://towardsdatascience.com/creating-neural-networks-from-scratch-in-python-6f02b5dd911)
