README.md 6.4 KB
Newer Older
1
# [ m e m c p ] :panda_face: r e a d   m e 
Amelie Royer's avatar
Amelie Royer committed
2

3
#### Installing AIToolbox (AITB)
Amelie Royer's avatar
Amelie Royer committed
4

5
##### 1. Pre-requisites
Amelie Royer's avatar
Amelie Royer committed
6 7 8 9 10 11
   * GCC 4.9 +
   * cmake [``cmake`` package]
   * Boost version 1.53+ [``libbost-dev`` package]
   * [Eigen 3.2+](http://eigen.tuxfamily.org/index.php?title=Main_Page) library
   * [lp_solve](http://lpsolve.sourceforge.net/5.5/) library [``lp-solve`` package]

12 13
##### 2. Build.
Once the pre-requisites are installed, clone the [AITB repository](https://github.com/Svalorzen/AI-Toolbox), then build it and test the installation with the following commands:
Amelie Royer's avatar
Amelie Royer committed
14 15

```bash
16
cd AIToolbox_root
Amelie Royer's avatar
Amelie Royer committed
17 18 19 20 21 22
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j
ctest -V
```

23 24 25 26 27 28 29 30 31 32 33
##### 3. Optional step  
 (``root`` designates the directory containing this README file).   

```bash
cd AIToolbox_root/include/AIToolbox/MDP/
mv SparseModel.hpp SparseModel_backup.hpp
cp root/Code/AiToolBox/SparseModel.hpp SparseModel.hpp
```

This file contains an updated version of the  constructor ``SparseModel(model)``, and is identical to the default AITB SparseModel otherwise. In fact, when dealing with the foodmart dataset for large state spaces, the probabilities in the transition function might become very low, which conflicts with the Eigen summation that AIToolbox uses by default in the creation of a SparseModel (approximation error). To prevent this, the new constructor can use *Kahan Summation* by setting the second (optional, defaulting to false) argument to true: ``SparseModel(model, true)``.

Amelie Royer's avatar
Amelie Royer committed
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

#### Generating the data (``Data/`` folder)

 * **1. Foodmart dataset.** Infers POMDP parameters from the Foodmart dataset. In aprticular, the reward function is 0 if the recommendation *y* does not match the user's choice *x*. Otherwise, it is the shop profit on selling item *x*.

     ``./prepare_foodmart.py -pl [1] -k [2] -ul [3] -a [4] -t [5] -d [6] -o [7] --norm --help``

     * ``[1]`` Product discretization level (Defaults to 4). Must be between 0 (*fine-grained, 1561 products*)  and 4 (*3 high-level categories*).
     * ``[2]`` History length (Defaults to 2). Must be strictly greater than 1.
     * ``[3]`` User discretization level. For now, only option 0 is implemented (6 user profiles).
     * ``[4]`` Scaling parameter for transition matching the recommended action (Defaults to 1.1). Must be greater than 1.
     * ``[5]`` Proportion of the dataset to keep for parameter inference (Defaults to 0.8).
     * ``[6]`` Path to the Foodmart dataset (either ``tar.gz`` archive or ``Foodmart/data`` directory).
     * ``[7]`` Path to the main output directory (Defaults to ``../Code/Models``).
     * ``[--norm]`` If present, normalize the transition probabilities. (*Note*: Do not use this option for this code, as the transition probabilities are normalized in the C routines anyway).
     * ``[--help]`` displays help about this command

**Example** *(6 environments, 3 actions, 13 states)* :  ``python prepare_foodmart.py -d Foodmart.tar.gz -pl 4 -k 2``





 * **2. Synthetic dataset.** Infers synthetic POMDP parameters to highlight the impact of using multiple environments. The model comprises as many environments as items (actions). More precisely, the *i*-th environment corresponds to users who choose item *i* with a high probability (80%) and the others items with a uniform probability in any circumstances.  he reward is 0 if the recommendation does not match the user's choice, and 1 otherwise.

     ``./prepare_synth.py -n [1] -k [2] -t [3] -o [4] --norm --help``

Amelie Royer's avatar
Amelie Royer committed
61
     * ``[1]`` Number of items/actions (Defaults to 3).
Amelie Royer's avatar
Amelie Royer committed
62 63 64 65 66 67 68 69 70 71 72
     * ``[2]`` History length (Defaults to 2). Must be strictly greater than 1.
     * ``[3]`` Number of test sessions to generate according to the synthetic distribution (Defaults to 2000).
     * ``[4]`` Path to the output directory (Defaults to ``../Code/Models``).
     * ``[--norm]`` If present, normalize the transition probabilities. (*Note*: Do not use this option for this code, as the transition probabilities are normalized in the C routines anyway).
     * ``[--help]`` displays help about this command

**Example** *(3 environments, 3 actions, 13 states)* :  ``python prepare_synth.py -n 3 -k 2``




Amelie Royer's avatar
Amelie Royer committed
73
#### Running the code (``Code/`` folder)
Amelie Royer's avatar
Amelie Royer committed
74 75
If needed, first set the correct library pathes in ``run.sh``. The script can then be used as follow:

76
``./run.sh -m [1] -d [2] -n [3] -k [4] -g [5] -s [6] -h [7] -e [8] -x [9] -b [10] -c -p --help``
Amelie Royer's avatar
Amelie Royer committed
77 78 79 80 81 82 83 84 85 86 87

   * ``[1]`` Model to use (Defaults to mdp). Available options are *mdp* (MDP model obtained by a weighted average of all the environments' transition probabilities and solved by Value iteration), *ip* (incremental pruning on the MEMDP), *pomcp* and *memcp*.
   * ``[2]`` Dataset to use (Defaults to fm). Available options are *fm* (foodmart) and *rd* (synthetic data).
   * ``[3]`` Product discretization level if foodmart dataset, or the number of items if synthetic data.
   * ``[4]`` History length (Defaults to 2). Must be strictly greater than 1.
   * ``[5]`` Discount Parameter gamma (Defaults to 0.95). Must be strictly between 0 and 1.
   * ``[6]`` Number of iterations for mdp, and number of simulation steps for pomcp and memcp (Defaults to 1500).
   * ``[7]`` Horizon parameter for the POMDP solvers. Defaults to 1.
   * ``[8]`` Convergence criterion for mdp and ip. Defaults to 0.01.
   * ``[9]`` Exploration parameter for pomcp and memcp. Defaults to 10000 (high exploration).
   * ``[10]`` Number of particles for the belief approximation in pomcp and memcp. Defaults to  100.
Amelie Royer's avatar
Amelie Royer committed
88
   * ``[-p]`` If present, use Kahan summation for more precision while handling small probabilities. Use this option if AIToolbox throws an ``Input transition table does not contain valid probabilities`` error when creating the SparseModel object. (*Note*: this requires to use the ``SparseModel.hpp`` file given in ``Code/AiToolBox`` instead of the default AIToolbox one, as described in the first section).
Amelie Royer's avatar
Amelie Royer committed
89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108
   * ``[-c]`` If present, recompile the code before running (*Note*: this should be used whenever using a dataset with different parameters as the number of items, environments etc are determined at compilation time).

**Example** *(foodmart, 6 environments, 3 actions, 13 states)* :
```bash
cd Data/
python prepare_foodmart.py -d Foodmart.tar.gz -pl 4 -k 2
cd ../Code/
./run.sh -m mdp -d fm -n 4 -k 2 -c
./run.sh -m memcp -d fm -n 4 -k 2 -c
```


**Example** *(synthetic, 3 environments, 3 actions, 13 states)* :
```bash
cd Data/
python prepare_synth.py -n 3 -k 2
cd ../Code/
./run.sh -m mdp -d rd -n 3 -k 2 -c
./run.sh -m memcp -d rd -n 3 -k 2 -c
```