README.md 6.37 KB
Newer Older
1
2
# CARE-*less* care|n2v
Simple IPython based user interface to [CARE](http://csbdeep.bioimagecomputing.com/) a toolbox for Content-aware Image Restoration and to [Noise2Void](https://github.com/juglab/n2v) a self-supervised deep learning based denoising.
Christoph Sommer's avatar
doc    
Christoph Sommer committed
3

Christoph Sommer's avatar
Christoph Sommer committed
4
# CARE
Christoph Sommer's avatar
doc    
Christoph Sommer committed
5
6
7
## How to use:
CARE needs pairs of registered images - low (input) and high (output) quality. It trains a convolutional neural network how to transform low quality images - which might even be of less physical resolution - into high quality images. After training, newly recorded low quality images or movies can be predicted. 2D, 3D and multi-channel images are supported. For each channel a separate network is trained.

Christoph Sommer's avatar
Christoph Sommer committed
8
#### Vanilla screencast for input selection and training
9
![CAREless user interface](vid/bif_care_demo_01.mp4)
Christoph Sommer's avatar
Christoph Sommer committed
10

Christoph Sommer's avatar
doc    
Christoph Sommer committed
11
#### Input selection and patch extraction
12
13
0. Clone this repository with `git clone https://git.ist.ac.at/bioimaging/careless`
1. Copy and rename the IPython notebook template file: `careless_care.ipynb` to `my_care_project.ipynb`
Christoph Sommer's avatar
Christoph Sommer committed
14
2. Open your renamed `my_care_project.ipynb` file in Jypyter or IPyhton notebook.
Christoph Sommer's avatar
Christoph Sommer committed
15
3. In order to train CARE, the path to the image pairs needs to be specified. Then, choose images for low and high quality respectively using a wild-card (e.g. `low*.tif` and `high*.tif`). The images will be converted and image patches are extracted. This step is required for efficient GPU execution. Choose patch sizes for your input dimensions `(Z)YX` and set how many patches should be extracted per image pair. After image patches have been extracted, they are saved to the output directory.
Christoph Sommer's avatar
doc    
Christoph Sommer committed
16
17

#### Training the network
Christoph Sommer's avatar
Christoph Sommer committed
18
The training of a neural network is done iteratively in `epochs`. In each epoch, the network weights' are updated by optimizing a loss function on `steps_per_epoch` batches of image patches. The size of the batches is given by `batch_size`. To make use of all your image data, select `steps_per_epoch = #patches / batch_size`. Per default, 10% of patches are used for validation and not used in training.
19
20
21

4. Select training parameters and execute training code block.

22
23
24
25
26
27
28
29
30
31
32
#### Seting up Fiji for prediction with CARE models
Using Tensorflow 1.12.0 / NVidia toolkit 9.0
1. Use the Fiji [CSBDeep](http://sites.imagej.net/CSBDeep/) update site

Using Tensorflow 1.13.1 / NVidia toolkit 10.0
1. get libtensorflow-1.13.1.jar and [tensorflow_jni.dll](https://www.tensorflow.org/install/lang_java)
2. put libtensorflow_jni-1.13.1.jar into "Fiji.app\jar\"
3. put tensorflow_jni.dll into folder "Fiji.app\lib\win64\"

Restart Fiji

33
#### Using the trained network for prediction
Christoph Sommer's avatar
Christoph Sommer committed
34
35
You can predict new images in the IPython notebook directly using the prediction widgets, or use the Fiji:

36
37
38
0. Add (or enable) [CSBDeep](http://sites.imagej.net/CSBDeep/) to your Fiji update sites
1. Open image you want to predict in Fiji
2. In Fiji choose `Plugins->CSBDeep->Run your network`
39
3. Select network file `<careless-out-folder>/models/CH_X_model/TF_SavedModel.zip` as 'Import model (.zip)' of your trained channel
Christoph Sommer's avatar
Christoph Sommer committed
40
4. Set additional parameters such as number of tiles (higher number, in case your entire image cannot be loaded on your GPU memory) and press OK
Christoph Sommer's avatar
doc    
Christoph Sommer committed
41

Christoph Sommer's avatar
Christoph Sommer committed
42
43
44
---

# Noise2Void
Christoph Sommer's avatar
Christoph Sommer committed
45
46
## How to use:
Noise2void does not require pairs of images.
47
1. Copy and rename the IPython notebook template file: `careless_n2v.ipynb` to `my_n2v_project.ipynb`
Christoph Sommer's avatar
Christoph Sommer committed
48
49
2. Open your renamed `my_n2v_project.ipynb` file in Jypyter or IPyhton notebook.
3. Follow steps in the notebook
Christoph Sommer's avatar
typo    
Christoph Sommer committed
50

Christoph Sommer's avatar
n2v doc    
Christoph Sommer committed
51
# Installation (care and n2v)
Christoph Sommer's avatar
doc    
Christoph Sommer committed
52
### NVidia Cuda and cuDNN
53
54
55
1. Install [NVidia CUDA toolkit](https://developer.nvidia.com/cuda-toolkit-archive)
2. Install [NVidia cuDNN](https://developer.nvidia.com/cudnn)

Christoph Sommer's avatar
Christoph Sommer committed
56
57
The installation of the CUDA toolkit is straight-forward. To download cuDNN you have to create a Developer account at NVidia. After downloading integrate the folders `bin`, `include` and `lib` into the according folders of your CUDA toolkit installation.

58
#### Known working versions
59
60
61
62
63
64
65
1. NVidia CUDA toolkit  9.0 + cuDNN 7.3.0 + tensorflow 1.12.0
2. NVidia CUDA toolkit 10.0 + cuDNN 7.5.1 + tensorflow 1.13.1, 1.14.0 (recommended)

**Note:**
* tensorflow versions 2.* will not work.
* Do not use NVidia CUDA toolkit > 10.0

Christoph Sommer's avatar
doc    
Christoph Sommer committed
66
67

### Python dependencies
Christoph Sommer's avatar
Christoph Sommer committed
68
We strongly recommend using the [Anaconda Python distribution (64bit)](https://www.anaconda.com/distribution/) with Python == 3.6. Furthermore, we recommend to create a new conda virtual environment for Python 3.6 with:
Christoph Sommer's avatar
Christoph Sommer committed
69
70

```
71
72
conda create -n py36_careless python=3.6 pyqt nodejs
conda activate py36_careless
Christoph Sommer's avatar
Christoph Sommer committed
73
```
Christoph Sommer's avatar
Christoph Sommer committed
74

75
Install all mayor dependencies, including CARE and Noise2Void by:
Christoph Sommer's avatar
doc    
Christoph Sommer committed
76

77
78
```
cd <this-path>
79
pip install -e .
80
```
Christoph Sommer's avatar
Christoph Sommer committed
81
If you observe problems installing javabridge see troubleshooting below.
Christoph Sommer's avatar
doc    
Christoph Sommer committed
82

83
If your Python distribution comes without PyQt5, install it by:
Christoph Sommer's avatar
doc    
Christoph Sommer committed
84

85
```
Christoph Sommer's avatar
Christoph Sommer committed
86
conda install pyqt
87
```
Christoph Sommer's avatar
doc    
Christoph Sommer committed
88
89

### IPython widgets
Christoph Sommer's avatar
Christoph Sommer committed
90
91
The user interface is written using IPython/Jupyter widgets, which requires the installation of [node.js](https://nodejs.org/en/)).
Install it by using the official installer or by: `conda install -c conda-forge nodejs` from an Anaconda prompt.
92

Christoph Sommer's avatar
Christoph Sommer committed
93
Finally, you have to install jypyter widgets and enable them, by:
Christoph Sommer's avatar
doc    
Christoph Sommer committed
94

95
96
```
jupyter labextension install @jupyter-widgets/jupyterlab-manager
Christoph Sommer's avatar
doc    
Christoph Sommer committed
97
jupyter nbextension enable --py widgetsnbextension
98

Christoph Sommer's avatar
doc    
Christoph Sommer committed
99
100
```
More information on [Jupyter/IPython widgets](https://ipywidgets.readthedocs.io/en/stable/user_install.html).
Christoph Sommer's avatar
doc    
Christoph Sommer committed
101

102
103
104
105
106
107
108
### Example data
The authors of [CARE](https://github.com/CSBDeep/CSBDeep/tree/master/examples) provide example data from different modalities.

* [3D denoising](http://csbdeep.bioimagecomputing.com/example_data/tribolium.zip)

Unzip, copy and rename (e. g. *_low.tif*, *_high.tif*) the images form `low` and `GT` into a single folder.

Christoph Sommer's avatar
Christoph Sommer committed
109
### Comparison of CARE with N2V
110
![CAREless user interface](img/example_result.png "Comparison of CARE with N2V")
Christoph Sommer's avatar
Christoph Sommer committed
111

Christoph Sommer's avatar
doc    
Christoph Sommer committed
112

Christoph Sommer's avatar
doc    
Christoph Sommer committed
113
### Troubleshooting and known issues
114
115
* tensorflow>1.12.x requires NVidia tookit 10.0
* Currently NVidia toolkit > 10.0 is not supported by tensorflow==1.* versons
Christoph Sommer's avatar
Christoph Sommer committed
116
117
118
* To install bioformats/ javabridge you need the Microsoft Visual Studio compiler runtime 2015 (14.0) installed, which is included with Microsoft Visual Studio community edition >=2017
* To install bioformats/ javabridge you need Java SDK 8 (1.8) or download [pre-compiled .whl](https://www.lfd.uci.edu/~gohlke/pythonlibs/) packages and install them by:

Christoph Sommer's avatar
Christoph Sommer committed
119
120
121
    ```pip install <javabridge_cp36_amd64>.whl` and `pip install <bioformatse_cp36_amd64>.whl```

* It may be required to launch Anaconda / Anaconda prompt as administrator
Christoph Sommer's avatar
Christoph Sommer committed
122

Christoph Sommer's avatar
doc    
Christoph Sommer committed
123

Christoph Sommer's avatar
Christoph Sommer committed
124
125


Christoph Sommer's avatar
Christoph Sommer committed
126