finetuning.tex 2.14 KB
 Amelie Royer committed Oct 02, 2018 1 2 3 \vspace{0.45cm} \begin{minipage}[c]{0.46\textwidth} \textbf{Problem Formulation:} Amelie Royer committed Oct 02, 2018 4 \begin{itemize} Amelie Royer committed Oct 02, 2018 5 6 7 8 \item Pretrained model $M$ for task $T: X \rightarrow Y$ \item How to most efficiently fine-tune $M$ for new task $T': X' \rightarrow Y$ where $\mathcal{P}(X) \neq \mathcal{P}(X')$ \item \textbf{Baseline:} Fine-tuning \begin{itemize} Amelie Royer committed Oct 02, 2018 9 10 \item slow/intensive for large models \item might overfit (small $X'$, many parameters) Amelie Royer committed Oct 02, 2018 11 12 \end{itemize} \end{itemize} Amelie Royer committed Oct 02, 2018 13 \end{minipage} Amelie Royer committed Oct 02, 2018 14 15 % \begin{minipage}[c]{0.49\textwidth} Amelie Royer committed Oct 02, 2018 16 17 18 \begin{center} \newcommand{\pacsheight}{2.5cm} \textbf{Example (PACS~\blue{[1]} dataset and variants)} Amelie Royer committed Oct 02, 2018 19 20 \vspace{0.25cm} Amelie Royer committed Oct 02, 2018 21 \begin{minipage}[c]{0.16\textwidth} Amelie Royer committed Oct 02, 2018 22 23 \centering \includegraphics[height=\pacsheight]{pacs_1.png} Amelie Royer committed Oct 02, 2018 24 \centering \scriptsize \blue{Photo (base)} Amelie Royer committed Oct 02, 2018 25 \end{minipage}~\hspace{5pt}\vrule\hspace{5pt}~ Amelie Royer committed Oct 02, 2018 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 \begin{minipage}[c]{0.14\textwidth} \centering \includegraphics[height=\pacsheight]{pacs_2.png} \centering \scriptsize Noisy \end{minipage}~ \begin{minipage}[c]{0.14\textwidth} \centering \includegraphics[height=\pacsheight]{pacs_3.png} \centering \scriptsize Blurry \end{minipage}~ \begin{minipage}[c]{0.14\textwidth} \centering \includegraphics[height=\pacsheight]{pacs_4.png} \centering \scriptsize Painting \end{minipage}~ \begin{minipage}[c]{0.14\textwidth} \centering \includegraphics[height=\pacsheight]{pacs_5.png} \centering \scriptsize Cartoon \end{minipage}~ \begin{minipage}[c]{0.14\textwidth} \centering \includegraphics[height=\pacsheight]{pacs_6.png} \centering \scriptsize Sketch \end{minipage} \end{center} Amelie Royer committed Oct 02, 2018 52 \end{minipage} Amelie Royer committed Oct 02, 2018 53 Amelie Royer committed Oct 02, 2018 54 55 56 57 58 59 60 61 62 \begin{minipage}[c]{0.45\textwidth} \textbf{Proposed}: Flexible Fine-tuning of internal layers \begin{itemize} \item Layer Selection Criterion \item Experiments in various settings \item Projection module (parametrized case) \end{itemize} \end{minipage} \begin{minipage}[c]{0.5\textwidth} Amelie Royer committed Oct 02, 2018 63 \raisebox{-0.5\height}{\includegraphics[width=0.58\textwidth]{fine-tuning/seelction_criterion.pdf}} Amelie Royer committed Oct 02, 2018 64 `~~ Amelie Royer committed Oct 02, 2018 65 \raisebox{-0.5\height}{\includegraphics[width=0.33\textwidth]{fine-tuning/selection_criterion_pacs.png}} Amelie Royer committed Oct 02, 2018 66 \end{minipage}