Restyle the thesis

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Masters/Doctoral Thesis 

% Class File

% Version 1.4 (25/3/16)

%

% This class has been downloaded from:

% http://www.LaTeXTemplates.com

%

% Authors:

% Vel (vel@latextemplates.com)

% Johannes Böttcher

%

% Notes:

% 1) This class file defines the structure and layout of the template file (main.tex).

% 2) It has been written in such a way that under most circumstances you should not need

% to edit it. Updates would be harder. If you do make changes, please change the name of

% the file and add a comment to make your changes more visible.

%

% Class license:

% LPPL v1.3c (http://www.latex-project.org/lppl)

%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------

%	CLASS DEFINITION AND PARAMETERS

%----------------------------------------------------------------------------------------

\NeedsTeXFormat{LaTeX2e}[1996/12/01]

\newcommand{\classname}{MastersDoctoralThesis}

\ProvidesClass{\classname}[2016/03/25 v1.4 LaTeXTemplates.com]

\providecommand{\baseclass}{book}

\RequirePackage{etoolbox}

\RequirePackage{xparse}

\newbool{nolistspace}

\newbool{chapteroneline}

\newbool{listtoc}

\newbool{toctoc}

\newbool{parskip}

\newbool{hyperrefsupport}

\booltrue{hyperrefsupport}

\newbool{headsepline}

\DeclareOption{nohyperref}{\boolfalse{hyperrefsupport}}

\DeclareOption{nolistspacing}{\booltrue{nolistspace}}

\DeclareOption{liststotoc}{\booltrue{listtoc}}

\DeclareOption{chapterinoneline}{\booltrue{chapteroneline}}

\DeclareOption{toctotoc}{\booltrue{toctoc}}

\DeclareOption{parskip}{\booltrue{parskip}}

\DeclareOption{headsepline}{\booltrue{headsepline}}

\DeclareOption*{\PassOptionsToClass{\CurrentOption}{\baseclass}}

\ProcessOptions\relax

\LoadClass{\baseclass}

%----------------------------------------------------------------------------------------

%	CLASS OPTIONS

%----------------------------------------------------------------------------------------

\ifbool{parskip}{\RequirePackage{parskip}} % If the parskip option is passed to the class, require the parskip package 

\newcommand{\abovechapterspace}{\vspace*{50pt}}

\patchcmd{\@makechapterhead}{\vspace*{50\p@}}{\abovechapterspace}{}{}

\patchcmd{\@makeschapterhead}{\vspace*{50\p@}}{\abovechapterspace}{}{}

\newcommand{\autodot}{}

\newcommand{\mdtChapapp}{}

\ifbool{chapteroneline}{%

	\patchcmd{\@makechapterhead}{%

		\if@mainmatter

		\huge\bfseries \@chapapp\space \thechapter

		\par\nobreak

		\vskip 20\p@

		\fi

	}{%

		\if@mainmatter

		\Huge\bfseries \mdtChapapp\thechapter\autodot\enspace

		\fi

	}{}{}

}

\ifbool{listtoc}{% If the liststotoc option has been passed to the class, add the lists to the table of contents

	\patchcmd{\listoftables}{\@starttoc{lot}}{%

		\addchaptertocentry{\listtablename}\@starttoc{lot}}{}{}%

	\patchcmd{\listoffigures}{\@starttoc{lof}}{%

		\addchaptertocentry{\listfigurename}\@starttoc{lof}}{}{}%

}

\ifbool{toctoc}{% If the toctotoc options has been passed to the class, add the table of contents to the table of contents

	\patchcmd{\tableofcontents}{\@starttoc{toc}}{%

		\addchaptertocentry{\contentsname}\@starttoc{toc}}{}{}%

}

\patchcmd{\tableofcontents}{\MakeUppercase}{\MakeMarkcase}{}{}

\patchcmd{\tableofcontents}{\MakeUppercase}{\MakeMarkcase}{}{}

\patchcmd{\listoffigures}{\MakeUppercase}{\MakeMarkcase}{}{}

\patchcmd{\listoffigures}{\MakeUppercase}{\MakeMarkcase}{}{}

\patchcmd{\listoftables}{\MakeUppercase}{\MakeMarkcase}{}{}

\patchcmd{\listoftables}{\MakeUppercase}{\MakeMarkcase}{}{}

% If the option `nolistspacing' is given, the spacing in the different lists is reduced to single spacing. This option is only useful, if the spacing of the document has been changed to onehalfspacing or doublespacing.

\ifbool{nolistspace}{

	\patchcmd{\listoffigures}{%

		\@starttoc{lof}}{%

			\begingroup%

			\singlespace\@starttoc{lof}\endgroup%

		}{}{}%

	\patchcmd{\listoftables}{%

		\@starttoc{lot}}{%

			\begingroup%

			\singlespace\@starttoc{lot}\endgroup%

		}{}{}%

	\patchcmd{\tableofcontents}{%

		\@starttoc{toc}}{%

			\begingroup%

			\singlespace\@starttoc{toc}\endgroup%

		}{}{}%

}{}

% Addchap provides unnumbered chapters with an entry in the table of contents as well as an updated header

\ProvideDocumentCommand{\addchap}{ s o m }{%

	\chapter*{#3}%

	\markboth{}{}%

	\IfBooleanTF{#1}{%

	}{%

		\IfNoValueTF{#2}{%

			\addchaptertocentry{#3}%

			\markboth{\MakeMarkcase{#3}}{\MakeMarkcase{#3}}%

		}{%

			\addchaptertocentry{#2}%

			\markboth{\MakeMarkcase{#2}}{\MakeMarkcase{#2}}%

		}%

	}%

}%

\ProvideDocumentCommand{\addsec}{ s o m }{%

	\section*{#3}%

	\markright{}%

	\IfBooleanTF{#1}{%

	}{%

		\IfNoValueTF{#2}{%

			\addcontentsline{toc}{section}{#3}%

			\markright{\MakeMarkcase{#3}}%%

		}{%

			\addcontentsline{toc}{section}{#2}%

			\markright{\MakeMarkcase{#2}}%

		}%

	}%

}%

%----------------------------------------------------------------------------------------

%	REQUIRED PACKAGES

%----------------------------------------------------------------------------------------

\RequirePackage{babel} % Required for automatically changing names of document elements to languages besides english

\RequirePackage{scrbase} % Required for handling language-dependent names of sections/document elements

\RequirePackage{scrhack} % Loads fixes for various packages

\RequirePackage{setspace} % Required for changing line spacing

\RequirePackage{longtable} % Required for tables that span multiple pages (used in the symbols, abbreviations and physical constants pages)

\RequirePackage{siunitx} % Required for \SI commands

\RequirePackage[pdftex]{graphicx} % Required to include images

\graphicspath{{Figures/}{./}} % Specifies where to look for included images

\RequirePackage{booktabs} % Required for better table rules

\RequirePackage{caption} % Required for customising the captions

\captionsetup{justification=centerlast,font=small,labelfont=sc,margin=5pt}

%----------------------------------------------------------------------------------------

\newcommand{\checktoopen}{% New command to move content to the next page which prints to the next odd page if twosided mode is active  

\if@openright\cleardoublepage\else\clearpage\fi

\ifdef{\phantomsection}{\phantomsection}{}% The \phantomsection command is necessary for hyperref to jump to the correct page

}

\newcommand\bhrule{\typeout{--------------------}}

\newcommand\tttypeout[1]{\bhrule\typeout{\space #1}\bhrule}

\newcommand{\HRule}{\rule{\linewidth}{0.5mm}} % New command to make the lines in the title page

\newcommand{\decoRule}{\rule{.8\textwidth}{.4pt}} % New command for a rule to be used under figures

\renewcommand{\abovechapterspace}{\vspace*{10pt}} % Reduce the whitespace above a chapter heading

\setcounter{tocdepth}{3} % The depth to which the document sections are printed to the table of contents

\providecommand\addchaptertocentry[1]{%

\addcontentsline{toc}{chapter}{#1}%

}

%----------------------------------------------------------------------------------------

%	MARGINS

%----------------------------------------------------------------------------------------

\RequirePackage{geometry}

\geometry{

	headheight=4ex,

	includehead,

	includefoot

}

\raggedbottom

%----------------------------------------------------------------------------------------

%	PENALTIES

%----------------------------------------------------------------------------------------

\doublehyphendemerits=10000 % No consecutive line hyphens

\brokenpenalty=10000 % No broken words across columns/pages

\widowpenalty=9999 % Almost no widows at bottom of page

\clubpenalty=9999 % Almost no orphans at top of page

\interfootnotelinepenalty=9999 % Almost never break footnotes

%----------------------------------------------------------------------------------------

%	HEADERS AND FOOTERS

%----------------------------------------------------------------------------------------

\RequirePackage[markcase=used]{scrlayer-scrpage}

\providepairofpagestyles{thesisSimple}{%

\clearpairofpagestyles%

\automark[chapter]{chapter}

\ihead{\headmark}% Inner header

\ohead[\pagemark]{\pagemark}% Outer header

}

\ifoot{}% Inner footer

\ofoot{}% Outer footer

\pagestyle{thesisSimple}

\providepairofpagestyles[thesisSimple]{thesis}{%

\automark*[section]{}%

}

\providepairofpagestyles[thesisSimple]{review}{%

	\ofoot[\ttitle/\authorname]{\ttitle/\authorname}

	\ifoot[\today]{\today}

}

\pagestyle{thesis}

\ifbool{headsepline}{\KOMAoption{headsepline}{true}}{}

\PreventPackageFromLoading[\ClassError{\classname}{Package `fancyhdr' is

incompatible\MessageBreak with this class}{The pagesyles are defined 

using package `scrlayer-scrpage', please consult the\MessageBreak 

KOMA-script documentation for details.}]{fancyhdr}

\newcommand{\blank@p@gestyle}{empty}

\newcommand{\blankpagestyle}[1]{\def\blank@p@gestyle{#1}}

\def\cleardoublepage{\clearpage\if@twoside \ifodd\c@page\else

\hbox{}

\thispagestyle{\blank@p@gestyle}

\newpage

\if@twocolumn\hbox{}\newpage\fi\fi\fi}

%----------------------------------------------------------------------------------------

%	DEFINE CUSTOM THESIS INFORMATION COMMANDS

%----------------------------------------------------------------------------------------

\def\authorname{}

\def\ttitle{}

\newcommand*{\supervisor}[1]{\def\supname{#1}}

\newcommand*{\thesistitle}[1]{\def\@title{#1}\def\ttitle{#1}}

\newcommand*{\examiner}[1]{\def\examname{#1}}

\newcommand*{\degreetitle}[1]{\def\degreename{#1}}

\renewcommand*{\author}[1]{\def\authorname{#1}}

\newcommand*{\addresses}[1]{\def\addressname{#1}}

\newcommand*{\university}[1]{\def\univname{#1}}

\newcommand*{\department}[1]{\def\deptname{#1}}

\newcommand*{\group}[1]{\def\groupname{#1}}

\newcommand*{\faculty}[1]{\def\facname{#1}}

\newcommand*{\universityitmo}[1]{\def\univnameitmo{#1}}

\newcommand*{\departmentitmo}[1]{\def\deptnameitmo{#1}}

\newcommand*{\groupitmo}[1]{\def\groupnameitmo{#1}}

\newcommand*{\facultyitmo}[1]{\def\facnameitmo{#1}}

\newcommand*{\universityvu}[1]{\def\univnamevu{#1}}

\newcommand*{\departmentvu}[1]{\def\deptnamevu{#1}}

\newcommand*{\groupvu}[1]{\def\groupnamevu{#1}}

\newcommand*{\facultyvu}[1]{\def\facnamevu{#1}}

\newcommand*{\subject}[1]{\def\subjectname{#1}}

\newcommand*{\keywords}[1]{\def\keywordnames{#1}}

%----------------------------------------------------------------------------------------

%	DECLARATION PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\authorshipname}{Declaration of Authorship}

\providecaptionname{american,australian,british,canadian,english,newzealand,UKenglish,USenglish}{\authorshipname}{Declaration of Authorship} % Declaration of Authorship text for English countries

\providecaptionname{german,ngerman,austrian,naustrian}{\authorshipname}{Eidesstattliche Erkl\"arung} % Declaration of Authorship text for Germanic countries

\newenvironment{declaration}{

	\checktoopen

	\tttypeout{\authorshipname}

	\thispagestyle{plain}

	\null\vfil

	{\noindent\huge\bfseries\authorshipname\par\vspace{10pt}}

}{}

%----------------------------------------------------------------------------------------

%	DEDICATION PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand\dedicatory[1]{

	\checktoopen

	\tttypeout{Dedicatory}

	\null\vfil

	\thispagestyle{plain}

	\begin{center}{\Large\slshape #1}\end{center}

	\vfil\null

}

%----------------------------------------------------------------------------------------

%	ABSTRACT PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\byname}{by}

\providecaptionname{german,ngerman,austrian,naustrian}{\byname}{von}

\providecaptionname{american,australian,british,canadian,english,newzealand,UKenglish,USenglish}{\byname}{by}

\newenvironment{porcodio}{

% 	\checktoopen

% 	\tttypeout{\abstractname}

	\null\vfil

% 	\thispagestyle{plain}

% 	\begin{center}

% 	{\normalsize \MakeUppercase{\univname} \par} % University name in capitals

% 	\bigskip

% 	{\huge\textit{\abstractname} \par}

% 	\bigskip

% 	{\normalsize \facname \par} % Faculty name

% 	{\normalsize \deptname \par} % Department name

% 	\bigskip

% 	{\normalsize \degreename\par} % Degree name

% 	\bigskip

% 	{\normalsize\bfseries \@title \par} % Thesis title

% 	\medskip

% 	{\normalsize \byname{} \authorname \par} % Author name

% 	\bigskip

% 	\end{center}

}

{

  \vfil\vfil\vfil\null

}

%----------------------------------------------------------------------------------------

%	ABBREVIATIONS PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\abbrevname}{List of Abbreviations}

\providecaptionname{english,british,american}{\abbrevname}{List of Abbreviations}

\providecaptionname{ngerman,german,austrian,naustrian}{\abbrevname}{Abk\"urzungsverzeichnis}

\NewDocumentEnvironment{abbreviations}{ m }{%

	\ifbool{nolistspace}{\begingroup\singlespacing}{}

	\ifbool{listtoc}{\addchap{\abbrevname}}{\addchap*{\abbrevname}}

	\begin{longtable}{#1}

}{%

\end{longtable}

\addtocounter{table}{-1} % Don't count this table as one of the document tables

\ifbool{nolistspace}{\endgroup}{}

}

%----------------------------------------------------------------------------------------

%	PHYSICAL CONSTANTS PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\constantsname}{Physical Constants}

\providecaptionname{english,british,american}{\constantsname}{Physical Constants}

\providecaptionname{ngerman,german,austrian,naustrian}{\constantsname}{Physikalische Konstanten}

\NewDocumentEnvironment{constants}{ m }{%

	\ifbool{nolistspace}{\begingroup\singlespacing}{}

	\ifbool{listtoc}{\addchap{\constantsname}}{\addchap*{\constantsname}}

	\begin{longtable}{#1}

}{%

\end{longtable}

\addtocounter{table}{-1} % Don't count this table as one of the document tables

\ifbool{nolistspace}{\endgroup}{}

}

%----------------------------------------------------------------------------------------

%	SYMBOLS PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\symbolsname}{List of Symbols}

\providecaptionname{english,british,american}{\symbolsname}{List of Symbols}

\providecaptionname{ngerman,german,austrian,naustrian}{\symbolsname}{Symbolverzeichnis}

\NewDocumentEnvironment{symbols}{ m }{%

	\ifbool{nolistspace}{\begingroup\singlespacing}{}

	\ifbool{listtoc}{\addchap{\symbolsname}}{\addchap*{\symbolsname}}

	\begin{longtable}{#1}

}{%

\end{longtable}

\addtocounter{table}{-1} % Don't count this table as one of the document tables

\ifbool{nolistspace}{\endgroup}{}

}

%----------------------------------------------------------------------------------------

%	ACKNOWLEDGEMENTS PAGE DESIGN

%----------------------------------------------------------------------------------------

\newcommand{\acknowledgementname}{Acknowledgements}

\providecaptionname{american,australian,british,canadian,english,newzealand,UKenglish,USenglish} {\acknowledgementname}{Acknowledgements} % Acknowledgement text for English countries

\providecaptionname{german,ngerman,austrian,naustrian}{\acknowledgementname}{Danksagung} % Acknowledgement text for Germanic countries

\NewDocumentEnvironment{acknowledgements}{}{%

\checktoopen

\tttypeout{\acknowledgementname}

\thispagestyle{plain}

\begin{center}{\huge\textit{\acknowledgementname}\par}\end{center}

}

{

\vfil\vfil\null

}

%----------------------------------------------------------------------------------------

\ifbool{hyperrefsupport}{% If the nohyperref class option has not been specified

\AfterPackage{biblatex}{ % Load the hyperref package after the biblatex package for compatibility 

\RequirePackage{hyperref} % Required for customising links and the PDF

\hypersetup{pdfpagemode={UseOutlines},

bookmarksopen=true,

bookmarksopenlevel=0,

hypertexnames=false,

colorlinks=false, % Set to false to disable coloring links

citecolor=magenta, % The color of citations

linkcolor=red, % The color of references to document elements (sections, figures, etc)

urlcolor=magenta, % The color of hyperlinks (URLs)

pdfstartview={FitV},

unicode,

breaklinks=true,

}

\pdfstringdefDisableCommands{% If there is an explicit linebreak in a section heading (or anything printed to the pdf-bookmarks), it is replaced by a space

   \let\\\space%

}

}

}{%nothing

}

%----------------------------------------------------------------------------------------

\endinput

% lazyLizardTracer

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Conclusion}

% \begin{itemize}

% \item SUPG is working but not as fast as expected. Touch sensors maybe be needed

% to achieve good performances. Also learning is slower than RLPower (open vs closed

% loop).

% \todo[inline]{\textit{From the introduction}\newline

% Results show that the proposed controller architecture achieves a good gait, but the results are not as good as compared to other open-loop architectures.

% Results also show that the architecture is sometimes capable of reaching some kind of directed locomotion, but not targeted locomotion.\newline

% More fine tuning of the experiment and more training time would be needed to improve results.

% But we are generally not impressed on the results and we do not think this solution should be investigated any further.

% }

% \end{itemize}

Our research goals were to 

\begin{itemize}

  \item \textbf{\textit{Train the proposed controller to achieve reasonable gaits}}. I.e. make the robot move.

  \item \textbf{\textit{Train the proposed controller to achieve targeted locomotion, in the form of phototaxis}}. I.e. make the robot move towards the light.

\end{itemize}

To determine if we achieved our goals, we ask ourselves three questions:

\begin{itemize}

 \item[$\diamond$] is the robot capable of moving?

 \item[$\diamond$] is the robot capable of learning to move at least in one direction?

 \item[$\diamond$] are the multiple targets at the same time interfering with the learning?

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Is the robot capable of moving?}

Our first goal was to make the robot develop some \emph{movement} in any direction.

In \ref{ch:Results:gait}, we determined that the robot is capable of moving and generating some gaits; therefore we consider our first goal fully achieved.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Is the robot capable of learning to move at least in one direction?}

Our second goal was to achieve \emph{targeted locomotion}.

We learned that the robot is capable of learning to move in one direction. This is called {\bf Directed Locomotion}.

All robots, when asked to learn just one direction, achieved some result.

Some directions are more difficult than others to learn depending on the shape of the robot.

There are better alternative open-loop controllers more efficient in accomplishing the same task.

This is not what we originally intended.

The robot is not capable of learning to move to a target. This is called {\bf Targeted Locomotion}.

We can demonstrate this by watching the replay and seeing that in the long term the robots are moving in the opposite direction for some targets.

The task is too complicated for our controller to learn all at once

% A mesh of multiple controllers that are able of simplier tasks is the step we would take next.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Considerations about the approach}

We did not achieve our research goal and we have to have reflected on what went wrong and what could have been done to improve the results.

We conclude that a black-box approach for such a complex task was a leap too big for our algorithm to make.

Evidence that the complexity of the problem was our main obstacle can be found in the results: every time there is a decrease in complexity, there is a corresponding boost in the performance of the results.

E.g. when we introduced the kickstarting hack, only the simple shapes (fewer control joints) could take benefit;

when we separated the light targets as different tasks, we found a decisive increase in all the results, with some more than doubled.

The proposed \supg algorithm \cite{morse2013single} did not fulfil its promises to be a fast and stable algorithm.

The possible cause is that \supg excelled in a very specific configuration, where the environment was simulated and the robot was a quadruped animat, with perfect touch sensors hard-wired into the algorithm.

The original paper presents a much simpler problem, with a smaller configuration space: the objective of the original \supg algorithm was to develop a gait for a more symmetrical robot which could go in any direction with no sense of the outside apart from the touch sensors.

In our case, we needed much more flexibility and complexity, as we needed to control different, potentially asymmetrical robots.

The objective to learn is also more difficult, as the robot has to have a deeper understanding of the objective and adequately alternate rotation and directional movement.

The \supg algorithm simply showed its limits.

We think that, with an increase in 50 or 100 fold number of evaluations, some real results could be achieved.

This was not explored, as the scope falls outside the scope of the thesis, where the proposed environment is about learning in the real world using real hardware.

% Instead we suggest looking at other methodologies that can create results in reasonable times.

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% \subsection{Are the multiple targets at the same time interfering with the learning?}

%

% Clearly for some shapes the separation of targets meant a great improvement over the single results.

% This is probably due to some shapes needing completely different gaits for locomotion in different directions and the inability of the controller to learn to separate actions like rotate first then move forward.

%

% We tested that all directions were learned better when learned separately.

% This shows us as the simplification of the problem brought us to a better result.

% We conclude that the problem is too complex to be learned all at once by our proposed solution.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Future works}

This thesis demonstrates that a naive approach is not enough and therefore we discourage to keep working on a one-do-all, black-box controller solution.

With these results in mind, our suggestion is to take a more modular approach: simpler objectives to learn for simple controller modules that can be later combined into a more complex controller.

Some inspiration could be drawn from \cite{benbrahim1997biped} in which Benbrahim and Franklin design a modular controller (each module defines a behaviour) with a central system that continuously switches between behaviours depending on the need.

Another (non-exclusive) approach that could be taken is to offload the learning into a simulated world.

Trough the mixed use of an arena in the real world, accelerated simulations and GPU computing, we think is possible to achieve interesting results.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% \chapter{Future improvements}

% \begin{itemize}

% \item establish HyperNEAT coordinates programmatically from the body

% \item being able to load a brain and visualise the tests

% \item make the brain transferable to real hardware

% (make light sensors realistic in simulation)

% \item adjust evolution parameters to improve over speed and quality of the evolution

% \item evolve the position of the sensors in the body (bad position could lead to a bad evaluation

% \item use bits from TD-learning or Q-learning to improve learning

% \end{itemize}