View source

[[Category:Agents]]
[[Category:Java]]
=Introduction=

The Fourier Basis is a generic basis for linear function approximation in continuous-state reinforcement learning domains.
It is simple, easy to use, and seems to work reliably.

In the simplest case, 
specifying a Fourier Basis for an <i>n</i> dimensional continuous domain requires only one parameter: <i>k</i>, which
specifies the order terms to include (all terms of all combinations of variables of 
order between <i>0</i> and <i>k</i> inclusive are included). This parameter is
necessary because the Fourier Basis has infinitely many terms. 

Since the number of basis functions scales exponentially with the number of state
variables <i>n</i> (as is true for all fixed bases), in high-dimensional
problems the basis function coefficient ranges will probably need to be constrained. 
This code includes a FourierCoefficientGenerator class
which can be used to write constrained coefficient generators. Two example derived classes are included: FullFourierCoefficientGenerator (which generates all coefficients) and IndependentFourierCoefficientGenerator (which only generates terms using individual 
variables).  Both classes take <i>k</i> as a parameter.
The Fourier Basis is also well suited as a fixed basis for use in feature selection, since it has infinitely many 
basis functions.  

The Fourier Basis is described in the following paper:
* [http://www-all.cs.umass.edu/~gdk/ G.D. Konidaris] and [http://www.cs.brown.edu/~sosentos/ S. Osentoski]. [http://www-all.cs.umass.edu/~gdk/pubs/fourier-tech.pdf Value Function Approximation in Reinforcement Learning using the Fourier Basis.] Technical Report UM-CS-2008-19, Department of Computer Science, University of Massachusetts Amherst, June 2008. 

=Technical Details=

==Policy==
*e-greedy

==Exploration Policy==
*e-greedy

==State Representation/Function Approximation==
*State variables should be continuous. The function approximation code scales them to between 0 and 1 internally.

==Actions==
*This implementation handles only discrete actions.

=Other Details=
* RL-Viz Compatible
* Value function visualizable
* Language: Java
* License: [http://www.apache.org/licenses/LICENSE-2.0.html Apache 2.0]

<!-- 
The SAR and EAR is actually code that means start auto replace and end auto replace.  This page will be
automatically updated from time to time by a bot, and the bot will replace the text between the SAR and the 
EAR.  Editing it manually will not survive past the next release of that software. -->
{{Template:Java_Environment_Download_Use|SAR=1|Filename=FourierSarsaAgent-Java-R1332.tar.gz|Jarname=fourierBasisAgentLib.jar|Projectdir=FourierSarsaAgent-Java-R1332|Anttarget=build|EAR=1}}


== Getting Help ==
Please send all questions to either the current maintainer (below) or to the 
[http://groups.google.com/group/rl-library RL-Library mailing list].

== Current Maintainer ==
* [http://www-all.cs.umass.edu/~gdk/ George Konidaris]
* gdk@cs.umass.edu
* http://www-all.cs.umass.edu/~gdk/