Zubr is a strict form programming language. A program in Zubr consists of three sections:
• declaration section
• perkun section
• definition section
The simplest Zubr program is:
// begin of the program
%%
values {}
variables {}
%%
// end of the program
When you place the above text in a file (for example test.zubr) and run:
$ zubr test.zubr > Optimizer.java
it will create an optimizer in Java.
Below is the list of all Zubr keywords:
• actions
• beliefs
• cout
• eol
• generator
• haskell
• hidden
• illegal
• impossible
• input
• loop
• model
• output
• payoff
• prolog
• set
• state
• states
• value
• values
• variable
• variables
• visible
• xml
A valid identifier is any sequence of letters/digits beginning with a letter that is not a keyword.
// This is a Zubr example code
%%
values
{
value FALSE, TRUE, DO_NOTHING, MOVE;
}
variables
{
input variable alpha:{FALSE, TRUE};
output variable beta:{DO_NOTHING, MOVE};
}
%%
// end of example
The values section of the Perkun code within Zubr specification contains the keyword "value" followed by the valid identifiers,
separated by commas.
The values are used to denote the possible values of the variables.
The Perkun variables is a complex topic and will be discussed later.
One-line comments begin in Zubr with a hashtag "#" or with "//".
Zubr allows three kinds of variables:
• input variables
• hidden variables
• output variables
The hidden variables are essential for the machine learning.
They represent the state variables of the world that are not directly visible to the agent.
Yet they do affect the state of the world.
Let us forget for now the hidden variables though, and introduce a simple Zubr code with one input variable and one output variable:
%%
values
{
value FALSE, TRUE;
value MOVE, DO_NOTHING;
}
variables
{
input variable what_I_can_see:{FALSE, TRUE};
output variable action:{MOVE, DO_NOTHING};
}
%%
As you can see the variables section contains now two variables:
• what_I_can_see
• action
Each variable declaration is followed by a colon and a list of possible values in curly brackets, separated by commas. For example what_I_can_see may have value FALSE or TRUE.
We know already how to declare Zubr values and variables. We know that the agent i.e. Zubr optimizer
can see the input variables and can control the output variables. The question is how to control the output
variables. What do we want to achieve? What is "good" for the agent and what is not?
In order to define it we use the getPayoff method in the Zubr program.
Try the following code:
%option getpayoff own // method getPayoff
%%
values
{
value FALSE, TRUE;
value MOVE, DO_NOTHING;
}
variables
{
input variable what_I_can_see:{FALSE, TRUE};
output variable action:{MOVE, DO_NOTHING};
}
%%
protected float getPayoff(VisibleState vs) {
switch (vs.getVariableValue("what_I_can_see"))
{
case FALSE:
return 0.0f;
case TRUE:
return 100.0f; // TRUE is "better" than FALSE
}
return 0.0f;
}
In the declaration section we used the Zubr option:
%option getpayoff own
It will create an optimizer that "prefers" TRUE rather than FALSE as input.
What does the payoff value mean? It is a value representing "how good" the payoff function is
(for the respective combination of the input variable values). The higher the payoff value the "better" it is.
In the example above the agent "likes" to see the value "TRUE" on its input (variable what_I_can_see),
while it dislikes the value "FALSE", because for "TRUE" the payoff value equals 100.0, while for "FALSE"
it equals 0.0.
Zubr hidden variables
We do not perceive directly everything that is happening in the world. Or to say it otherwise - in order to understand the world it is useful to imagine that some facts about it remain hidden. Likewise - in Zubr it is assumed that the world has a state described by the visible variables (so called input variables) and by the invisible variables (so called hidden variables). How to declare them in Zubr?
%%
values { value FALSE, TRUE; }
variables
{
input variable a:{FALSE, TRUE}, b:{FALSE, TRUE};
hidden variable c:{FALSE, TRUE}, d:{FALSE, TRUE};
}
%%
The above Zubr code defines two input variables (a and b) and two hidden variables (c and d). All of them may have either value FALSE or TRUE.
But how can we use the hidden variables? What is it good for, to take into account the variables we do not perceive? Well, in order to understand this we need to learn how to use the getModelProbability method.
Zubr model
We know already how to declare the variables (input,output or hidden ones) and how to define the payoff, i.e. tell Zubr which input variable values are good and which are not so good. In order to do its job Zubr needs to know how its decisions will affect the world, i.e. how the world state in the future depends on its state now and the optimizer's choices.
Imagine a world where we have only one variable - input variable alpha. Its value can be either FALSE or TRUE. Let us introduce an output variable beta which can control two actions:
• DO_NOTHING - alpha remains as it is now
• MOVE - alpha switches to not alpha
Let us further assume that the optimizer "likes" alpha=TRUE, and "dislikes" alpha=FALSE. For this simple world our Zubr program will look as follows:
%option getmodelprobability own // method getModelProbability
%option getpayoff own // method getPayoff
%%
values
{
value FALSE, TRUE, DO_NOTHING, MOVE;
}
variables
{
input variable alpha:{FALSE, TRUE};
output variable beta:{DO_NOTHING, MOVE};
}
%%
protected float getModelProbability(VisibleState vs1, State s1, Action a, VisibleState vs2, State s2) {
switch (a.getVariableValue("beta")) {
case DO_NOTHING:
if (vs1.getVariableValue("alpha") ==
vs2.getVariableValue("alpha"))
return 1.0f;
else
return 0.0f;
break;
case MOVE:
if (vs1.getVariableValue("alpha")
!= vs2.getVariableValue("alpha"))
return 1.0f;
else
return 0.0f;
break;
}
return 0.0f;
}
protected float getPayoff(VisibleState vs) {
switch (vs.getVariableValue("alpha"))
{
case FALSE:
return 0.0f;
case TRUE:
return 100.0f; // TRUE is "better" than FALSE
}
return 0.0f;
}
// end of example
In the method getModelProbability you can query the parameter vs1 and vs2 (VisibleState) for input variable values, s1 and s2 (State) for hidden variable values and a (Action) for output variable values. The method getVariableValue defined in all of them returns Value (an enum containing all the Zubr values).
Once you process the file with zubr it creates a Java code to be compiled and executed. But for the example above the optimizer created does not communicate with the outer world. In order to provide means for such communication we need to provide methods getInput and execute. Otherwise you can extend the Optimizer class and redefine the default implementation.
%option getinput own // with this option we tell zubr we will provide our own implementation of the method getInput
// We can import Java packages here.
import javax.swing.JFrame;
import javax.swing.JOptionPane;
import java.awt.Dimension;
%%
// here we put the Perkun code (values and variables)
values
{
value FALSE, TRUE;
}
variables
{
input variable alpha:{FALSE, TRUE};
}
%%
// here we put Java code to be included in the result class
private static JFrame frame;
// this is the implementation of the method we promised to provide:
protected void getInput(Map<Variable, Value> m) {
Object[] possibilities = {"FALSE", "TRUE"};
String n = (String)JOptionPane.showInputDialog(frame, "Alpha=?", "Continue?", JOptionPane.PLAIN_MESSAGE,
null, possibilities, "FALSE");
if (n == null) {
loopIsRunning = false; // here we tell the optimizer we want to break the loop
}
else
if (n == "FALSE") {
m.put(mapNameToVariable.get("alpha"), Value.FALSE);
}
else
if (n == "TRUE") {
m.put(mapNameToVariable.get("alpha"), Value.TRUE);
}
}
// the OptimizerThread class creates an instance of the MyOptimizer class (created by zubr)
// and runs the loop.
protected static class OptimizerThread extends Thread {
private Optimizer optimizer;
public void run() {
optimizer = new Optimizer();
optimizer.loop(1);
}
}
public static void main(String[] args) {
frame = new JFrame();
frame.setTitle("Optimizer");
frame.setSize(new Dimension(800, 600));
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setVisible(true);
OptimizerThread t = new OptimizerThread();
t.start();
}
Here we provided a main function which will create a Swing frame, instantiate an OptimizerThread and start it. The method getInput fills a map Map<Variable,Value> with input variable values. To access the variables we can use the Map<String, Variable> named mapNameToVariable.
Just to be sure we will need one more method - a redefinition of the error handling method onErrorInPopulateBeliefForConsequence. As usual we will tell zubr this method will be provided by us using an %option in the declaration section:
%option onerrorinpopulatebeliefforconsequence own
// with this option we tell zubr we will provide our own implementation
// of the method onErrorInPopulateBeliefForConsequence
%%
Then in the definition section we provide the implementation:
// this is the implementation of the method we promised to provide:
protected void onErrorInPopulateBeliefForConsequence(Belief formerBelief, Action formerAction, VisibleState vs) {
JOptionPane.showMessageDialog(frame, "error in populate belief for consequence", "Error", JOptionPane.ERROR_MESSAGE);
System.exit(0);
}
This error means that something unexpected happens. We could handle it in a more smooth manner, but for now this implementation will be sufficient.
Congratulations, now you know everything what you have to tell zubr. Values, variables, payoff, model. The time has come to see what zubr can do for you given this information. It will create a Java class that attempts to maximize the expected value of the payoff function by appropriate choosing the actions. First you give it current values of the input variables, then it responds with the optimal action chosen (the values of the output variables). As you can see we will interact with the optimizer and this interaction will involve both input and output.
The "execute" method is called to execute an action once the optimal action is found. We can as usual inform zubr that we are going to provide its implementation:
%option execute own // method execute
Then in the definition section we can write:
protected void execute(Action a) {
JOptionPane.showMessageDialog(frame, a.getVariableValue("beta").getName(), "Action", JOptionPane.INFORMATION_MESSAGE);
}
The enum Value has a method getName() which can convert it to its string representation.
The loop command
The parameter of the loop command that we call in the OptimizerThread is the depth of the game tree.
%option class
We can modify the target class name (default is Optimizer) by using zubr option class in the declaration section:
%option class MyOptimizer // this modifies the target class name, it is optional
iterators
In order to skip certain combinations we may define our own implementations of the VisibleStateIterator, StateIterator and ActionIterator (they are interfaces created by Zubr). Zubr creates also default implementations of these iterators called DefaultVisibleStateIterator, DefaultStateIterator and DefaultActionIterator.
protected interface VisibleStateIterator {
public boolean getFinished();
public void increment();
public VisibleState createVisibleState();
}
protected interface StateIterator {
public boolean getFinished();
public void increment();
public State createState();
}
protected interface ActionIterator {
public boolean getFinished();
public void increment();
public Action createAction();
}
If we want to use a different iterator we need to redefine one of the below methods:
protected ActionIterator createNewActionIterator(VisibleState s) {
return new DefaultActionIterator();
}
protected VisibleStateIterator createNewVisibleStateIterator(VisibleState s) {
return new DefaultVisibleStateIterator();
}
protected StateIterator createNewStateIterator(VisibleState s) {
return new DefaultStateIterator();
}
If we want to do it directly in the zubr specification we will have to tell zubr about it:
%option createnewactioniterator own
%option createnewvisiblestateiterator own
%option createnewstateiterator own
The most simple way to provide a new iterator is to extend a default iterator, for example redefining its method increment. All of them contain protected Map<Variable, Value> mapVariableToValue, which can be accessed to check if we want to skip the combination of values.