Perkun is a strict form programming language. A program in Perkun consists of four sections and any number of commands:
• values section
• variables section
• payoff section
• model section
• commands
The simplest Perkun program is:
# begin of the program
values {}
variables {}
payoff {}
model {}
# end of the program
When you place the above text in a file (for example test.perkun) and run:
$ perkun test.perkun
it will do nothing. That is correct. But if you remove any of the sections you will get an error.
Also correct.
Below is the list of all Perkun keywords:
• actions
• beliefs
• cout
• eol
• generator
• haskell
• hidden
• illegal
• impossible
• input
• loop
• model
• output
• payoff
• prolog
• set
• state
• states
• value
• values
• variable
• variables
• visible
• xml
A valid identifier is any sequence of letters/digits beginning with a letter that is not a keyword.
# This is a Perkun example code
values
{
value false, true, do_nothing, switch;
}
variables
{
input variable alpha:{false, true};
output variable beta:{do_nothing, switch};
}
payoff
{
set({alpha=>false},0.0);
set({alpha=>true},100.0);
}
model
{
set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
}
loop(1);
# end of example
The values section of the Perkun code contains the keyword "value" followed by the valid identifiers,
separated by commas.
The values are used to denote the possible values of the variables.
The Perkun variables is a complex topic and will be discussed later.
You can print out all the values with the command
cout << values << eol;
For example:
values
{
value false, true, do_nothing, switch;
}
variables{}
payoff{}
model{}
cout << values << eol;
The above Perkun code will print out:
# values
false
true
do_nothing
switch
There are many other commands apart from "cout << values << eol;". They can all be placed directly after the model section.
One-line comments begin in Perkun with a hashtag "#".
Perkun allows three kinds of variables:
• input variables
• hidden variables
• output variables
The hidden variables are essential for the machine learning.
They represent the state variables of the world that are not directly visible to the agent.
Yet they do affect the state of the world.
Let us forget for now the hidden variables though, and introduce a simple Perkun code with one input variable and one output variable:
values
{
value false, true;
value move, do_nothing;
}
variables
{
input variable what_I_can_see:{false, true};
output variable action:{move, do_nothing};
}
payoff {}
model {}
As you can see the variables section contains now two variables:
• what_I_can_see
• action
Unlike many other programming languages Perkun does not have the concept of a variable type. Instead each variable declaration is followed by a colon and a list of possible values in curly brackets, separated by commas. For example what_I_can_see may have value false or true.
Try using the command "cout << variables << eol;":
values
{
value false, true;
value move, do_nothing;
}
variables
{
input variable what_I_can_see:{false, true};
output variable action:{move, do_nothing};
}
payoff {}
model {}
cout << variables << eol;
When you put the above text in a file and call Perkun it will respond:
# variables
input variable what_I_can_see:{false,true};
output variable action:{move,do_nothing};
We know already how to declare Perkun values and variables. We know that the agent i.e. Perkun program
can see the input variables and can control the output variables. The question is how to control the output
variables. What do we want to achieve? What is "good" for the agent and what is not?
In order to define it we use the payoff section of the Perkun program.
Try the following code:
values
{
value false, true;
value move, do_nothing;
}
variables
{
input variable what_I_can_see:{false, true};
output variable action:{move, do_nothing};
}
payoff
{
set({what_I_can_see=>false},0.0);
set({what_I_can_see=>true},1.0);
}
model {}
cout << payoff << eol;
It will print out the contents of the payoff section:
# payoff
what_I_can_see=false payoff 0
what_I_can_see=true payoff 1
The payoff section contains the commands "set" with two arguments:
• input state query
• payoff value (a real number)
The query contains all input variables followed by "=>" and the corresponding value, separated by commas,
enclosed in curly brackets. In short it is an expression:
{INPUT_VARIABLE1=>VALUE,INPUT_VARIABLE2=>VALUE,...}
Note that the query must contain all the input variables.
What does the payoff value mean? It is a value representing "how good" the payoff function is
(for the respective combination of the input variable values). The higher the payoff value the "better" it is.
In the example above the agent "likes" to see the value "true" on its input (variable what_I_can_see),
while it dislikes the value "false", because for "true" the payoff value equals 1.0, while for "false"
it equals 0.0.
What happens if you fail to specify payoff for some or all input variables values combinations? It will be random. Try:
values {}
variables {}
payoff {}
model {}
cout << payoff << eol;
Anyway it does not make much sense since in this case the payoff function is constant (there are no input variables).
Try:
values
{
value false, true;
value move, do_nothing;
}
variables
{
input variable what_I_can_see:{false, true};
output variable action:{move, do_nothing};
}
payoff {}
model {}
cout << payoff << eol;
After running the above code with perkun you will see random values for both "false" and "true" of the input variable what_I_can_see.
Perkun hidden variables
We do not perceive directly everything that is happening in the world. Or to say it otherwise - in order to understand the world it is useful to imagine that some facts about it remain hidden. Likewise - in Perkun it is assumed that the world has a state described by the visible variables (so called input variables) and by the invisible variables (so called hidden variables). How to declare them in Perkun?
values { value false, true; }
variables
{
input variable a:{false, true}, b:{false, true};
hidden variable c:{false, true}, d:{false, true};
}
payoff {}
model {}
cout << variables << eol;
The above Perkun code defines two input variables (a and b) and two hidden variables (c and d). All of them may have either value false or true. When you run it with perkun it will write out:
# variables
input variable a:{false,true};
input variable b:{false,true};
hidden variable c:{false,true};
hidden variable d:{false,true};
But how can we use the hidden variables? What is it good for, to take into account the variables we do not perceive? Well, in order to understand this we need to learn how to use the model section of the Perkun program.
Perkun model
The time has come to learn the last part of the Perkun program: the model. We know already how to declare the variables (input,output or hidden ones) and how to define the payoff, i.e. tell Perkun which input variable values are good and which are not so good. In order to do its job Perkun needs to know how its decisions will affect the world, i.e. how the world state in the future depends on its state now and the Perkun's choices.
Imagine a world where we have only one variable - input variable alpha. Its value can be either false or true. Let us introduce an output variable beta which can control two actions:
• do_nothing - alpha remains as it is now
• switch - alpha switches to not alpha
Let us further assume that Perkun "likes" alpha=true, and "dislikes" alpha=false. For this simple world our Perkun program will look as follows:
values
{
value false, true, do_nothing, switch;
}
variables
{
input variable alpha:{false, true};
output variable beta:{do_nothing, switch};
}
payoff
{
set({alpha=>false},0.0);
set({alpha=>true},100.0);
}
model
{
set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
}
cout << model << eol;
The model section is filled with the set instructions, but they have slightly different syntax than the set
instructions we know from the payoff section. Each set instruction in the model section has syntax:
set(INITIAL_STATE_QUERY,ACTION_QUERY,TERMINAL_STATE_QUERY,VALUE);
The INITIAL_STATE_QUERY and TERMINAL_STATE_QUERY are regular queries containing all input and hidden variables.
The ACTION_QUERY is a regular query containing all output variables.
The VALUE is a probability value.
In short set(A,B,C,D) means "if A and you do B then the probability to jump to C is D".
When you put the above Perkun code in a file and execute it with perkun it will write down:
# model
# {beta=>do_nothing}
set({alpha=>false },{beta=>do_nothing },{alpha=>false },1.00000);
set({alpha=>false },{beta=>do_nothing },{alpha=>true },0.00000);
# {beta=>switch}
set({alpha=>false },{beta=>switch },{alpha=>false },0.00000);
set({alpha=>false },{beta=>switch },{alpha=>true },1.00000);
# {beta=>do_nothing}
set({alpha=>true },{beta=>do_nothing },{alpha=>false },0.00000);
set({alpha=>true },{beta=>do_nothing },{alpha=>true },1.00000);
# {beta=>switch}
set({alpha=>true },{beta=>switch },{alpha=>false },1.00000);
set({alpha=>true },{beta=>switch },{alpha=>true },0.00000);
It will be done by the command "cout << model << eol;".
Note that the sum of the probabilities over the terminal state queries equals 1.0 for each initial state query and each action query. Perkun checks this with a certain tolerance (and shows a warning on models that violate this rule).
Creating the Perkun models is difficult. The difficulty is of the quantitative nature - they tend to be very big. If you skip the set instructions in the model section the model will be random. Try running with perkun the following code:
values { value false,true; }
variables
{
input variable a:{false,true},b:{false,true},c:{false,true},
d:{false,true},e:{false,true};
output variable f:{false,true},g:{false,true},h:{false,true};
}
payoff {}
model {}
cout << model << eol;
The model printed out will likely be bigger than 1MB of code. Remember, the models "explode" with the number of variables.
Congratulations, now you know everything what you have to tell Perkun. Values, variables, payoff, model. The time has come to see what Perkun can do for you given this information. It will attempt to maximize the expected value of the payoff function by appropriate choosing the actions. First you give it current values of the input variables, then it responds with the optimal action chosen (the values of the output variables). As you can see we will interact with Perkun and this interaction will involve both input and output.
The loop command
As you remember the four sections in the Perkun code may be followed by instructions, and now is the time to learn another one: loop. It takes one argument - an integer constant, which denotes how deep Perkun should look into the game tree. Try the following code:
values
{
value false, true, do_nothing, switch;
}
variables
{
input variable alpha:{false, true};
output variable beta:{do_nothing, switch};
}
payoff
{
set({alpha=>false},0.0);
set({alpha=>true},100.0);
}
model
{
set({alpha=>false},{beta=>do_nothing},{alpha=>false},1.0);
set({alpha=>false},{beta=>do_nothing},{alpha=>true},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>false},0.0);
set({alpha=>true},{beta=>do_nothing},{alpha=>true},1.0);
set({alpha=>false},{beta=>switch},{alpha=>false},0.0);
set({alpha=>false},{beta=>switch},{alpha=>true},1.0);
set({alpha=>true},{beta=>switch},{alpha=>false},1.0);
set({alpha=>true},{beta=>switch},{alpha=>true},0.0);
}
loop(1);
The Perkun will respond with the message and a prompt:
loop with depth 1
I expect the values of the variables: alpha
perkun>
Now it is your turn. You can enter one of the two values that the input variable alpha may have, i.e. false or true. Let us first try false:
perkun> false
belief:
alpha=false 1
optimal action:
beta=switch
perkun>
As the optimal action the beta=switch was chosen! Why? Because Perkun can see alpha=false, it does not like it (see the payoff) and knows that if it uses switch the alpha will change its value to not alpha. How does it know it? From the model. That is it, this is how Perkun works. OK. Let us continue. Let us now enter true:
perkun> true
belief:
alpha=true 1
optimal action:
beta=do_nothing
perkun>
Again Perkun chooses an optimal action, but this time it is beta=do_nothing! Why? Because it can see alpha=true, it likes it (see the payoff) and knows that do_nothing will keep the value of alpha unmodified. The last fact, again, is known from the model.
Just for fun let us lie to Perkun and enter now "false". This is impossible, because the model says that alpha after beta=do_nothing is unmodified. But let us do it:
perkun> false
result
alpha=false its probability was 0
exception caught
line 28 at T_INT_LITERAL: syntax error
error
What happened? Perkun has thrown an exception and stated that alpha=false was impossible! Quite correct. The bad news is that it terminated the session. But if you use Perkun as a library there is a way to overcome it and continue the session, so that you do not have to worry that using Perkun would break your programs.
Illegal actions
You may be wondering how to prevent some actions to be chosen by Perkun in some situations. There is a special instruction for that used in the model section. It is called "illegal".
The syntax is:
illegal(VISIBLE_STATE_QUERY,ACTION_QUERY);
The VISIBLE_STATE_QUERY contains the input variable values, while the ACTION_QUERY contains the output variable values. That is it, Perkun won't try to use these actions when these situations occur!
Example:
illegal({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>false},
{action=>goto_Wyzima});
illegal({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>true},
{action=>goto_Wyzima});
illegal({where_is_Dorban=>place_Shadizar, do_I_see_vampire=>false},
{action=>goto_Shadizar});
illegal({where_is_Dorban=>place_Shadizar, do_I_see_vampire=>true},
{action=>goto_Shadizar});
illegal({where_is_Dorban=>place_Novigrad, do_I_see_vampire=>false},
{action=>goto_Novigrad});
illegal({where_is_Dorban=>place_Novigrad, do_I_see_vampire=>true},
{action=>goto_Novigrad});
Impossible states
If we have a deterministic model then it can be represented as a collection of directed graphs, each graph for one action (fixed values of the output variables). It may happen that in these graphs certain states cannot be achieved.
There is an instruction for the model section that denotes that. It is called "impossible".
Its syntax is:
impossible(STATE_QUERY);
The STATE_QUERY contains values of the input and hidden variables.
Example:
impossible({where_is_Dorban=>place_Wyzima, do_I_see_vampire=>false,
where_is_vampire=>place_Wyzima});
Perkun and XML
In order to print out an XML document describing the Perkun specification use the instruction:
cout << xml << eol;
Prolog generators
The following instruction prints out a Prolog code that will help you to create the payoff and the model:
cout << prolog generator << eol;
Haskell generators
The following instruction prints out a Haskell code that will help you to create the payoff and the model:
cout << haskell generator << eol;