6. Simulation

Section 5, Evaluating Decision Models, described how to use the World Modeler Server API to evaluate, or execute, a Decision Model for a single set of input arguments (that is for a single set of Action and External values). One of the benefits of using World Modeler Server is that Decision Models can be automatically executed many times over a range of input values to give the decision-maker more insight into how the Actions they may take, or the Externals they may encounter will affect the Outcomes they are trying to achieve. We call such sets of runs over a range of inputs Simulations. As well as providing clearer insight regarding the relationship between decision inputs and Outcomes, Simulations can also be set up to search for the inputs that result in optimal Outcomes.

Simulations are run in the World Modeler Server API by invoking the simulate operation, whose signature is shown below.

POST		/simulate/{id}

Where id is the ID of the Decision Model to be simulated. The details of the request body are described in the section entitled Using the evaluate API call, below. Note that, as with the evaluate API call described in section 5, the simulate operation utilizes the Processors subsytem in World Modeler Server. If you are not familiar with World Modeler Server’s Processors model, or need a refresher, it is recommended that you read the material under the heading Execution Processors in World Modeler Server in section 5 before proceeding to the sections below.

Using the simulate API call

The simulate operation is invoked as shown below. In particular, its body must contain a JSON object formatted as shown below.

URL: https://host.domain/simulate/{id} Method: POST

Body:

{
  "processorID":           "string" or null,
  "returnElements":        ["string","string",...],
  "simulationParams":      simulation parameters,  // see below  
  "runImmediate":          true or false,
  "saveAs":                "name" or null,
  "summary":               "short summary string" or null,
  "documentation":         "longer documentation string" or null,
  "documentationMIMEType": "MIME type of the documentation string" or null,
  "userData":              "string" or null
}

URL Arguments

id	The ID of the Decision Model to be simulated.

Request body JSON

processorID	The ID of a processor which has been created by a previous call to evaluate or simulate and is still valid. If there have been no previous calls toevaluate or simulate, then processorID should be null, or an empty string and the new processor ID will be returned in the response if the call succeeded (see The Processor ID in section 5). Note that the processorID specified must be associated with the Decision Model whose ID is specified in the id URL argument.
returnElements	An array of strings, each of which is the a. ID, b. FullyQualifiedName, or c. Name of a Decision Model Element in the Decision Model whose ID is specified in the id URL argument. Note that the search for matching elements is performed in the order specified above. The purpose of this is to allow the caller to limit the data that is returned when the processor simulates the model to only those elements that the caller is interested in. This improves efficiency and reduces network traffic when large models are evaluated. The following values may be specified null All elements in the model and their values are returned [] (empty array) No elements in the model are returned ["element1","element2"...] The elements whose IDs, FullyQualifiedNames, or Names are included in the array.
simulationParams	The parameters that control the simulation. See Simulation Parameters, below.
runImmediate	If the saveAs argument is populated, the caller has the option of letting the system create the simulation record, but defer actually running the simulation to a later call to simulate. Whether the simulation is actually run is controlled by the value of the runImmediate argument. If set to true, the simulation will be run as defined in the simulationParams. If set to false, the saved simulation record will be created and returned to the caller, but no results will be generated.
saveAs	If the inputs and results of the simulation are to be saved, then this argument specifies the name used to refer to the saved simulation. If this argument is null, then the simulation will not be saved.
summary	May be null. If the saveAs argument is not null, then this argument allows a short summary string to be included in the record of the saved simulation run. If saveAs is null, then this argument is ignored.
documentation	May be null. If the saveAs argument is not null, then this argument allows a longer documentation string to be included in the record of the saved simulation run. The MIME type of the string must be specified in the documentationMIMEType argument. If saveAs is null, then this argument is ignored.
documentationMIMEType	If both the saveAs argument and the documentation argument are not null, then this argument must be set to the string identifying the MIME type of text in the documentation argument. If either saveAs or documentation are null, then this argument is ignored.
userData	An optional, user-defined text string that can be passed with the call. If defined, the string will be returned in the response to this call. This can be used for responses to requests, for example to sort the sequence of returned responses so it matches the order in which the requests were made.

Simulation parameters

The value of the simulationParams key in the above JSON schema is a SimulationParams objects that defines the various parameters that control the simulation process. The simulationParams object is defined as shown below.

"simulationParams":{
   "inputParams":{
            "decisionModelElement1": { sample generator object (see below) },
            "decisionModelElement2": { sample generator object (see below) },
            "decisionModelElement3": { sample generator object (see below) },                         
             ...
    },
    "stopCondition": "expression source string returning a boolean condition
                      which will cause simulation to stop when true",
    "goal":           "(optional) The goal or goals the simluation is configured to seek.",
    "constraint":     "(optional) The conditions that make a simulated scenario invalid"
}

Further details on each of the values in the simulationParams object are provided in the table below.

inputParams	Specification of which elements of the Decision Model will act as inputs to the simulation (that is, the independent variables whose values are changed from one simulation run to the next). See Sample Generators below for more details.
stopCondition	A string that defines a boolean expression or code block which, when it returns true, will cause simulation processing to stop and the operation to return to the caller (see Calculating the value of a Decision Model Element in section 4 for more information about expressions and code blocks). The expression may include references to any element of the Decision Model, any standard Javascript library function, and any property of the WMSimulation object (see below). For example: // Assume the model has an element AvailableBudget. // Stop when the available budget is exhausted. AvailableBudget <= 0; If a stopCondition value is not specified or set to null, the following expression is used by default to ensure the simulation does not continue indefinitely. WMSimulation.runCount >= WMSimulation.maxRunCount; where MaxSimiulationRunCount is defined by the system administrator in the World Modeler Server configuration and is set to 10,000 by default.
goal	Optional section specifying optimal conditions, or goals, that the simulation is searching for. The goal section, if included, is structured as shown below. "goal": { "goal1": { "condition": "simple condition expression", "priority": 1 }, "goal2": { "condition": "cond1 && (cond2 || !cond3)", "priority": 2 } } See the section on Goals and constraints, below, for more information.
constraint	See the Constraints section, below.

Goals and constraints

Many simulations are run to find the set of input variables (i.e., Actions and Externals—see Elements of a Decision Model) that yeild the most optimal outcomes. We call these outcomes goals . The World Modeler Server API supports goals via the information provided in the goals section of the JSON sent in the message body when the simulate operation is invoked.

Goals can be quite simple to specify. Suppose a Decision Model has an Outcome called Profit and the goal being sought is for Profit to be greater than 100. Stating the goal in this case is straightforward. We could just write

Profit > 100

The simulator can then run through different combinations of input variables until the goal condition is true.

JSON format for specifying goals in the HTTP body of a simulate invocation

The JSON passed in the body of each invocation of simulate (see Simulation Parameters, above) may optionally contain a key named goal. If present, goal must contain at at least one key (but possibly more) that represents a goal name. This can be any string that is a valid JSON object key and should be descriptive of the goal. The simplest form for the value of a goal name is the source text for a boolean expression evaluates to @true@ when the goal is met.

For example, if the goal of a simulation is that Profit is greater than 100, the goal section would be as follows;

"goal": { 
    "profitGT100": "Profit > 100"
}

The second form for specifying the contents of a goal name is used when several goal conditions exist and must be prioritized and we will cover this in Multiple goal conditions, below.

While the JSON block above illustrates an example of a simple goal, in practice, goals are usually more complicated than a simple variable = value statement. Here are some further considerations that often come into play:

• Goals may have multiple conditions that need to be met, each of which may have a different weighting.
• There may be constraints that either disqualify certain scenarios (hard constraints), or make them less desirable (soft constraints).
• Goals may be hierarchical. There may be a primary goal and once this is achieved, there may be secondary, tertiary (and so on) goals that are also sought.

Taking all this into account requires that we think of reaching a goal as more than testing a condition.

There is another reason why representing a goal as a condition is not ideal. As the simulator is evaluating scenarios, some will be closer to meeting the goal, and others will be furhter from it. This can be used by the simulator to optimize the goal search, but not if the only information available to the simulator is a true or false determination of whether each scenario satisfies the goal or not; true or false does not indicate how far a given scenario is from reaching the goal, nor in which direction the goal lies, so the simulator has no access to information that may make the goal search much more efficient.

Let's address each of these in turn and see how the World Modeler Server API handles them, starting with the last point first, that is, how to specify goals in a way that supports search optimization.

Supporting optimized goal search

Suppose, as we have been doing, that we assume a goal is expressed as a boolean conditional that compares a model value (or an expression containing model values—usually Outcomes) to a target value or expression. The goal is achieved when a value expression and target expression are found for which the comparison operator (let us refer to this as compare()) returns true, that is:

compare(value, target) == true

How is this different from simply writing out the conditional that defines the goal? That is, assume the goal condition is variable a is greater than variable b, how is

a > b == true

different from

WM.GreaterThan(a, b) == true

where we assume WM.GreaterThan returns true if a is greather than b and false otherwise? They are different because the expression a > b simply returns true or false and, in general, there is no easy way the World Modeler Processor can separately identify the value expression, coditional operator, and target expression. Therefore, there is no general way of determining either "distance" to the goal, nor the "direction" to it.

By contrast, the GreaterThan(value, target) function can unambigously identify the value and the target, and independently evaluate each of these expressions. These can be passed to any internal optimization algorithms that may help the simulator converge on the goal more quickly. So, to use the World Modeler optimizer, the JSON specifying the "profit greater than 100" goal should be written as

"goal":{
    "profitGT100": "WM.GreaterThan(Profit, 100)"
}

Note that the WM.GreaterThan function is one of several built-in comparison operators that work with the World Modeler optimizer.

Multiple goal conditions

In many situations, the goal being sought doesn't just involve a single condition on a single model element, but may involve many conditions on many model elements. This is easy to accommodate since all that is reqruied to define a goal is a boolean-valued expression. Multiple boolean expressions, each representing a goal condition, can be combined using the familiar boolean operators "and" (&&), "or" (||), and "not"(!'). So, for example, if the goal condition is stated as "Profit greater than 100, and either Cost less than 50 or UnitsSold is the maximum value for the simulation. The "goals" section of the simulate input parameters is then:

"goals": {
    "profitCostUnits": "GreaterThan(Profit, 100) && (LessThan(Cost, 50) || Max(UnitsSold) )"
}

Hierarchical goals

Sometimes, when multiple goals are being sought, they are arranged in a hierarchy. For example, suppose the

"Profit greater than 100"

goal is extended to

"(1) Profit greater than 100, and (2) of the scenarios that satisfy (1), choose the one(s) with the minimum CarbonEmissions".

In general, a goal hierarchy is an ordered sequence of goal conditions, where the domain of each level of the sequence is the collection of scenarios returned by the previous level.

To represent the hierarchical Profit and CarbonEmissions goals in the goals section of the simulation parameters, the following syntax is used.

"goal": {
    "profitGT100": {
         "condition": "GreaterThan(Profit, 100)",
         "priority": 1
    },
    "minCO2": {
         "condition": "Min(CarbonEmissions)",
         "priority": 2
    }
}

Constraints

Along with goal conditions that a simulation is being run to achieve, there are often constraints that a simulation must avoid. That is, any scenario that violates the constraint conditions must be rejected, even if the scenario satisfies the goals. Constraints are specified in the same way as goals (see JSON format for specifying goals in the HTTP body of a simulate invocation, above), except that the simulation will reject any scenario where the constraint condition, or conditions, are false. For example, a single constraint can be specified as:

"constraint": {
   "budgetLimit": "GreaterThan( AvailableFunds, 0)"
}

Any scenario where AvailableFunds is not greater than 0 will be rejected.

Hierarchical constraints are supported in a way that is analogous to hierarchical goals:

"constraint": {
    "budgetLimit": { 
        "condition":": "GreaterThan( AvailableFunds, 0)",
        "priority": 1
    },
    "emissionsLimit": { 
        "condition":": "LessThan( CO2Outupt, 40)",
        "priority": 2
     }
}

In the above example, the World Modeler Processor will test each of the constraints in ascending priority order. That is, the budgetLimit constraint first. If the condition evaluates to false, the constraint will cause the scenario to be rejected. If the condition is true, the simulator will then evaluate the condition in emissionsLimit constraint and accept or reject the scenario depending on the result.

Sample generators

When a simulation is run, the Decision Model associated with it is evaluated, or executed, over and over again in a loop that cycles through the different choices (or Actions) open to the decision-maker, and the various circumstances that may exist for each choice (or Externals). We collectively refer to Actions and Externals as the Input Parameters of a simulation, and to a particular combination of Actions and Externals as a scenario (for more information about the elements of a Decision Model and the various roles they play, see Elements of a Decision Model in section 4). Each cycle of a simulation loop consists of three basic steps:

1. Generate the set of Input Parameters (Actions and Externals) for the loop iteration,
2. Evaluate the Decision Model with the inputs generated in a), and
3. After the evaluation is complete, determine if another iteration is required by evaluating the stop condition and goal condition.

The values in step a) are created by functions called sample generators. They are called this because their purpose during each iteration is to generate a value sampled from the domain of the Input Parameter they are bound to. For example, if DayOfWeek is an Input Parameter to the Decision Model being simulated, then its sample generator might randomly select a day from Monday through Friday each time it is run. The logic used by the sample generator to populate its associated argument with values is up to the caller. While this allows for great flexibility, as callers can create their own sample generator functions, there are several built-in functions that will probably be adequate for most cases.

The first step in configuring a Simulation of a Decision Model is to specify which Input Parameters (that is, which Actions and/or Externals) will be varied from one evaluation run of the model to the next, and over what ranges of values. This is done using inputParams object (see Simulation parameters, above), whose form is represented in JSON as:

"inputParams": {
  "inputParam1": {
      "sampler": "string specifying the sampler function",
      sampler configuration parameters...
  },
  "inputParam2": {
      "sampler": "string specifying the sampler function",
      sampler configuration parameters...
  },
  ... etc.
}

A named key is included in the inputParams object for each of the Input Parameters that is to be varied in the simulation and its value set to an object. This object is required to have at least one key named sampler whose value is a string that identifies how the domain of the Input Parameter is sampled by the simulator. The supported sampling distributions and their configuration are shown in the table below.

Continuous Distributions

Sampler Value and Description	inputParam JSON Format
"ContinuousUniform" Random, double precision samples drawn with equal probability over an interval.	{ "sampler": "ContinuousUniform", "samplerParams": { "min": min value of sample range, "max": max value of sample range } }
"Interpolated" Random samples drawn from the range [x0, xn] with probability y given by linearly interpolating from a table of control points: [[x0, y0], … , [xn, yn]]	{ "sampler": "Interpolated", "samplerParams": { "controlPoints": [ [x1,y1], [x2,y2],... ] } }
"Normal" Random samples drawn from the range [min, max] with probability given by the normal distribution with the specified mean and standard deviation (sd).	{ "sampler": "Normal", "samplerParams": { "mean": distribution mean, "sd": distribution standard deviation } }
"LogNormal" Random samples drawn from the log normal distribution with the specified μ and σ parameters. .	{ "sampler": "LogNormal", "samplerParams": { "mu": log scale, "sigma": standard deviation } }
"Beta" Random samples drawn from the range [0, 1] whose probability is given by the beta distribution with the specified α and β parameters.	{ "sampler": "Beta", "samplerParams": { "alpha": alpha parameter, "beta": beta parameter } }
"Cauchy" Random samples drawn from the range [0, 1] whose probability is given by the Cauchy distribution with the specified x0 and γ parameters.	{ "sampler": "Cauchy", "samplerParams": { "x0": x0 parameter, "gamma": gamma parameter } }
"Chi" Random samples drawn from the range [0, ∞) whose probability is given by the chi distribution with the number of degrees of freedom specified by the integer k	{ "sampler": "Chi", "samplerParams": { "k": degrees of freedom parameter } }
"ChiSquared" Random samples drawn from the range [0, ∞) whose probability is given by the chi squared distribution with the number of degrees of freedom specified by the integer k	{ "sampler": "ChiSquared", "samplerParams": { "k": degrees of freedom parameter } }
"Erlang" Random samples drawn from the range [0, ∞) whose probability is given by the Erlang distribution with the shape parameter k and the rate parameter λ	{ "sampler": "Erlang", "samplerParams": { "k": shape parameter, "lambda": rate parameter } }
"Exponential" Random samples drawn from the range [0, ∞) whose probability is given by the exponential distribution with the rate parameter λ	{ "sampler": "Exponential", "samplerParams": { "lambda": rate parameter } }
"Gamma" Random samples drawn from the range [0, ∞) whose probability is given by the Gamma Distribution with parameters shape and rate.	{ "sampler": "Gamma", "samplerParams": { "shape": shape parameter, "rate": rate parameter } }
"InverseGamma" Random samples drawn from the range [0, ∞) whose probability is given by the Inverse-Gamma Distribution with parameters shape and rate.	{ "sampler": "InverseGamma", "samplerParams": { "shape": shape parameter, "rate": rate parameter } }
"Laplace" Random samples drawn from the range [0, ∞) whose probability is given by the Laplace Distribution with location parameter μ and scale or "diversity" parameter b.	{ "sampler": "Laplace", "samplerParams": { "mu": shape parameter, "b": scale parameter } }
"Pareto" Random samples drawn from the range [0, ∞) whose probability is given by the Pareto Distribution with shape parameter α and scale or parameter m.	{ "sampler": "Pareto", "samplerParams": { "alpha": shape parameter, "m": scale parameter } }
"Rayleigh" Random samples drawn from the range [0, ∞) whose probability is given by the Rayleigh Distribution with scale parameter σ.	{ "sampler": "Rayleigh", "samplerParams": { "sigma": scale parameter } }
"Stable" Random samples drawn from the range (-∞, ∞) whose probability is given by the Stable Distribution with stability parameter α, skewness parameter β, scale parameter c, and location parameter μ.	{ "sampler": "Stable", "samplerParams": { "alpha": stability parameter, "beta": skewness parameter, "c": scale parameter, "mu": location parameter } }
"StudentT" Random samples drawn from the range (-∞, ∞) whose probability is given by the Student's-T Distribution with location parameter μ, scale parameter σ, and degrees of freedom parameter ν.	{ "sampler": "StudentT", "samplerParams": { "mu": location parameter, "sigma": scale parameter, "nu": degrees of freedom parameter } }
"Weibull" Random samples drawn from the range [0], ∞) whose probability is given by the Weibull Distribution with shape parameter k and scale parameter λ.	{ "sampler": "Weibull", "samplerParams": { "k": shape parameter, "lambda": scale parameter } }
"Triangular" Random samples drawn from the range [0], ∞) whose probability is given by the Triangular Distribution with lower bound a, upper bound b>, and mode c. Note that this is equivalent to the linearly interpolated distribution with control points at [[a,0], [c,(b-a)/2], [b,0]]	{ "sampler": "Triangular", "samplerParams": { "a": lower bound, "b": upper bound, "c": mode } }

Discrete Distributions

Sampler Value and Description	inputParam JSON Format
"DiscreteUniform" Random, integer samples drawn with equal probability over an interval.	{ "sampler": "DiscreteUniform", "samplerParams": { "min": min value of sample range, "max": max value of sample range } }
"Bernoulli" Integer samples drawn with probability given by the Bernoulli Distribution parameterized by the probability p of the value 1.	{ "sampler": "Bernoulli", "samplerParams": { "p": Probability of outcome = 1 } }
"Binomial" Integer samples drawn with probability given by the Bnomial Distribution parameterized by the probability p of a successful outcome and the number of trials n.	{ "sampler": "Binomial", "samplerParams": { "p": Successful outcome probability, "n": number of trials } }
"NegativeBinomial" Integer samples drawn with probability given by the Nevative Bnomial Distribution parameterized by the probability p of a successful outcome and the number of successes r required to stop the experiment (the "success stop count").	{ "sampler": "NegativeBinomial", "samplerParams": { "p": Successful outcome probability, "r": success stop count } }
"Geometric" Integer samples drawn with probability given by the Geometric Distribution parameterized by p, the probability of generating an outcome of 1.	{ "sampler": "Geometric", "samplerParams": { "p": Probability of outcome = 1 } }
"HypergGeometric" Integer samples drawn with probability given by the Hypergeometric Distribution parameterized by the population size N, the number of successes s and the number of draws d.	{ "sampler": "Hypergeometric", "samplerParams": { "N": population, "s": number of successes, "d": number of draws } }
"Poisson" Integer samples drawn with probability given by the Poisson Distribution parameterized by the coefficient λ.	{ "sampler": "Poisson", "samplerParams": { "lambda": lambda coefficient } }
"Categorical" Integer samples drawn with probability given by the Categorical Distribution parameterized by a non-zero-length array of floating point numbers K, where each element of K represents a particular outcome, and its value is the probability of that outcome.	{ "sampler": "Categorical", "samplerParams": { "K": outcome probability array } }
"CMP" Integer samples drawn with probability given by the Conway-Maxwell-Poisson Distribution parameterized by the coefficient λ, from the Poisson distribution, and the decay parameter ν.	{ "sampler": "CMP", "samplerParams": { "lambda": Poisson lambda coefficient, "nu": decay paraneter } }
"Zipf" Integer samples drawn with probability given by Zipf's Law parameterized by the coefficients b and n from the frequency relationship: f(x)  ∝  1 (x + b) n	{ "sampler": "Zipf", "samplerParams": { "b": b parameter, "n": n parameter } }

WMSimulation object

While simulation running, simulation data can be obtained from the WMSimulation object

Access to simulator objects

WMSimulation.runCount

Returns the number of runs the simulator has executed to this point.

WM.WMSimulation.maxRunCount

Returns the maximum number of runs the simulator will run.

WM.WMSimulation.simulationParams

Returns the simulationParams for this simulation run.

WM.WMSimulation.arguments

Argument values used for this simulation run

WM.WMSimulation.stopCondition

Function reference that tests the stop condition

WM.WMSimulation.goal

The goal object, if one was specified.

WM.WMSimulation.results

The results generated by the simulation runs so far.

WM.WMSimulation.decisionModel

Reference to the Decision Model being run in this simulation.

Comparison functions that utilize the World Modeler optimizer

WM.EqualTo( value, target )

Returns true if value is equal to target.

WM.NotEqualTo( value, target )

Returns true if value is not equal to target.

WM.LessThan( value, target )

Returns true if value is less than target.

WM.LessThanOrEqualTo( value, target )

Returns true if value is less than or equal to target..

WM.GreaterThan( value, target )

Returns true if value is greater than target.

WM.GreaterThanOrEqualTo( value, target )

Returns true if value is greater than or equal to target.

WM.Min( value, label )

Returns true if value is the minimum of all calls to WM.Min during the simulation. If multiple values are being tracked independently, each can be identified by specifying a distinct string-valued label. This is optional.

WM.Max( value, label)

Returns true if value is the maximum of all calls to WM.Max during the simulation. If multiple values are being tracked independently, each can be identified by specifying a distinct string-valued label. This is optional.

Managing saved simulation runs

⇦ Prev: 5. Evaluating Decision Models