Nantes Université

Skip to content
Extraits de code Groupes Projets
Non vérifiée Valider 47896e3e rédigé par Dan Tripp's avatar Dan Tripp Validation de GitHub
Parcourir les fichiers

Docs: changing sub-model example. Adding some wording on randomness and heuristics. (#112)

* docs: expanding on 'Excluding Entire Parameters' (The "Dummy Value Technique")

* Docs: changing sub-model example. Adding some wording on randomness and heuristics.
parent 00adeb6f
Aucune branche associée trouvée
Aucune étiquette associée trouvée
Aucune requête de fusion associée trouvée
......@@ -72,6 +72,8 @@ The test cases are printed to the standard output stream. The first line of the
If a model and options given to the tool do not change, every run will result in the same output. However, the output can be randomized if ```/r``` option is used. A randomized generation prints out the seed used for that particular execution to the error output stream. Consequently, that seed can be fed into the tool with ```/r:seed``` option to replay a particular generation.
Different random seed values will often produce a different number of total test cases. This is because packing n-way combinations is a “hard problem” for which PICT and other tools use heuristics. These heuristics are deterministic, but they are dependent on initial conditions. Sometimes the algorithm is lucky and packs all of your desired combinations into fewer test cases. Variations of 5% - 10% are common.
All errors, warning messages, and other auxiliary information is printed to the error stream.
# Constraints
......@@ -191,11 +193,11 @@ Less typing and better for maintainability.
## Sub-Models
Sub-models allow the bundling of certain parameters into groups that get their own combinatory orders. This can be useful if combinations of certain parameters need to be tested more thoroughly or must be combined in separation from the other parameters in the model. The sub-model definition has the following format:
Sub-models allow the bundling of certain parameters into groups that get their own combinatory orders. This can be useful if combinations of certain parameters need to be tested more thoroughly, or less thoroughly, or in separation from the other parameters in the model. The sub-model definition has the following format:
{ <ParamName1>, <ParamName2>, <ParamName3>, ... } @ <Order>
For example, sub-modeling is useful when hardware and software parameters are combined together. Without sub-models, each test case would produce a new, unique hardware configuration. Placing all hardware parameters into one sub-model produces fewer distinct hardware configurations and potentially lowers the cost of testing. The order of combinations that can be assigned to each sub-model allows for additional flexibility.
For example, sub-modeling is useful when hardware and software parameters are combined together. Without sub-models, each test case would produce a new, unique (and costly) hardware configuration. Placing all hardware parameters into one sub-model produces fewer unique hardware configurations and so potentially lowers the cost of testing. The order of combinations that can be assigned to each sub-model allows for additional flexibility.
PLATFORM: x86, x64, arm
CPUS: 1, 2, 4
......@@ -205,20 +207,21 @@ For example, sub-modeling is useful when hardware and software parameters are co
Browser: Edge, Opera, Chrome, Firefox
APP: Word, Excel, Powerpoint
{ PLATFORM, CPUS, RAM, HDD } @ 3
{ OS, Browser } @ 2
{ PLATFORM, CPUS, RAM, HDD } @ 2
The test generation for the above model would proceed as follows:
$
|
| order = 2 (defined by /o)
|
+------------------------------+-----------------------------+
| | |
| order = 3 | order = 2 |
| | |
{ PLATFORM, CPUS, RAM, HDD } { OS, Browser } APP
$
|
| order = 3 (defined by /o:3)
|
+--------------------+-------------------+
| |
| order = 2 | order = 3
| |
{ PLATFORM, CPUS, RAM, HDD } OS, Browser, APP
For the above model with sub-model, PICT invoked with an /o:3 argument will generate roughly 10 hardware configurations, compared to roughly 40 hardware configurations without the sub-model. This reduced number of hardware configurations comes at the cost of a greater number of total test cases: roughly 140 test cases with the sub-model, compared to roughly 60 test cases without the sub-model (again, all with /o:3).
Notes:
1. You can define as many sub-models as you want; any parameter can belong to any number of sub-models. However, the model hierarchy can be just one level deep.
......
0% Chargement en cours ou .
You are about to add 0 people to the discussion. Proceed with caution.
Terminez d'abord l'édition de ce message.
Veuillez vous inscrire ou vous pour commenter