Nantes Université

Skip to content
Extraits de code Groupes Projets
Non vérifiée Valider 9857a153 rédigé par Jacek Czerwonka's avatar Jacek Czerwonka Validation de GitHub
Parcourir les fichiers

Delete docs directory

parent eade4624
Aucune branche associée trouvée
Aucune étiquette associée trouvée
Aucune requête de fusion associée trouvée
Masquer les modifications d'espaces
En ligne Côte à côte
docs/_config.yml supprimé 100644 → 0
+ 0
− 3
Voir le fichier @ eade4624
title: Pairwise Testing
description: Combinatorial Test Case Generation
theme: jekyll-theme-slate
docs/about.md supprimé 100644 → 0
+ 0
− 11
Voir le fichier @ eade4624
---
layout: default
---
[(back)](./)
Thanks for stopping by. My name is Jacek Czerwonka and I work for Microsoft. For a while now, in addition may day job, I have been involved in designing and implementing pairwise-related tools and evangelizing pairwise testing at Microsoft. This web site is a collection of links related to pairwise that I have accumulated in that period of time. I hope you find this information useful.
If you come across something that should be included here, have a comment, or simply want to chat about pairwise testing, email me.
jcz (at) ieee (dot) com
docs/efficiency.md supprimé 100644 → 0
+ 0
− 26
Voir le fichier @ eade4624
---
layout: default
---
[(back)](./)
The basic measure of efficiency of a pairwise test generation tool is the number of tests a tool generates given some size of input. For example, when the input has four parameters with 3 values each, denoted 3⁴ in the table below, tools create between 9 and 11 test cases. All of the test suites meet the pairwise testing criterion of covering each pair of values in at least one test case however some tools can pack all these combinations them into fewer tests. This may not matter for small problems, test suites for large test domains can exhibit bigger differences.
Test generation efficiency is one aspect of a tool that a user will want to consider when choosing a tool to use, but numbers are easier to compare than less tangile aspects like "usability", so a standard set of benchmarks emerged over the years. The table below summarizes efficiencies for several tools that happened to publish their numbers.
| # | Model | 3⁴ | 3¹³ | 4¹⁵ 3¹⁷ 2²⁹ | 4¹ 3³⁹ 2³⁵ | 2¹⁰⁰ | 10²⁰ | Source |
| :---- | :----: | ----: | ----: | ----: | ----: | ----: | ----: | :---- |
| 1 | AETG | 9 | 15 | 41 | 28 | 10 | 180 | Y. Lei and K. C. Tai [In-parameter-order: a test generation strategy for pairwise testing](http://www-cse.uta.edu/~ylei/paper/hase.pdf), p. 8. |
| 2 | IPO | 9 | 17 | 34 | 26 | 15 | 212 | K. C. Tai and Y. Lei [A Test Generation Strategy for Pairwise Testing](http://ranger.uta.edu/~ylei/paper/ipo-tse.pdf) p. 2. |
| 3 | TConfig | 9 | 15 | 40 | 30 | 14 | 231 | A. W. Williams [Determination of Test Configurations for Pair-wise Interaction Coverage](http://www.site.uottawa.ca/~awilliam/papers/Testcom2000.pdf), p. 15. |
| 4 | CTS | 9 | 15 | 39 | 29 | 10 | 210 | A. Hartman and L. Raskin [Problems and Algorithms for Covering Arrays](http://www.haifa.il.ibm.com/projects/verification/mdt/papers/AlgorithmsForCoveringArraysPublication191203.pdf), p. 11. |
| 5 | Jenny | 11 | 18 | 38 | 28 | 16 | 193 | Supplied by Bob Jenkins. |
| 6 | TestCover | 9 | 15 | 29 | 21 | 10 | 181 | Supplied by George Sherwood. |
| 7 | DDA | ? | 18 | 35 | 27 | 15 | 201 | C. J. Colbourn, M. B. Cohen, R. C. Turban [A Deterministic Density Algorithm for Pairwise Interaction Coverage](http://www.public.asu.edu/~rturban/dda.pdf), p. 6. |
| 8 | AllPairs [McDowell] | 9 | 17 | 34 | 26 | 14 | 197 | Supplied by Bob Jenkins. |
| 9 | PICT | 9 | 18 | 37 | 27 | 15 | 210 | Supplied by Jacek Czerwonka. |
| 10 | EXACT | 9 | 15 | ? | 21 | 10 | ? | J. Yan, J. Zhang [Backtracking Algorithms and Search Heuristics to Generate Test Suites for Combinatorial Testing](http://doi.ieeecomputersociety.org/10.1109/COMPSAC.2006.33), p. 8. |
| 11 | IPO-s | 9 | 17 | 32 | 23 | 10 | 220 | A. Calvagna, A. Gargantini [IPO-s: Incremental Generation of Combinatorial Interaction Test Data Based on Symmetries of Covering Arrays](http://www2.computer.org/portal/web/csdl/doi/10.1109/ICSTW.2009.7), p. 17. |
| 12 | ecFeed | 10 | 19 | 37 | 28 | 16 | 203 | Supplied by Patryk Chamuczynski. |
| 13 | JCUnit | 10 | 23 | 49 | 33 | 18 | 245 | Supplied by Hiroshi Ukai [link](https://github.com/dakusui/jcunit/blob/0.8.x-develop/src/test/java/com/github/dakusui/jcunit8/experiments/StandardFactorSpaces.java). |
| 14 | CoverTable | 9 | 17 | 34 | 26 | 12 | 195 | CoverTable's [webpage](https://github.com/walkframe/covertable). |
docs/index.md supprimé 100644 → 0
+ 0
− 18
Voir le fichier @ eade4624
---
layout: default
---
Pairwise testing is an effective test case generation technique that is based on the observation that most faults are caused by interactions of at most two factors. Pairwise-generated test suites cover all combinations of two therefore are much smaller than exhaustive ones yet still very effective in finding defects.
### Resources
You can read more about this technique in many [papers](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C48&q=pairwise+testing&btnG=) and [books](https://www.amazon.com/gp/search?ie=UTF8&tag=pairwise-20&linkCode=ur2&linkId=d32d92503bf7cc820739af70b6913097&camp=1789&creative=9325&index=books&keywords=software%20combinatorial%20testing) published over the years.
### Tools
There are many [tools](./tools.md) that adopted and integrated pairwise testing as one of the test generation methods.
### Links
* [Pairwise testing tools](./tools.md)
* [Comparison of tools efficiency](./efficiency.md)
* [PICT Pairwise Test Generation tool](https://github.com/microsoft/pict)
* [About this site](./about.md)
docs/tools.md supprimé 100644 → 0
+ 0
− 65
Voir le fichier @ eade4624
---
layout: default
---
[(back)](./)
# Available Tools
| Tool | Authors | Notes |
| ---- | ---- | ---- |
| [CATS (Constrained Array Test System)](http://testcover.com/pub/background/index.php) | [Sherwood] Bell Labs. | |
| [OATS (Orthogonal Array Test System)](http://www.isixsigma.com/library/content/c030106a.asp) | [Phadke] ATT | |
| [AETG](http://aetgweb.argreenhouse.com/) | Telecordia | Web-based, commercial |
| [IPO (PairTest)](http://www-cse.uta.edu/~ylei/paper/hase.pdf) | [Tai/Lei] | |
| [TConfig](http://www.site.uottawa.ca/~awilliam/) | [Williams] | Java-applet |
| [TCG (Test Case Generator)](http://csmiss.jpl.nasa.gov/new/set/2000-prop1.doc) | NASA | |
| [AllPairs](http://www.satisfice.com/testmethod.shtml) | Satisfice | Perl script, free, GPL |
| [Pro-Test](http://www.sigmazone.com/protest.htm) | SigmaZone | GUI, commercial |
| [CTS (Combinatorial Test Services)](http://www.alphaworks.ibm.com/tech/cts) | IBM | Free for non-commercial use |
| [Jenny](http://burtleburtle.net/bob/math/jenny.html) | [Jenkins] | Command-line, free, public-domain |
| [ReduceArray2](http://www.sstc-online.org/proceedings/2004/PDFFiles/GTD683.pdf) | STSC, U.S. Air Force | Spreadsheet-based, free |
| [TestCover](http://www.testcover.com) | Testcover.com | Web-based, commercial |
| [DDA](http://www.public.asu.edu/~rturban/dda.pdf) | [Colburn/Cohen/Turban] | |
| [Test Vector Generator](http://sourceforge.net/projects/tvg/) | | GUI, free |
| [OA1](http://www.software-metrics.org/tools_testing.asp) | k sharp technology | |
| [TESTONA](https://www.assystem-germany.com/en/products/testona/) | Assystem Germany | GUI, free for non-comercial use |
| [AllPairs](http://www.mcdowella.demon.co.uk/allPairs.html) | [McDowell] | Command-line, free |
| [Intelligent Test Case Handler (replaces CTS)](http://alphaworks.ibm.com/tech/whitch) | IBM | Free for non-commercial use |
| [CaseMaker](http://www.casemakerinternational.com/) | Díaz & Hilterscheid | GUI, commercial |
| [PICT](http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi) | Microsoft Corp. | Command-line, open source at <a href="http://github.com/microsoft/pict">http://github.com/microsoft/pict</a> |
| [rdExpert](http://www.phadkeassociates.com/index_files/producthome.htm) | Phadke Associates, Inc. | |
| [OATSGen](http://delivery.acm.org/10.1145/1250000/1241582/p2-krishnan.pdf?key1=1241582&key2=7825558711&coll=ACM&dl=ACM&CFID=15151515&CFTOKEN=6184618) | Motorola | |
| [SmartTest](http://www.smartwaretechnologies.com/smarttestprod.htm) | Smartware Technologies Inc. | GUI, commercial |
| [EXACT](http://doi.ieeecomputersociety.org/10.1109/COMPSAC.2006.33 ) | [Yan/Zhang] | |
| [AllPairs](http://engineering.meta-comm.com/allpairs.aspx) | MetaCommunications | Free |
| [ATD](http://www.atyoursideconsulting.com/products/atd/atd_description.html) | AtYourSide Consulting | GUI, commercial |
| [ACTS [formerly: FireEye]](http://csrc.nist.gov/acts) | NIST | GUI |
| [Bender RBT Inc.](http://www.benderrbt.com/bendersoftware.htm) | BenderRBT | GUI, commercial |
| [Pairwise Test Case Generator](http://www.testersdesk.com/pairwse_testersdesk.html) | TestersDesk | Web-based |
| [Combo-Test](http://aehrc.com/research/health-data-management-and-semantics/combo-test) | The Australian eHealth Research Centre | Command-line, free |
| [IPO-s](http://www2.computer.org/portal/web/csdl/doi/10.1109/ICSTW.2009.7) | [Calvagna/Gargantini] | |
| [VPTAG ](http://sourceforge.net/projects/vptag/) | [Robert Vanderwall] | |
| [SpecExplorer](http://msdn.microsoft.com/en-us/library/ee620448.aspx) | Microsoft Corp. | GUI, free |
| [IBM Functional Coverage Unified Solution](http://researcher.ibm.com/project/1871) | IBM | GUI, commercial |
| [CombTestWeb ](http://alarcosj.esi.uclm.es/CombTestWeb/) | Universidad de Castilla-La Mancha | Web-based, free |
| [Hexawise](http://hexawise.com/) | Hexawise | Web-based, free & commercial |
| [PictMaster](http://en.sourceforge.jp/projects/pictmaster/) | IWATSU System & Software | Spreadsheet-based, free |
| [NTestCaseBuilder](https://www.nuget.org/packages/NTestCaseBuilder/) | [Murphy] | .NET library |
| [tcases](https://code.google.com/p/tcases/) | [Kimbrough] | Command-line, free |
| [Pairwiser](https://inductive.no/pairwiser/) | Inductive AS | Web-based, free & commercial |
| [NUnit](http://nunit.org/index.php?p=pairwise&r=2.6.4) | Poole et al | Unit test framework |
| [ecFeed](http://ecfeed.com/) | ecFeed AS | Standalone, Eclipse plug-in, and jUnit runner |
| [TechQA](http://pairwise.techqa.org:8080/) | | Web-based, free |
| [Pairwise Online Tool](https://pairwise.teremokgames.com/4s8/) | [Dementiev] | Web-based, free |
| [JCUnit](https://github.com/dakusui/jcunit) | [Ukai] | Unit test framework |
| [CAGen](https://matris.sba-research.org/tools/cagen/) | SBA Research | Web-based and command-line |
| [CTWedge](https://foselab.unibg.it/ctwedge/) | University of Bergamo | Web-based |
| [CAMetrics](https://matris.sba-research.org/tools/cametrics) | SBA Research | Web-based |
| [SQA Mate Tools: Pairwise](https://sqamate.com/tools/pairwise?rh=from.pairwise.org) | [Sotskov] | Web-based |
| [AllPairsPy](https://github.com/thombashi/allpairspy/) | [Hombashi] | Python library |
| [Pairwise Pict Online](https://pairwise.yuuniworks.com/) | [Tamura] | PICT on the web |
| [CoverTable](https://github.com/walkframe/covertable) | [Yasuyuki] | Python and TypeScript. Open source |
# Links
* [Comparison of efficiency](./efficiency.md)
0% ou .
You are about to add 0 people to the discussion. Proceed with caution.
Terminez d'abord l'édition de ce message.
Veuillez vous inscrire ou vous pour commenter