Example of creating a 'Test-rig' for Asset Testing :
A 'Test' is defined to exercise a particular interaction/reaction/behavior of a Object (NPC/prop/weapon/whatever) in the game and may be broken up into 'sub-tests' that will exercise a variation of the aspect being tested.
The tools that allow a player to create 'staged' choreographed 'plays' for the game (Like those Ghost incidents in BS1) can create 'Scripts' to make 'actors' (NPC or player Avatar) interact with terrain/props/each other in an exact fashion, thus exercise functionality and behavioral logic. Scripts can activate standard actions which all Objects of their type perform. (ex- activate a 'greeting' to another NPC or for a 'machine' to have its player activation button presses or for an NPC to 'runaway' from an enemy).
Testing using a 'scripted' setup (Test-rig script) takes alot of tediousness out of recreating the desired test -- you write the Script once and run it over and over with little extra effort. You can clone it and make small changes to test slightly different things with very littel effort.
For the Test, a simple situation in a 'Sandbox' arena would be created with all the necessary Objects placed (via Script). Part of the 'staging' tools that create these 'Play' scripts will be editors similar to game level creation tools which already exist in the game industry. The 'scenes' being created will be small and simple and made with a USER-FRINDLY interface - with 'pulldown lists' to select options/Objects, and a 3D view of the 'scene' with drag-n-drop used as you edit. The tool will record the identities of the Objects the player sets-up on his 'stage' and creats a 'Script' of directives that can reproduce the exact situation the player created. Those Scripts can then be replayed in the same tool to rcreate the staged scene.
The 'staging' tool integrates with a simulator that runs the Object's behaviors, so that the player can hit a 'execute' button and the Objects in the 'play' will act as if in the real game (and react to each other accordingly). The player then watches to verify the behavior they are testing. Due to the complexities of the 'simulation' the tool will actually tie-in to a Test Server which will perform the actual execution.
If the Asset is an 'auto-generated' asset, then the autogeneration would be run thru the Test Server to create it, but with Parameters specified for the current Test's circumstances (all variations of 'parameters' have a seperate set of Tests that should exist for the Asset Type in question). If the autogeneration Asset is designed for random variations, the Script might be run over and over to see if valid Objects are consistantly produced. All variations are described in the Assets design documentation.
The Objects created to interact in the Test would be set into a correct state required by the "Test's" circumstances (example- a Spicer would have its hitpoints set to near death to cause it to do a 'retreat' type of behavior and an enemy would be placed near it that it would likely try to run away from or use some other tactic).
Once the 'stage' is set, the Tester may take the role of a player Avatar to MANUALLY test varying actions -- to easily try variations against the tested Object (variations in actions, timing, position, sequence) and do it over and over to look for correct consistant behavior ( many time the reactions may be randomized or behavior tactics with multiple options). Complicated NPC behaviors require this manual testing (semi-automatic if you count that the situation was setup by a Script).
Debug tools allow looking into the internal state of Objects so the tester (Author at least) can make sure that those attributes are what they are expected to be (and watch them change as the 'Test' situation unfolds). The tools allow tracing progress thru execution of the Objects logic so the Tester can see the expected calculations are being run.
Scripts can invoke other scripts, so a common 'stage' setting could be written (Scripted) once and then used repeatedly in many other Tests.
The Author of the Asset should create a number of these 'Tests' as part of their proving that their Asset behaves correctly (Authors are supposed to test their creation). The scripts they would produce are then attached to the Asset Project for retesting later. Common testing 'scenes' would be pre-provided as examples by the Community.
Many people may have trouble using the Tools to write these 'Test Scripts', but with good Examples and Tutorials and being able to easily modify them (to do whats needed), many more players can make use of this methodology.
FAQs will exist with lots of advice about how proper testing should be done and the IDE will have checkists for the 'Tests' and 'Examples' normally expected to be provided for submissions. If all else fails, there are Experts in the Community who can assist (via collaboration) to write proper Tests (at least the minimum automatic Tests). Unfortunately if too many people 'beg off' and dont create their own tests then the 'Experts' will be swamped, so its important that the testing and Test-rig script creation is as easy as possible.
One important side-effect of the Authors knowing how to use these scripting tools is that the very same methods are used to create the 'Plays' (forced acting) imbedded in Dialog Trees and Quests and the 'staging' used for Quests and for Auto-Generation of terrain.
Creation of Tests itself is a Collaboration. Many Objects are similar to other existing Object Types and can use THOSE Object's Tests for the 'new' Object.
(example- a 'Chair' has a bunch of expected in-game reactions/behaviors and anything classified as a 'chair' should Pass these 'Chair' (Type Template) Tests. If the new 'Chair' being created has something different about it (maybe it is a 'Recliner') then that additional functionality would have to be seperately tested -- IN Addition to the standard 'Chair' tests . It would be likely that a new Taxonomy' (classification) would be created for this new 'Recliner Chair', listed underneath the more generic 'Chair' classification. The Author would be expected to define some Tests to demonstrate the new abilities/use of this 'Recliner Chair' and THOSE Tests (after some later review) would become 'standard' tests for subsequent 'Recliner Chairs' (and be used by the next person who wanted to creat one of those).
. . . . .