-
Notifications
You must be signed in to change notification settings - Fork 10
/
.plan
56 lines (42 loc) · 2.21 KB
/
.plan
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
Support Different Types of Testing:
Specs
Fuzz
Performance
Allow for sections within a description to have distinct before/after clauses
Possible UI sketch:
desc["Blah blah"]{
before {...}
should["blah blah"] {...}
holds["blah blah";`blah`blah`times!("blah";{blah > x};100)]{[fuzzDict];...} / Returns a value to be checked
perf["blah blah"; ((), `max)!(),0Nt;1 10 100!(dataset1;dataset2;dataset3)]{...}
after {...}
alt { / Only support one level of this
before {...}
should["blah blah"] {...} / These have the new before, but the old after
should["blah blah"] {...}
}
}
Possible data structure:
expec: `name`before`after`code!("...";{...};{...};{...})
fuzz: `name`before`after`code`runs`var!("...";{...};{...};{...};0N;({...};...);[`type | `type$() | `type$n#0 | val | `type$n#val | {...} | `name1`name2!var])
perf: `name`before`after`code`max`timeout`times`datasets!("...";{...};{...};{...};0Nt;0Nt;0N;1 10 100!(dataset1;dataset2;dataset3))
After running, the structures will also hold results (maybe they should just go in their own results dict?):
expec: `name`before`after`code`pass`passcode!("...";{...};{...};{...};1b;" ")
fuzz: `name`before`after`code`times`follows`gen`pass`passstring!("...";{...};{...};{...};0N;({...};...);({...};...);1b;"...")
perf: `name`before`after`code`min`max`times`pass`maxtime`mintime!("...";{...};{...};{...};0Nt;0Nt;0N;0Nt;0Nt)
Present cleanest possible interface:
Isolate all test related code into one namespace
Only present UI functions into main namespaces at test runtime (can delete from `.q using .[`.q;();_;`name]
Testing code uses no state except for a master list of tests (and some intermediate state for gathering functions)
Modular output facility:
Test code will generate intermediate data structures, not direct output.
Output should be generated by functions consuming elements from intermediate structures and returned as text blobs.
Must provide combinators for handling the user functions:
xmlWriterDesc:{[handlers;desc];
...
expecs: handlers[`expecs][xmlWriterExpec;desc[`expecs]];
fuzzes: handlers[`fuzzes][xmlWriterFuzz;desc[`fuzzes]];
perfs: handlers[`perfs][xmlWriterPerf;desc[`perfs]];
...
rstring
}