Trish & I are doing the p5subset → Stylesheets process. Most of our time has been spent handling Test/. I think it is just too cumbersome — the amount of time we (Council, collectively) spend wrangling with Test/ is likely by now much greater than the amount
of time it saves by catching bugs early.
I think we need to do one of three things, toot sweet:
- Re-work the Test/Makefile so that the process is
much easier: e.g., when comparisons are deferred, a shell script to perform them needs to be generated; this might be hard to impossible.
- Stop using Test/, it is just not helpful enough to be worth it. Leave the directory in the repo so that we can use it when there is a particularly gnarly case, or when a human is not waiting for it, but remove it from the
general test procedure we use, e.g. when generating p5subset for the Stylesheets repo.
- Remove Test/ from the repo (and various instructions) entirely (and rename Test2/ to Test/ :-)
My vote is for (2). I could be perhaps be talked into (3) reasonably easily; it would take some serious convincing to get me to vote for (1).
P.S. BTW, Martin & I intended that we would do (3) when we developed Test2/. (I wanted to hang onto Test/, i.e. do (2), for awhile to ensure that everything tested by Test/ was also tested by Test2/.)