Hi Syd,
I have put it at the end of the document for now:
https://docs.google.com/document/d/1UhCg76_AgnS5PW0g_EN6zPZE4tTUs0bNeMXFVmZ4VuQ/edit#
Best,
Martina
Von: Tei-council <tei-council-bounces@lists.tei-c.org>
Im Auftrag von Bauman, Syd
Gesendet: Dienstag, 5. April 2022 19:32
An: tei-council@lists.tei-c.org
Betreff: [Tei-council] faster testing
One of the reasons the test procedure in Stylesheets/Test2/ is dramatically faster than the one in Stylesheets/Test/ is that it is, by default, run in parallel.[1,2]
You can also ask the
make command to run multiple jobs at once. The switches that control this are --jobs= and --output-sync=. I just tried an experiment, comparing
how long it took to run make vs
make --jobs=7 --output-sync=lines. (I chose 7 because my system has 8 threads, and I wanted to have some CPU available. What little I have found
on the web seems to suggest I may as well go ahead and use 8.)
The result was faster, although not even close to 7 times faster: down to 03:32 from 04:36. I compared the output of the 2 commands, and they were identical.
On GNU/Linux, at least, the
nproc command will tell you how many threads are available. Thus using the command
$ make --jobs=`nproc 2>/dev/null || echo 1` -Oline
seems to make sense to me. (-O is shorthand for
--output-sync=.)
It is also possible to get the Makefile to do that on its own. My first thought is that might not be such a good idea, because you may want to run with --jobs=1 in order to force error messages
into the right order. (I.e., in case -Oline wasn’t good enough.)
I ran the experiment again, this time using --jobs=8 and getting screen captures of the process monitor roughly 40 s after the make command started.[3] The timing results were very similar (down
to 03:36 from 04:36), but the order of output lines was different. (Same output; i.e., they were identical after sorting and removing timestamps.)
So I think anyone running the Stylesheets test process would do well to use the --jobs switch. You could use any of
`nproc 2> /dev/null || echo 1`
# if there is a chance nproc fails, so it defaults to '1'Martina — could you either update the notes we have about updating P5 subset, or remind me where they are and I will? Thanks.
Notes
[1] ant test runs them in parallel; if you want them in series (likely because
the order of messages was confusing when run in parallel) use ant testSeries.
[2] There are other reasons, like it is written to be less redundant, and the JVM is only spun up once, rather than once for every test.
[3] For evidence as to why --jobs makes
make faster, see
and
