Sounds like it’s the schema which has crossed a threshold. Might be time for the pruning shears. Also possible some recent change has ramped up complexity?

On Feb 23, 2022, at 20:01, Martin Holmes wrote:

It's definitely rnv which is the problem, and I can't find a workaround. :-(

I don't think it's actually anything to do with the size of the file, but I can't figure it out. Any ideas?

Cheers, Martin

...

On 2022-02-23 16:14, Martin Holmes wrote: Hi all, Today, my Jenkins started to fail on the TEIP5-Test-dev build with an out-of-memory error when running RNV, which I can also reproduce on my desktop: ... java -jar ../Utilities/lib/trang.jar testplus.rng testplus.rnc xmllint --noent --dropdtd testplus.xml | rnv testplus.rnc stdin:1:0: error: out of memory error: invalid input make[1]: *** [Makefile:78: testplus.special] Error 1 make[1]: Leaving directory '/home/mholmes/tei/TEI/P5/Test' make: *** [Makefile:195: test] Error 2 The failure is on line 78 of the Test/Makefile, and it's when testplus.xml is processed through xmllint and passed to rnv for validation against its matching rnc file. It looks like Peter's Jenkins is not suffering from this, while Raff's has not run the job since last August, so something else is problematic there. Can those of you who run the docker instance try running the tests and see what you get? I suspect that the testplus.xml file has just somehow edged over a size threshold and we can work around it by throwing a bit of memory at xmllint or at rnv, but I'd like to get a bit more info first. Cheers, Martin ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre I acknowledge and respect the lək̓ʷəŋən peoples on whose traditional territory the university stands and the Songhees, Esquimalt and WSÁNEĆ peoples whose historical relationships with the land continue to this day. _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre

I acknowledge and respect the lək̓ʷəŋən peoples on whose traditional territory the university stands and the Songhees, Esquimalt and WSÁNEĆ peoples whose historical relationships with the land continue to this day. _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

I tried the -s switch and it made no difference. Did it work for you? Cheers, Martin On 2022-02-23 18:51, Bauman, Syd wrote:

Ha! I created a ticket https://github.com/TEIC/TEI/issues/2239 for this very problem before I saw this in my inbox. (Sorry.)

But as it says on ticket, not only does it happen (with rnv v. 1.7.8) on my local system, it happens when I test with a tiny schema and a tiny file.

How do we throw more memory at rnv? It is a compiled C program. (Although it “uses Expat”, so could the problem be there?) Answer: if we pass rnv the -s switch it “uses less memory and runs slower”.

------------------------------------------------------------------------

Today, my Jenkins started to fail on the TEIP5-Test-dev build with an out-of-memory error when running RNV, which I can also reproduce on my desktop:

... java -jar ../Utilities/lib/trang.jar testplus.rng testplus.rnc xmllint --noent --dropdtd  testplus.xml | rnv testplus.rnc stdin:1:0: error: out of memory error: invalid input make[1]: *** [Makefile:78: testplus.special] Error 1 make[1]: Leaving directory '/home/mholmes/tei/TEI/P5/Test' make: *** [Makefile:195: test] Error 2

The failure is on line 78 of the Test/Makefile, and it's when testplus.xml is processed through xmllint and passed to rnv for validation against its matching rnc file.

It looks like Peter's Jenkins is not suffering from this, while Raff's has not run the job since last August, so something else is problematic there.

Can those of you who run the docker instance try running the tests and see what you get? I suspect that the testplus.xml file has just somehow edged over a size threshold and we can work around it by throwing a bit of memory at xmllint or at rnv, but I'd like to get a bit more info first.

_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca

I compiled my desktop version from the dtolpin repo: https://github.com/dtolpin/RNV/blob/master/Makefile.gnu which has version 1.8.0; my Jenkins has the same version. But obviously that's not the issue, because my you have 1.7.8. My desktop is Ubuntu 21.10, and Jenkins server is 20.04. I find these expat-related things: /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.1 /usr/lib/x86_64-linux-gnu/cmake/expat-2.4.1 but I'm not sure how to interpret them. They are timestamped February 17, though, so obviously something changed recently. So my best guess is that an update to libexpat broke RNV completely. As Hugh says, it might just be a case of one more dependency we could drop. I think the RNV validation is just backing up the Jing validation really. Cheers, Martin On 2022-02-24 03:13, Bauman, Syd wrote:

Just tried now. (On tiny file.) Nope, -s switch did not help.

What version of Ubuntu are you running? I plan to try this on several different machines over the next few hours.

I do wonder where 1.8.0 came from. The place I think of as the canonical source http://ftp.davidashen.net/PreTI/RNV/ only has 1.7.8.

------------------------------------------------------------------------ I tried the -s switch and it made no difference. Did it work for you?

_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre I acknowledge and respect the lək̓ʷəŋən peoples on whose traditional territory the university stands and the Songhees, Esquimalt and WSÁNEĆ peoples whose historical relationships with the land continue to this day.