I very recently started having to increase the value of --maxdepth on my system when running that ant task. I think (but am not at all sure) it is because of my recent upgrade to a new OS, which involved a new version of libxslt and xsltproc, which maybe has a lower default value of maxdepth? Anyway, I set it to 10000 in one branch or PR somewhere, and that worked. (The maxdepth is the number of recursive template call allowed before presumed infinite and thus failure. The Schematron skeleton XSLT stylesheet now used parses the values of XPaths a character at a time using a recursive template. They (the XPaths being parsed) can get very long, even moreso given that it does not perform whitespace normalization beforehand. I do not know what the default maxdepth used to be, but it is 3000 on my system now, and I tried 10000 and that worked.) ________________________________ From: Tei-council on behalf of Martin Holmes Sent: Monday, February 1, 2021 13:01 To: tei-council@lists.tei-c.org Subject: Re: [Tei-council] Jenkins builds failing and not failing I've figured out where mine is failing: it's on this bit of antbuilder.xml, which was recently changed by Syd and commented accordingly: <!-- These next 2 are XSLT 1, which will not run in SaxonHE 9.8.0.5, which is what we are currently running. Thus I have used `xsltproc` here. We could, alternatively, update to SaxonHE 9.8.0.7 which hsa re-instated XSLT 1.0 compatability mode. —Syd, 2020-11-28 --> <exec executable="xsltproc" failonerror="true"> <arg line="--output p5odds.included.isosch"/> <arg line="Utilities/iso_dsdl_include.xsl"/> <arg line="p5odds.isosch"/> </exec> The failure looks like this: xsltproc --output p5odds.included.isosch Utilities/iso_dsdl_include.xsl p5odds.isosch runtime error: file Utilities/iso_dsdl_include.xsl line 1433 element value-of xsltApplySequenceConstructor: A potential infinite template recursion was detected. I think this was caused by Syd's commit in November, but I'm wondering if the fact that he didn't see the same error means that we have differing versions of xsltproc on our systems; my desktop and servers are running Ubuntu 20.04, and xsltproc --version gives me: xsltproc --version Using libxml 20910, libxslt 10134 and libexslt 820 xsltproc was compiled against libxml 20910, libxslt 10134 and libexslt 820 libxslt 10134 was compiled against libxml 20910 libexslt 820 was compiled against libxml 20910 Syd, what do you have? Peter, what's running on your server? Cheers, Martin On 2021-02-01 9:49 a.m., Martin Holmes wrote:

Hi all,

If you look at Peter's Jenkins, you'll see that the TEI-P5-Test-dev job appears to succeed (it's green). However, the console log for the latest build (653) shows 14 errors:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjenkins.te...

On my Jenkins, the latest build (547) also fails, but differently:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjenkins2.t...

I'm going to try to figure out why my build is failing, but I'm even more puzzled by the main Jenkins, which seems to be giving the impression that all is well. Peter, do you know what's happening here? NVDL errors seem to be ignored.

Cheers, Martin

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lists...

I'm seeing the same thing -- my job config had "mark build failed on error" unchecked. I've checked it now, but that's a weird thing, eh? The last changes to the job config in git were in 2018. Cheers, Martin On 2021-02-01 10:27 a.m., Peter Stadler wrote:

Here’s my `xsltproc --version` output: Using libxml 20904, libxslt 10129 and libexslt 817 xsltproc was compiled against libxml 20904, libxslt 10129 and libexslt 817 libxslt 10129 was compiled against libxml 20904 libexslt 817 was compiled against libxml 20904

The main Jenkins wasn’t failing because (I believe) the "Mark build Failed on Error“ checkbox was not checked. That’s somehow strange (that this surfaced only now) because that has never been set, see https://github.com/TEIC/Jenkins/blob/22793db166503d1b0363ce86910f495cea0b091... But maybe I’m on the wrong track?

Yet, the main issue seems to be the "potential infinite template recursion“. Should we increase the maxdepth parameter in the antbuilder.xml file (only for this call)? I think I’d prefer this instead of a global change to the build system but that’s just my 2c

Best Peter

...

Am 01.02.2021 um 19:12 schrieb Bauman, Syd :

I very recently started having to increase the value of --maxdepth on my system when running that ant task. I think (but am not at all sure) it is because of my recent upgrade to a new OS, which involved a new version of libxslt and xsltproc, which maybe has a lower default value of maxdepth? Anyway, I set it to 10000 in one branch or PR somewhere, and that worked.

(The maxdepth is the number of recursive template call allowed before presumed infinite and thus failure. The Schematron skeleton XSLT stylesheet now used parses the values of XPaths a character at a time using a recursive template. They (the XPaths being parsed) can get very long, even moreso given that it does not perform whitespace normalization beforehand. I do not know what the default maxdepth used to be, but it is 3000 on my system now, and I tried 10000 and that worked.)

From: Tei-council on behalf of Martin Holmes Sent: Monday, February 1, 2021 13:01 To: tei-council@lists.tei-c.org Subject: Re: [Tei-council] Jenkins builds failing and not failing

I've figured out where mine is failing: it's on this bit of antbuilder.xml, which was recently changed by Syd and commented accordingly:

<!-- These next 2 are XSLT 1, which will not run in SaxonHE 9.8.0.5, which is what we are currently running. Thus I have used `xsltproc` here. We could, alternatively, update to SaxonHE 9.8.0.7 which hsa re-instated XSLT 1.0 compatability mode. —Syd, 2020-11-28 --> <exec executable="xsltproc" failonerror="true"> <arg line="--output p5odds.included.isosch"/> <arg line="Utilities/iso_dsdl_include.xsl"/> <arg line="p5odds.isosch"/> </exec>

The failure looks like this:

xsltproc --output p5odds.included.isosch Utilities/iso_dsdl_include.xsl p5odds.isosch runtime error: file Utilities/iso_dsdl_include.xsl line 1433 element value-of xsltApplySequenceConstructor: A potential infinite template recursion was detected.

I think this was caused by Syd's commit in November, but I'm wondering if the fact that he didn't see the same error means that we have differing versions of xsltproc on our systems; my desktop and servers are running Ubuntu 20.04, and xsltproc --version gives me:

xsltproc --version

Using libxml 20910, libxslt 10134 and libexslt 820 xsltproc was compiled against libxml 20910, libxslt 10134 and libexslt 820 libxslt 10134 was compiled against libxml 20910 libexslt 820 was compiled against libxml 20910

Syd, what do you have? Peter, what's running on your server?

Cheers, Martin

On 2021-02-01 9:49 a.m., Martin Holmes wrote:

...

Hi all,

If you look at Peter's Jenkins, you'll see that the TEI-P5-Test-dev job appears to succeed (it's green). However, the console log for the latest build (653) shows 14 errors:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjenkins.te...

On my Jenkins, the latest build (547) also fails, but differently:

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjenkins2.t...

I'm going to try to figure out why my build is failing, but I'm even more puzzled by the main Jenkins, which seems to be giving the impression that all is well. Peter, do you know what's happening here? NVDL errors seem to be ignored.

Cheers, Martin

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lists... _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre

Am 01.02.2021 um 19:33 schrieb Martin Holmes :

Are these irrelevant distractions? They aren't suppressed in the tei-log-parse-rules file in the Jenkins repo.

I don’t know. But I think in general it makes sense to fail the build when there are errors. Which I’ve just checked and so the build is marked as failed. I see that your Jenkins is happily going on with the build, so probably you haven’t checked this box? If those errors are indeed non-brainers we should add it to the log-parse-rules Peter

So, one more thing I just discovered (for myself): This checkbox "Mark build Failed on Error“ does only apply to the log parsing. Usually, the ant task would fail, and the build would be flagged as failed by Jenkins. In this case, the ant script just goes on and we only find the errors in the log file. So, it might not be bad to check this box but ideally the ant task should already fail. Best Peter

Am 01.02.2021 um 19:43 schrieb Martin Holmes :

They don't look like no-brainers to me. O Schematron Gurus, what do you make of these?

Error: unrecognized element in ISO Schematron namespace: check spelling and capitalizationparam Schema implementation error: This schema has abstract patterns, yet they are supposed to be preprocessed out already

Cheers, Martin

On 2021-02-01 10:38 a.m., Peter Stadler wrote:

...

...

Am 01.02.2021 um 19:33 schrieb Martin Holmes :

Are these irrelevant distractions? They aren't suppressed in the tei-log-parse-rules file in the Jenkins repo. I don’t know. But I think in general it makes sense to fail the build when there are errors. Which I’ve just checked and so the build is marked as failed. I see that your Jenkins is happily going on with the build, so probably you haven’t checked this box? If those errors are indeed non-brainers we should add it to the log-parse-rules Peter

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre

The errors we're seeing now, though, are from the Makefile; the nvdl stuff is run at line 191 and 192 of the Makefile, without any ant involvement. You can set failonerror on an ant call to exec, but the log parser is the only thing that will catch the Make stuff, I think. I notice that the nvdl schema includes the old Schematron namespace: <!-- foreign namespaces used in TEI --> <namespace ns="http://www.ascc.net/xml/schematron"> <validate schema="schematron.rng"/> <attach useMode="allow"/> </namespace> <namespace ns="http://purl.oclc.org/dsdl/schematron"> <validate schema="iso-schematron.rng"/> <attach useMode="allow"/> </namespace> which we probably don't care about any more. I think it's a long time since anyone really looked at the nvdl. Cheers, Martin On 2021-02-01 10:47 a.m., Peter Stadler wrote:

So, one more thing I just discovered (for myself): This checkbox "Mark build Failed on Error“ does only apply to the log parsing. Usually, the ant task would fail, and the build would be flagged as failed by Jenkins. In this case, the ant script just goes on and we only find the errors in the log file.

So, it might not be bad to check this box but ideally the ant task should already fail.

Best Peter

...

Am 01.02.2021 um 19:43 schrieb Martin Holmes :

They don't look like no-brainers to me. O Schematron Gurus, what do you make of these?

Error: unrecognized element in ISO Schematron namespace: check spelling and capitalizationparam Schema implementation error: This schema has abstract patterns, yet they are supposed to be preprocessed out already

Cheers, Martin

On 2021-02-01 10:38 a.m., Peter Stadler wrote:

...

...

Am 01.02.2021 um 19:33 schrieb Martin Holmes :

Are these irrelevant distractions? They aren't suppressed in the tei-log-parse-rules file in the Jenkins repo. I don’t know. But I think in general it makes sense to fail the build when there are errors. Which I’ve just checked and so the build is marked as failed. I see that your Jenkins is happily going on with the build, so probably you haven’t checked this box? If those errors are indeed non-brainers we should add it to the log-parse-rules Peter

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre

Hi Syd, It seems the setup on your machine is different from the norm, since you have onvdl mapped to the oxygen-patched jing; the run-onvdl script goes to download it from SourceForge (since Oxygen stopped supporting its own jar): http://downloads.sourceforge.net/project/onvdl/onvdl/20070517/onvdl-20070517... So it looks like the old implementation of nvdl shows one set of errors, and the current support rolled into jing does something different -- it looks like it just fails to run, as far as I can see. Either way, nvdl validation is not getting us anything useful at this point, surely? Cheers, Martin On 2021-02-01 11:46 a.m., Bauman, Syd wrote:

But the error I am actually getting looks more sinister than an unused namespace. I get the following on every call to onvdl, followed by a large java traceback:

./run-onvdl p5valid.nvdl v.xml /home/syd/bin/onvdl DEBUG: executing 'java -Xss8m -Xmx4g -jar /opt/Oxygen_XML_Editor_23/lib/oxygen-patched-jing.jar p5valid.nvdl v.xml' WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.thaiopensource.validate.schematron.ISOSchemaReaderImpl (file:/opt/Oxygen_XML_Editor_23/lib/oxygen-patched-jing.jar) to constructor com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl() WARNING: Please consider reporting this to the maintainers of com.thaiopensource.validate.schematron.ISOSchemaReaderImpl WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/xerces/impl/locator/AttributeLocator at com.thaiopensource.relaxng.pattern.IdSoundnessChecker.idref(IdSoundnessChecker.java:165)

That said, at first glance everything seems to have actually built OK.

------------------------------------------------------------------------ The errors we're seeing now, though, are from the Makefile; the nvdl stuff is run at line 191 and 192 of the Makefile, without any ant involvement. You can set failonerror on an ant call to exec, but the log parser is the only thing that will catch the Make stuff, I think.

I notice that the nvdl schema includes the old Schematron namespace:

<!-- foreign namespaces used in TEI -->      <namespace ns="https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ascc.net%2Fxml%2Fschematron&amp;data=04%7C01%7Cs.bauman%40northeastern.edu%7C2adddfbe2c454fcb34fa08d8c6e37928%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637478027897637837%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=VqPfXBMWFG4pahsArXGQnF0XJzJ%2BuD4reVMtsxbo1zA%3D&amp;reserved=0">        <validate schema="schematron.rng"/>        <attach useMode="allow"/>      </namespace>

     <namespace ns="https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpurl.oclc.org%2Fdsdl%2Fschematron&amp;data=04%7C01%7Cs.bauman%40northeastern.edu%7C2adddfbe2c454fcb34fa08d8c6e37928%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637478027897637837%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=7mDV8RxTIRDP388MuzMU4Nw%2BIIOYJDifHbuDfjUHP3Y%3D&amp;reserved=0">        <validate schema="iso-schematron.rng"/>        <attach useMode="allow"/>      </namespace>

which we probably don't care about any more. I think it's a long time since anyone really looked at the nvdl.

-- ------------------------------------------ Martin Holmes UVic Humanities Computing and Media Centre