I have always assumed that tests intended to fail should be in detest, not in the other tests. But to be honest I think the whole set of tests was assembled in a piecemeal manner without much of a guiding plan, so I wouldn't be surprised to discover that there are intentional invalidities elsewhere. But in this particular case, I see the problem. If you look at the parsed log of the last successful build (http://teijenkins.hcmc.uvic.ca/job/TEIP5-Test-dev/2848/parsed_console/), you'll see this: validateschematron: [echo] Validate using Schematron [xslt] Processing /var/lib/jenkins/jobs/TEIP5-Test-dev/workspace/P5/Test/testbasic.xml to /dev/null [xslt] Loading stylesheet /var/lib/jenkins/jobs/TEIP5-Test-dev/workspace/P5/Test/testbasic.xsl [xslt] A feature value cannot contain both text and element content (tei:* and text()[normalize-space(.) ne '']) [xslt] A feature value can contain only one child element (count(tei:*) gt 1) In other words, these tests DO fail the Schematron check; but because the Schematron error message does not contain the trigger word ERROR, the build is not marked as having failed. In other words, the wrong test is being done in the wrong place the the results are being ignored. Two takeaways from this: 1. All Schematron rules we care about should have "ERROR" in their error messages, otherwise Jenkins is not going to notice them. If that's deemed too scary for end users, we need to find another trigger string that's not so bad, and then configure the Jenkins log parser to pay attention to it. 2. We need to go through all the tests and make sure they're actually working properly. There could be many more examples of this. Cheers, Martin On 2016-05-21 03:16 PM, Syd Bauman wrote:
The Jenkins build of P5 is failing because of the following test in .../P5/Test/testbasic.xml:
<fLib xmlns="http://www.tei-c.org/ns/1.0"> <f name="xxx">A feature may have untyped content</f> <f name="yyy"> <string>or typed</string> </f> <f name="notgood"> <string>multiple types</string> <symbol value="doubleplusungood"/> </f> <f name="alsonotgood">mixed content <symbol value="doubleplusungood"/> </f> </fLib>
I think these are tests that are *supposed* to fail, and thus should be in the detest/ suite, not here in testbasic.xml.
1) Can someone (Martin?) affirm for me that testbasic.xml is supposed to be valid?
2) Can someone (Laurent?, Lou?, Piotr?) affirm for me that an <f> is supposed to have at most 1 child element (from class model.featureVal, i.e. one of <fs>, <vColl>, <vMerge>, <vNot>, <binary>, <default>, <numeric>, <string>, <symbol>, <vAlt>, or <vLabel>)?
If I've got this right, I'll just move these tests out of testbasic, and we'll be up and running again.