Re: [Tei-council] The problem with expected-results and p5subset

I got branch and master reversed. So if you're setting up a test for Stylesheets 7.49, and you want that specific test to use p5subset 3.7.0 (not yet released), it would: 1. Look on the tei-c site for 3.7.0. No joy; so 2. Look at the current Jenkins P5 master branch; if that version is not 3.7,0, then 3. Look on the current Jenkins for a P5 release branch labelled 3.7.0; if there's nothing there, then 4. Use the output from the current Jenkins P5-dev branch. I also think the idea that older tests, depending on older p5subsets, may still be run using those older p5subsets is a good idea. After all, we're encouraging people to tie their ODDs to a specific TEI version if they don't want unexpected changes, so we are expecting any new release of the Stylesheets to continue to work as expected with older versions of p5subset, unless we explicitly change the Stylesheets, in which case we need to examine the expected-results anyway, irrespective of p5subset version. Cheers, Martin On 2019-07-18 9:07 a.m., Martin Holmes wrote:

Hi Raff,

In that case, we ought to be implementing a specific test for that change, or integrating that test into one of the existing files. We might want to have a sort of fallback sequence for that scenario, where we specify the (upcoming) version of TEI that this should work with, but if that version isn't available on the TEI site, the test would check for output from a release branch of P5 with that number; failing that it would check the latest output from P5-master; if that doesn't match the version number, then it would default back to the latest output from the P5-dev source.

I know that's a bit complicated, but it should be write-once-run-anywhere, so as the Stylesheets and P5 source transitions from dev through release-branch to master, the Stylesheets should continue to find a p5subset that's appropriate for that test. Following the release of that version, of course, it would subsequently always find the source on the tei-c site.

Rather tortuous I know, but the current situation is way worse IMHO.

Cheers, Martin

On 2019-07-18 7:10 a.m., Raffaele Viglianti wrote:

...

Hi Martin,

This sounds like a good idea to me in principle: the Stylesheets tests should be meant to test changes to the Stylesheets and not be affected by changes to the source.

However, what do we do when we need to make sure that the Stylesheets are still behaving correctly after a major change to the source? In that case the Stylesheets tests are likely to fail because of changes in the source even if they are indeed behaving correctly.

Raff

Raff

On Wed, Jul 17, 2019 at 9:01 PM Martin Holmes mailto:mholmes@uvic.ca> wrote:

    Hi all,

    I think there's a possible partial solution to the chaos resulting from     the fact that Stylesheets tests depend on p5subset.xml, so whenever the     Specs change, the expected-results in the Stylesheets tests may also     change.

    This is because we always try to use the same p5subset for every test:     the current release. But if we make each test point at a specific     version of p5subset.xml in the Vault, then that version will never     change; so the Stylesheets output should also never change, unless we     change the Stylesheets themselves, in which case we want to see what     changes in the expected-results anyway.

    Anyone see any problem with this?

    Cheers,     Martin     _______________________________________________     Tei-council mailing list     Tei-council@lists.tei-c.org mailto:Tei-council@lists.tei-c.org     http://lists.lists.tei-c.org/mailman/listinfo/tei-council

_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council