circular build headache suggestion
The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously. This sucks. Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly. So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above. Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev".
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>:
The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously.
This sucks.
Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly.
So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above.
Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
HI Peter, The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. Cheers, Martin On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin,
I think I’m not yet convinced that the proposed setup will be any better. For several reasons:
a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization?
b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build.
Looking forward to the discussion :) Best Peter
[1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>:
The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously.
This sucks.
Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly.
So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above.
Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both.
If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said.
Best Peter
On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>: The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously. This sucks. Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly. So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above. Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Hi Peter, On 2020-06-15 8:34 a.m., Peter Stadler wrote:
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild
This means fixing six months' worth of piled-up changes right when you're ready to do a release, when there's no time to do it properly. You're just ensuring that people will pay hardly any attention to the differences because they're on a deadline. I think that's a terrible way to do it. Why not build Stylesheets-dev against P5-dev all the time, and fix those breakages carefully as we go along? There's no need for Stylesheets-dev ever to build against the current P5 release. Cheers, Martin
d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both.
If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said.
Best Peter
On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>: The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously. This sucks. Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly. So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above. Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca
Hi Martin, I think my assumption is that there are no (and should not be any) meaningful differences between testing the Stylesheets against GL-dev or against GL-release. NB, we’re only discussing testing the Stylesheets! These should work with more or less any P5 source and should assure us that we’re not introducing any regression. If we are implementing a new feature in the Guidelines which needs processing support in the Stylesheets we can (and should) add a test to the Stylesheets as a customization ODD. That said, I believe the "piled-up changes“ during release can safely be ignored or what (systematic) errors do you hope to spot by testing against the dev version? Best Peter
Am 15.06.2020 um 18:01 schrieb Martin Holmes <mholmes@uvic.ca>:
Hi Peter,
On 2020-06-15 8:34 a.m., Peter Stadler wrote:
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild
This means fixing six months' worth of piled-up changes right when you're ready to do a release, when there's no time to do it properly. You're just ensuring that people will pay hardly any attention to the differences because they're on a deadline. I think that's a terrible way to do it. Why not build Stylesheets-dev against P5-dev all the time, and fix those breakages carefully as we go along? There's no need for Stylesheets-dev ever to build against the current P5 release.
Cheers, Martin
d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both. If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said. Best Peter On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>: The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously. This sucks. Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly. So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above. Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Hi Peter, If you look at the current builds, you'll see that there's a significant difference in the Stylesheets Tests (they currently fail when building with P5-dev, but succeed against P5-release). This is because of changes to attribute classes. Syd and I fixed them to work against P5-dev, but I think you then fixed them back to work against P5-release. The point is that P5-dev and Stylesheets-dev simultaneously become P5-release and Stylesheets-release, and if they are broken against each other at that point, then someone has to do all those fixes on release day. Why do you need Stylesheets-dev Tests to succeed against P5-release? It would be nice, but it's less important than that the tests succeed against the version of P5 together with which they will be released. We should never have to expect to do last-minute bugfixing on release day. It just adds stress and more potential errors to a process which is complicated enough as it is. Cheers, Martin On 2020-06-15 9:25 a.m., Peter Stadler wrote:
Hi Martin,
I think my assumption is that there are no (and should not be any) meaningful differences between testing the Stylesheets against GL-dev or against GL-release. NB, we’re only discussing testing the Stylesheets! These should work with more or less any P5 source and should assure us that we’re not introducing any regression. If we are implementing a new feature in the Guidelines which needs processing support in the Stylesheets we can (and should) add a test to the Stylesheets as a customization ODD.
That said, I believe the "piled-up changes“ during release can safely be ignored or what (systematic) errors do you hope to spot by testing against the dev version?
Best Peter
Am 15.06.2020 um 18:01 schrieb Martin Holmes <mholmes@uvic.ca>:
Hi Peter,
On 2020-06-15 8:34 a.m., Peter Stadler wrote:
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild
This means fixing six months' worth of piled-up changes right when you're ready to do a release, when there's no time to do it properly. You're just ensuring that people will pay hardly any attention to the differences because they're on a deadline. I think that's a terrible way to do it. Why not build Stylesheets-dev against P5-dev all the time, and fix those breakages carefully as we go along? There's no need for Stylesheets-dev ever to build against the current P5 release.
Cheers, Martin
d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both. If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said. Best Peter On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
Am 12.06.2020 um 16:44 schrieb Syd Bauman <s.bauman@northeastern.edu>: The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now where the Stylesheets tests can succeed only against either the release branch of P5 or the dev branch of P5, but not both simultaneously. This sucks. Right now Stylesheets dev branch is successfully building against P5 release[1], but failing against P5 dev branch[2]. We believe that the reverse would make a lot more sense. While it is desirable that it build against both branches, it is frequently impossible. So we have to pick one, and we (Martin & Syd) think we should pick them as follows: * P5 release branch should be built with the P5 release Stylesheets * P5 dev branch should be built with the P5 dev Stylesheets * Stylesheets release should be built against P5 release branch * Stylesheets dev should be built against P5 release dev as a general rule. This means we would have to always release a new Stylesheets the same day we release P5 every time. It also means that if we ever had a reason we needed to do an interim release of either, we would have to fight with the circularity problem (probably by trying to make sure both branches work against each other, or temporarily suppressing some finicky tests). But the advantage is that "normal" releases and updates would work properly. So what we would like to do is: * Update Stylesheets deb branch tests so that they work against P5 deb (which will break them against P5 release branch) * Have Peter & Martin agree on a consistent set of explicit job names in Jenkins that perform the 4 builds listed above. Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca
I still believe testing dev against dev is nuts because then we introduce real circularity and would need to fix Stylesheets expected results with almost every commit to the Guidelines. Yet, there’s another option that came up during our Cologne(?) meeting: add p5subset.xml to the Stylesheets repo as a source for the Stylesheets tests. This would a) reduce external calls (see [1]), and b) provide a stable test setup, and c) could be changed independently from the Guidelines, and d) Stylesheets tests wouldn’t fail for the released version. We would only need to develop a policy when to modify this p5subset.xml (and hence the expected results). Before a release? After a release? Somewhere between? Anyway, I think this is a crucial issue and we should discuss this at our next Council meeting. Since this is only in four weeks, maybe we should schedule an extraordinary meeting? Best Peter [1] https://github.com/TEIC/Stylesheets/issues/347
Am 15.06.2020 um 18:37 schrieb Martin Holmes <mholmes@UVIC.CA>:
Hi Peter,
If you look at the current builds, you'll see that there's a significant difference in the Stylesheets Tests (they currently fail when building with P5-dev, but succeed against P5-release). This is because of changes to attribute classes. Syd and I fixed them to work against P5-dev, but I think you then fixed them back to work against P5-release.
The point is that P5-dev and Stylesheets-dev simultaneously become P5-release and Stylesheets-release, and if they are broken against each other at that point, then someone has to do all those fixes on release day. Why do you need Stylesheets-dev Tests to succeed against P5-release? It would be nice, but it's less important than that the tests succeed against the version of P5 together with which they will be released.
We should never have to expect to do last-minute bugfixing on release day. It just adds stress and more potential errors to a process which is complicated enough as it is.
Cheers, Martin
On 2020-06-15 9:25 a.m., Peter Stadler wrote:
Hi Martin, I think my assumption is that there are no (and should not be any) meaningful differences between testing the Stylesheets against GL-dev or against GL-release. NB, we’re only discussing testing the Stylesheets! These should work with more or less any P5 source and should assure us that we’re not introducing any regression. If we are implementing a new feature in the Guidelines which needs processing support in the Stylesheets we can (and should) add a test to the Stylesheets as a customization ODD. That said, I believe the "piled-up changes“ during release can safely be ignored or what (systematic) errors do you hope to spot by testing against the dev version? Best Peter
Am 15.06.2020 um 18:01 schrieb Martin Holmes <mholmes@uvic.ca>:
Hi Peter,
On 2020-06-15 8:34 a.m., Peter Stadler wrote:
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild
This means fixing six months' worth of piled-up changes right when you're ready to do a release, when there's no time to do it properly. You're just ensuring that people will pay hardly any attention to the differences because they're on a deadline. I think that's a terrible way to do it. Why not build Stylesheets-dev against P5-dev all the time, and fix those breakages carefully as we go along? There's no need for Stylesheets-dev ever to build against the current P5 release.
Cheers, Martin
d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both. If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said. Best Peter On 2020-06-15 2:42 a.m., Peter Stadler wrote:
Syd, Martin, I think I’m not yet convinced that the proposed setup will be any better. For several reasons: a) you say: „it sucks“ – but what exactly sucks? aa) That we can’t have both ("tests can succeed only against either the release branch of P5 or the dev branch of P5“)? You’re not fixing that but only switching the default from ’stylesheets-dev against tei-release’ to ’stylesheets-dev against tei-dev’ ab) … because you „believe that the reverse would make a lot more sense“: Why? Can you elaborate? In fact, is there any Stylesheet test that relies on a dev version of the P5 sources – which could not be rewritten as a customization? b) I admit that the current setup is by no means ideal but to all the circularity there’s a clear way out: The current released version of p5subset.xml (which lives outside any Jenkins) serves as a seed for the Stylesheets build which then enables the Guidelines build. ba) It’s always been like this. I only added the feature to set the localsource for the Stylesheets build last year [1] when we had trouble connecting to the TEI server and resorted to fetching p5subset.xml from an alternative location (= a Jenkins server). [I take it that this is a traditionalists argument …] bb) the proposed dev-against-dev testing only introduces real circularity, I fear. Because there might be changes to the Guidelines repo that break the Stylesheets build and vice versa. (Right now, we are only able to break the Guidelines build by a change to the Stylesheets.) The latter is much more predictable: If you’re changing the ODD processing there’s a good chance that the Guidelines build is affected. On the other hand–and even more so for external contributors–it’s totally incomprehensible that a change to some element description in the Guidelines repo might break the Stylesheets build. Looking forward to the discussion :) Best Peter [1] https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d...
> Am 12.06.2020 um 16:44 schrieb Syd Bauman > <s.bauman@northeastern.edu>: > The circularity of the P5 and Stylesheets builds depending on each other has bitten us once again. (Ouch!) We have a situation now > where the Stylesheets tests can succeed only against either the > release branch of P5 or the dev branch of P5, but not both > simultaneously. > This sucks. > Right now Stylesheets dev branch is successfully building against > P5 release[1], but failing against P5 dev branch[2]. We believe > that the reverse would make a lot more sense. While it is desirable > that it build against both branches, it is frequently impossible. > So we have to pick one, and we (Martin & Syd) think we should pick > them as follows: * P5 release branch should be built with the P5 > release Stylesheets * P5 dev branch should be built with the P5 dev > Stylesheets * Stylesheets release should be built against P5 > release branch * Stylesheets dev should be built against P5 release > dev as a general rule. This means we would have to always release a > new Stylesheets the same day we release P5 every time. It also > means that if we ever had a reason we needed to do an interim > release of either, we would have to fight with the circularity > problem (probably by trying to make sure both branches work against > each other, or temporarily suppressing some finicky tests). But the > advantage is that "normal" releases and updates would work > properly. > So what we would like to do is: * Update Stylesheets deb branch > tests so that they work against P5 deb (which will break them > against P5 release branch) * Have Peter & Martin agree on a > consistent set of explicit job names in Jenkins that perform the 4 > builds listed above. > Notes ----- [1] E.g. on jenkins.tei-c job "Stylesheets-dev" and > jenkins2.tei-c job "Stylesheets-dev-against-P5-release". [2] E.g. > on jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" and jenkins2.tei-c job "Stylesheets-dev-against-TEIP5-dev". _______________________________________________ Tei-council mailing > list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
HI Peter, On 2020-06-15 11:30 p.m., Peter Stadler wrote:
I still believe testing dev against dev is nuts because then we introduce real circularity and would need to fix Stylesheets expected results with almost every commit to the Guidelines.
That's exactly right, and that's exactly what should happen. The Stylesheets and the Guidelines move forward together; changes to either repo may require matching changes to the other. If you don't do this steadily as you go along, you have to do it all in one messy hack on release day. It makes no sense.
Yet, there’s another option that came up during our Cologne(?) meeting: add p5subset.xml to the Stylesheets repo as a source for the Stylesheets tests.
Which p5subset.xml, though? Imagine this fairly common scenario: - A new thing is added to the Glines (dev) which requires special processing in the Stylesheets. - We edit the Stylesheets to add this special processing. - That edit makes the Stylesheets tests fail, because their behaviour is different from before so expected-results has changed. - Someone fixes the expected-results so that they match the output of the Stylesheets when tested against the current P5 release. - On release day, suddenly the Stylesheets are being built/tested against what-was-P5-dev, and everything breaks. Mad panic, scramble around, fix expected-results without proper attention to detail, fail to catch actual error which was lurking but you didn't see it because you had to fix two dozen expected-results differences and it's release day. It's madness. I just don't understand how you could think this is a good idea. Contrast with: - A new thing is added to the Glines (dev) which requires special processing in the Stylesheets. - That edit makes the Stylesheets tests (against P5-dev) fail, because their behaviour is different from before so expected-results has changed. - Someone fixes the expected-results so that they match the output of the Stylesheets when tested against the P5-dev. - On release day everything just works because these two branches have been building against each other and have been updated carefully as we go along. I'm happy to have a meeting to discuss this, but if I'm wrong about any of this, I'd really like to hear from anyone and everyone correcting me. Is there anything wrong with my logic here?
This would a) reduce external calls
If we test against the results of the dev branch, then there's no external call; it pulls the local P5-dev branch results.
(see [1]), and b) provide a stable test setup, and c) could be changed independently from the Guidelines, and d) Stylesheets tests wouldn’t fail for the released version.
Yes they would, as soon as we release a new version, because then the released version is what-was-dev.
We would only need to develop a policy when to modify this p5subset.xml (and hence the expected results). Before a release? After a release? Somewhere between?
Exactly. It's nothing but confusion.
Anyway, I think this is a crucial issue and we should discuss this at our next Council meeting. Since this is only in four weeks, maybe we should schedule an extraordinary meeting?
Please let's do that. Cheers, Martin
Best Peter
[1] https://github.com/TEIC/Stylesheets/issues/347
Am 15.06.2020 um 18:37 schrieb Martin Holmes <mholmes@UVIC.CA>:
Hi Peter,
If you look at the current builds, you'll see that there's a significant difference in the Stylesheets Tests (they currently fail when building with P5-dev, but succeed against P5-release). This is because of changes to attribute classes. Syd and I fixed them to work against P5-dev, but I think you then fixed them back to work against P5-release.
The point is that P5-dev and Stylesheets-dev simultaneously become P5-release and Stylesheets-release, and if they are broken against each other at that point, then someone has to do all those fixes on release day. Why do you need Stylesheets-dev Tests to succeed against P5-release? It would be nice, but it's less important than that the tests succeed against the version of P5 together with which they will be released.
We should never have to expect to do last-minute bugfixing on release day. It just adds stress and more potential errors to a process which is complicated enough as it is.
Cheers, Martin
On 2020-06-15 9:25 a.m., Peter Stadler wrote:
Hi Martin, I think my assumption is that there are no (and should not be any) meaningful differences between testing the Stylesheets against GL-dev or against GL-release. NB, we’re only discussing testing the Stylesheets! These should work with more or less any P5 source and should assure us that we’re not introducing any regression. If we are implementing a new feature in the Guidelines which needs processing support in the Stylesheets we can (and should) add a test to the Stylesheets as a customization ODD. That said, I believe the "piled-up changes“ during release can safely be ignored or what (systematic) errors do you hope to spot by testing against the dev version? Best Peter
Am 15.06.2020 um 18:01 schrieb Martin Holmes <mholmes@uvic.ca>:
Hi Peter,
On 2020-06-15 8:34 a.m., Peter Stadler wrote:
comments inline …
Am 15.06.2020 um 16:30 schrieb Martin Holmes <mholmes@uvic.ca>:
HI Peter,
The reason we say it's most important that the Stylesheets-dev branch needs to build against the P5-dev branch is because they will be released together, and when that happens, they must work together. If we ignore that combination and prefer to focus on keeping the Stylesheets-dev branch building against the released p5subset.xml, we'll have a problem when we release, because the released p5subset will change. Yes, but I still think it’s better to fix the Stylesheets tests *once* in the release branch rather than every time you rephrase some description in the Guidelines. My proposal (which I never ‚officially‘ submitted) for doing this would look like this: a) create release branches for Guidelines and Stylesheets b) initially build Stylesheets-release-branch against Guidelines-dev-branch; this will fail c) fix expected results in Stylesheets-release-branch and rebuild
This means fixing six months' worth of piled-up changes right when you're ready to do a release, when there's no time to do it properly. You're just ensuring that people will pay hardly any attention to the differences because they're on a deadline. I think that's a terrible way to do it. Why not build Stylesheets-dev against P5-dev all the time, and fix those breakages carefully as we go along? There's no need for Stylesheets-dev ever to build against the current P5 release.
Cheers, Martin
d) build Guidelines-release-branch against Stylesheets-release-branch
Changes on either side may break the other build. If you change the content model of an element, or even its description, then the crude diffing tests in the Stylesheets Tests will fail (which is what's happening now); so the Stylesheets dev build can either have its expected-results matching what you get from the released p5subset.xml, or it can have them matching the p5subset.xml from the P5-dev branch, but not both. Yes, that’s the point and we all agree: you can’t have both. If you change the Stylesheets in any significant way, then it's likely the Glines build tests may fail because (e.g.) the HTML output has been improved so the expected-results are not the same. As I said. Best Peter On 2020-06-15 2:42 a.m., Peter Stadler wrote: > Syd, Martin, I think I’m not yet convinced that the > proposed setup will be any better. For several reasons: > a) you say: „it sucks“ – but what exactly sucks? aa) That > we can’t have both ("tests can succeed only against > either the release branch of P5 or the dev branch of > P5“)? You’re not fixing that but only switching the > default from ’stylesheets-dev against tei-release’ to > ’stylesheets-dev against tei-dev’ ab) … because you > „believe that the reverse would make a lot more sense“: > Why? Can you elaborate? In fact, is there any Stylesheet > test that relies on a dev version of the P5 sources – > which could not be rewritten as a customization? b) I > admit that the current setup is by no means ideal but to > all the circularity there’s a clear way out: The current > released version of p5subset.xml (which lives outside any > Jenkins) serves as a seed for the Stylesheets build which > then enables the Guidelines build. ba) It’s always been > like this. I only added the feature to set the > localsource for the Stylesheets build last year [1] when > we had trouble connecting to the TEI server and resorted > to fetching p5subset.xml from an alternative location (= > a Jenkins server). [I take it that this is a > traditionalists argument …] bb) the proposed > dev-against-dev testing only introduces real circularity, > I fear. Because there might be changes to the Guidelines > repo that break the Stylesheets build and vice versa. > (Right now, we are only able to break the Guidelines > build by a change to the Stylesheets.) The latter is much > more predictable: If you’re changing the ODD processing > there’s a good chance that the Guidelines build is > affected. On the other hand–and even more so for > external contributors–it’s totally incomprehensible that > a change to some element description in the Guidelines > repo might break the Stylesheets build. Looking forward > to the discussion :) Best Peter [1] > https://github.com/TEIC/Stylesheets/commit/5c8c0d67618b6916853dbb3626e72476d... > >> >
Am 12.06.2020 um 16:44 schrieb Syd Bauman
>> <s.bauman@northeastern.edu>: The circularity of the P5 >> and Stylesheets builds depending on each other has >> bitten us once again. (Ouch!) We have a situation now >> where the Stylesheets tests can succeed only against >> either the release branch of P5 or the dev branch of >> P5, but not both simultaneously. This sucks. Right now >> Stylesheets dev branch is successfully building >> against P5 release[1], but failing against P5 dev >> branch[2]. We believe that the reverse would make a lot >> more sense. While it is desirable that it build against >> both branches, it is frequently impossible. So we have >> to pick one, and we (Martin & Syd) think we should >> pick them as follows: * P5 release branch should be >> built with the P5 release Stylesheets * P5 dev branch >> should be built with the P5 dev Stylesheets * >> Stylesheets release should be built against P5 release >> branch * Stylesheets dev should be built against P5 >> release dev as a general rule. This means we would have >> to always release a new Stylesheets the same day we >> release P5 every time. It also means that if we ever >> had a reason we needed to do an interim release of >> either, we would have to fight with the circularity >> problem (probably by trying to make sure both branches >> work against each other, or temporarily suppressing >> some finicky tests). But the advantage is that "normal" >> releases and updates would work properly. So what we >> would like to do is: * Update Stylesheets deb branch >> tests so that they work against P5 deb (which will >> break them against P5 release branch) * Have Peter & >> Martin agree on a consistent set of explicit job names >> in Jenkins that perform the 4 builds listed above. >> Notes ----- [1] E.g. on jenkins.tei-c job >> "Stylesheets-dev" and jenkins2.tei-c job >> "Stylesheets-dev-against-P5-release". [2] E.g. on >> jenkins.tei-c job "Stylesheets-dev-against-TEIP5-dev" >> and jenkins2.tei-c job >> "Stylesheets-dev-against-TEIP5-dev". >> _______________________________________________ >> Tei-council mailing list Tei-council@lists.tei-c.org >> http://lists.lists.tei-c.org/mailman/listinfo/tei-council > >> _______________________________________________ Tei-council mailing > list Tei-council@lists.tei-c.org > http://lists.lists.tei-c.org/mailman/listinfo/tei-council
> -- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca _______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
_______________________________________________ Tei-council mailing list Tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council
-- ------------------------------------- Humanities Computing and Media Centre University of Victoria mholmes@uvic.ca
participants (3)
-
Martin Holmes -
Peter Stadler -
Syd Bauman