That would be a convincing argument if <app> were in any way suitable for encoding printed apparatus. I thought we’d decided that <note type="app"> was the solution for that? I’m pretty convinced <app> was never intended for that purpose and is unsuited for it, even with the current, messed-up content model. I can be persuaded that longstanding brokenness is just something we have to live with though. Solution #2 would be either to directly permit <app> where it can occur now, or to create a new model class for it and include that. It’s not hard, just sort of verbose. If people are convinced that tightening up <app>’s content model is unwise, I’ll do #2.

On Oct 13, 2015, at 8:43 , Lou Burnard wrote:

The trouble with your first solution is that it's likely to break things quite badly for people using <app> to record an existing apparatus from a printed source, for whom the availability of model.global elements like <pb/> or <lb/> or even <gap/> within <app> is actually quite reasonable and not "nuts" at all. You might prefer it if <app> became a purely data-only construct, but that's really quite a shift from its current model. So I don't think coming up with such a radically different content model for it counts as a simple bug fix. The argument "should never have been allowed in the first place" is a very dangerous one.

How would your solution (2) work exactly?

On 13/10/15 13:20, Hugh Cayless wrote:

...

I think we’re going to have to do a point release. Most of the Stylesheets errors are fixable just by regenerating the expected results, but the tests revealed a bug via Odd by Example, which won’t produce a valid XSD content model for <app>. The short version is, that by adding <app> to model.global.edit, <app>’s own rather messed up content model causes an invalid XSD to be generated (it permits model.global inside it). There are two possible fixes for this:

1) Fix <app>’s content model, which I think is actually seriously incorrect. It should be more like the content model of <choice>, since <app> contains either alternates (<lem>, <rdg>s, and <rdgGrp>s) or notes (<note>, <wit>, <witDetail>). It shouldn’t directly contain a <gap>, for example. That’s actually more than a little nuts.

2) Add a model.appLike (or similar) and allow it almost anywhere.

I’d prefer #1, because it fixes what I see as a bug. This would invalidate content like <app><lem>foo</lem><gap/><rdg>bar></rdg></app>, but I would argue very strongly that this should never have been allowed in the first place, and I can’t see how it would make any sense. I’d be astonished if anyone was actually doing this in the wild.

I also think this should have showed up way earlier, and the reason it didn’t is that the Stylesheets tests are pegged by default to the current release, so Jenkins is not checking them against the contents of the master branch. I do see the value of having them test against the current release in terms of pure Stylesheets development, but it’s a serious flaw for any work that should happen in tandem. I’d like to suggest that we add a second build of the Stylesheets in Jenkins that runs its tests against the last build of TEIP5. That would a) alert us immediately if tests need to be regenerated, and b) flag things like this Odd By Example bug early on.

Thoughts? I can implement either fix fairly quickly. And then do a 2.9.1 release.

Hugh

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

While I wish app's content model had always been the data-like construct that Hugh imagines, I think I'm with Lou in thinking that said ship has sailed. I'd vote for implementing #2 and seeing what breaks then. (before releasing). ;-) -James On 13/10/15 14:09, Hugh Cayless wrote:

That would be a convincing argument if <app> were in any way suitable for encoding printed apparatus. I thought we’d decided that <note type="app"> was the solution for that? I’m pretty convinced <app> was never intended for that purpose and is unsuited for it, even with the current, messed-up content model. I can be persuaded that longstanding brokenness is just something we have to live with though.

Solution #2 would be either to directly permit <app> where it can occur now, or to create a new model class for it and include that. It’s not hard, just sort of verbose. If people are convinced that tightening up <app>’s content model is unwise, I’ll do #2.

...

On Oct 13, 2015, at 8:43 , Lou Burnard wrote:

The trouble with your first solution is that it's likely to break things quite badly for people using <app> to record an existing apparatus from a printed source, for whom the availability of model.global elements like <pb/> or <lb/> or even <gap/> within <app> is actually quite reasonable and not "nuts" at all. You might prefer it if <app> became a purely data-only construct, but that's really quite a shift from its current model. So I don't think coming up with such a radically different content model for it counts as a simple bug fix. The argument "should never have been allowed in the first place" is a very dangerous one.

How would your solution (2) work exactly?

On 13/10/15 13:20, Hugh Cayless wrote:

...

I think we’re going to have to do a point release. Most of the Stylesheets errors are fixable just by regenerating the expected results, but the tests revealed a bug via Odd by Example, which won’t produce a valid XSD content model for <app>. The short version is, that by adding <app> to model.global.edit, <app>’s own rather messed up content model causes an invalid XSD to be generated (it permits model.global inside it). There are two possible fixes for this:

1) Fix <app>’s content model, which I think is actually seriously incorrect. It should be more like the content model of <choice>, since <app> contains either alternates (<lem>, <rdg>s, and <rdgGrp>s) or notes (<note>, <wit>, <witDetail>). It shouldn’t directly contain a <gap>, for example. That’s actually more than a little nuts.

2) Add a model.appLike (or similar) and allow it almost anywhere.

I’d prefer #1, because it fixes what I see as a bug. This would invalidate content like <app><lem>foo</lem><gap/><rdg>bar></rdg></app>, but I would argue very strongly that this should never have been allowed in the first place, and I can’t see how it would make any sense. I’d be astonished if anyone was actually doing this in the wild.

I also think this should have showed up way earlier, and the reason it didn’t is that the Stylesheets tests are pegged by default to the current release, so Jenkins is not checking them against the contents of the master branch. I do see the value of having them test against the current release in terms of pure Stylesheets development, but it’s a serious flaw for any work that should happen in tandem. I’d like to suggest that we add a second build of the Stylesheets in Jenkins that runs its tests against the last build of TEIP5. That would a) alert us immediately if tests need to be regenerated, and b) flag things like this Odd By Example bug early on.

Thoughts? I can implement either fix fairly quickly. And then do a 2.9.1 release.

Hugh

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- Dr James Cummings, James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

So I'm not sure the ship has really sailed. Also we've introduced important changes to <rdg> in this release already, so this fix seems small and adequate.

Yes but the changes in <rdg> and <lem> are more permissive while the change n solution one would be restrictive. I do not think that having a look at the examples is enough to draw conclusion on real usage in the community, so either we start a small survey to see if solution 1 (that for me is actually better) doesn't break validity of real existent collections or editions or go with solution 2. f

After playing around with this a little, I think a less intrusive fix is to make sure <app>’s content model can’t contain <app>, which is what’s causing the Odd by Example issue. This unfortunately means blowing up <app>’s content model, but it fixes the problem, and will perhaps make it easier to trim it down later on. The downside is a *really* ugly content model. I’ve also fixed all the Stylesheets tests, as well as a bug I found along the way, and I’ve made it so that P5 version isn’t significant for comparing test output to expected results—meaning just changing the version number won’t break everything. I’m going to try to get this all sorted out tomorrow and aim for a 2.9.1 release, probably on Thursday.

On Oct 13, 2015, at 10:49 , Hugh Cayless wrote:

Yeah, I think it’s a fixable bug, but we ought to do due diligence before whacking it. The need for a fix is urgent though, so I will implement something like model.appLike in a way that we can easily yank it back out if we decide on #1.

...

On Oct 13, 2015, at 10:30 , Fabio Ciotti wrote:

...

So I'm not sure the ship has really sailed. Also we've introduced important changes to <rdg> in this release already, so this fix seems small and adequate.

Yes but the changes in <rdg> and <lem> are more permissive while the change n solution one would be restrictive. I do not think that having a look at the examples is enough to draw conclusion on real usage in the community, so either we start a small survey to see if solution 1 (that for me is actually better) doesn't break validity of real existent collections or editions or go with solution 2.

f -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Yeah, but that means also putting it back in a bunch of places, and I decided after a while of trying it that there was too high a risk of my making a mistake. I just pushed my change, so we’ll see if it builds ok.

On Oct 14, 2015, at 5:40 , Lou Burnard wrote:

Or you could remove app from model.global, of course. I thought that was what your second choice implied?

On 14/10/15 03:06, Hugh Cayless wrote:

...

After playing around with this a little, I think a less intrusive fix is to make sure <app>’s content model can’t contain <app>, which is what’s causing the Odd by Example issue. This unfortunately means blowing up <app>’s content model, but it fixes the problem, and will perhaps make it easier to trim it down later on. The downside is a *really* ugly content model.

I’ve also fixed all the Stylesheets tests, as well as a bug I found along the way, and I’ve made it so that P5 version isn’t significant for comparing test output to expected results—meaning just changing the version number won’t break everything. I’m going to try to get this all sorted out tomorrow and aim for a 2.9.1 release, probably on Thursday.

...

On Oct 13, 2015, at 10:49 , Hugh Cayless wrote:

Yeah, I think it’s a fixable bug, but we ought to do due diligence before whacking it. The need for a fix is urgent though, so I will implement something like model.appLike in a way that we can easily yank it back out if we decide on #1.

...

On Oct 13, 2015, at 10:30 , Fabio Ciotti wrote:

...

So I'm not sure the ship has really sailed. Also we've introduced important changes to <rdg> in this release already, so this fix seems small and adequate.

Yes but the changes in <rdg> and <lem> are more permissive while the change n solution one would be restrictive. I do not think that having a look at the examples is enough to draw conclusion on real usage in the community, so either we start a small survey to see if solution 1 (that for me is actually better) doesn't break validity of real existent collections or editions or go with solution 2.

f -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

I have opened a ticket, err, issue (#1345) which I think addresses the underlying problem here. It's often the case that when something breaks one or other of our processors it's because of an underlying problem which he haven't correctly identified yet. I think that's the case here. We started out down this particular road because we wanted to address concerns about the content model of <app>'s children, and in so doing have uncovered another related issue. So let's try to do the job properly, rather than get panicked into ad hoc short term fixes. Although I originally argued against Hugh's proposed solution #1 (fix the content model of <app>), like our esteemed shadow chancellor I have now changed my mind. (OBSCURE BRITISH POLITICAL REFERENCE NEVER MIND) On 15/10/15 03:54, Syd Bauman wrote:

I'm still trying to grok this all, but Hugh, could you explain why the need for a fix is urgent? This only breaks XSD, and only XSD that uses <app>, right? (All those epigraphers and manuscripters who want <app> are smart enough to use RNG, aren't they?)

I'm not suggesting that we purposefully dally, but I'm wondering if we truly need to be stressed.

...

Yeah, I think it’s a fixable bug, but we ought to do due diligence before whacking it. The need for a fix is urgent though, so I will implement something like model.appLike in a way that we can easily yank it back out if we decide on #1.

I don't think I have any objections. You are right that the content model looks icky. -James On 15/10/15 13:12, Hugh Cayless wrote:

I’m not panicked, but I would like to stabilize the 2.9.0 release, and the Stylesheets build. The latter can currently *only* be done via a new release, so I would like to do a 9.4.1 release today, and then we can plan to do a nicer fix in the next release (which you’ll remember we planned to do before year’s end). https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5... https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5..., is, I’ll freely admit, hideous, but it does the job of permitting everything currently permitted inside <app> except <app>. I’d like to run with this, fix the release and the Stylesheets, and get everything back on track. Any objections?

...

On Oct 15, 2015, at 6:18 , Lou Burnard wrote:

I have opened a ticket, err, issue (#1345) which I think addresses the underlying problem here.

It's often the case that when something breaks one or other of our processors it's because of an underlying problem which he haven't correctly identified yet. I think that's the case here. We started out down this particular road because we wanted to address concerns about the content model of <app>'s children, and in so doing have uncovered another related issue.

So let's try to do the job properly, rather than get panicked into ad hoc short term fixes. Although I originally argued against Hugh's proposed solution #1 (fix the content model of <app>), like our esteemed shadow chancellor I have now changed my mind. (OBSCURE BRITISH POLITICAL REFERENCE NEVER MIND)

On 15/10/15 03:54, Syd Bauman wrote:

...

I'm still trying to grok this all, but Hugh, could you explain why the need for a fix is urgent? This only breaks XSD, and only XSD that uses <app>, right? (All those epigraphers and manuscripters who want <app> are smart enough to use RNG, aren't they?)

I'm not suggesting that we purposefully dally, but I'm wondering if we truly need to be stressed.

...

Yeah, I think it’s a fixable bug, but we ought to do due diligence before whacking it. The need for a fix is urgent though, so I will implement something like model.appLike in a way that we can easily yank it back out if we decide on #1. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- Dr James Cummings, James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

Thanks James. In the absence of any screams of "NO DON’T!!", I’m going to go ahead with the point release and then fixing the Stylesheets build. Please don’t push to the TEI or Stylesheets repos while 2.9.1 is in progress.

On Oct 15, 2015, at 11:25 , James Cummings wrote:

I don't think I have any objections. You are right that the content model looks icky.

-James

On 15/10/15 13:12, Hugh Cayless wrote:

...

I’m not panicked, but I would like to stabilize the 2.9.0 release, and the Stylesheets build. The latter can currently *only* be done via a new release, so I would like to do a 9.4.1 release today, and then we can plan to do a nicer fix in the next release (which you’ll remember we planned to do before year’s end). https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5... https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5...<https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5... https://github.com/TEIC/TEI/blob/18fe150c5cc7d46c871e2dfd32c98faad74ca55c/P5...>, is, I’ll freely admit, hideous, but it does the job of permitting everything currently permitted inside <app> except <app>. I’d like to run with this, fix the release and the Stylesheets, and get everything back on track. Any objections?

...

On Oct 15, 2015, at 6:18 , Lou Burnard wrote:

I have opened a ticket, err, issue (#1345) which I think addresses the underlying problem here.

It's often the case that when something breaks one or other of our processors it's because of an underlying problem which he haven't correctly identified yet. I think that's the case here. We started out down this particular road because we wanted to address concerns about the content model of <app>'s children, and in so doing have uncovered another related issue.

So let's try to do the job properly, rather than get panicked into ad hoc short term fixes. Although I originally argued against Hugh's proposed solution #1 (fix the content model of <app>), like our esteemed shadow chancellor I have now changed my mind. (OBSCURE BRITISH POLITICAL REFERENCE NEVER MIND)

On 15/10/15 03:54, Syd Bauman wrote:

...

I'm still trying to grok this all, but Hugh, could you explain why the need for a fix is urgent? This only breaks XSD, and only XSD that uses <app>, right? (All those epigraphers and manuscripters who want <app> are smart enough to use RNG, aren't they?)

I'm not suggesting that we purposefully dally, but I'm wondering if we truly need to be stressed.

...

Yeah, I think it’s a fixable bug, but we ought to do due diligence before whacking it. The need for a fix is urgent though, so I will implement something like model.appLike in a way that we can easily yank it back out if we decide on #1. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- Dr James Cummings, James.Cummings@it.ox.ac.uk mailto:James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford -- tei-council mailing list tei-council@lists.tei-c.org mailto:tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

sorry to be asking another stupid question, but what is a "CI" ? I get the gist of the question and don't disagree, but I am puzzled by the jargon. Unless it;s a typo. On 13/10/15 16:10, Hugh Cayless wrote:

I also want to follow up on this: I think this is a REALLY SERIOUS problem. The stylesheets’ CI ought to flag issues with the current state of the Guidelines. Keeping up with some of the expected results stuff may be a small pain, but figuring out what bugs other people’s additions have caused is hard. I’d like to push that fixing process closer to the checkin, rather than waiting to deal with it after a release.

...

On Oct 13, 2015, at 8:20 , Hugh Cayless wrote:

I also think this should have showed up way earlier, and the reason it didn’t is that the Stylesheets tests are pegged by default to the current release, so Jenkins is not checking them against the contents of the master branch. I do see the value of having them test against the current release in terms of pure Stylesheets development, but it’s a serious flaw for any work that should happen in tandem. I’d like to suggest that we add a second build of the Stylesheets in Jenkins that runs its tests against the last build of TEIP5. That would a) alert us immediately if tests need to be regenerated, and b) flag things like this Odd By Example bug early on.