I get a stack of errors in P5.xml, culminating in these: /home/mholmes/temp/p5.xml:119462:85: error: text not allowed here /home/mholmes/temp/p5.xml:119463:14: error: text not allowed here /home/mholmes/temp/p5.xml:119463:33: error: text not allowed here /home/mholmes/temp/p5.xml:119463:54: error: text not allowed here /home/mholmes/temp/p5.xml:119463:75: error: text not allowed here /home/mholmes/temp/p5.xml:119463:98: error: text not allowed here /home/mholmes/temp/p5.xml:119474:59: error: text not allowed here /home/mholmes/temp/p5.xml:119475:35: error: text not allowed here /home/mholmes/temp/p5.xml:119482:22: error: text not allowed here Cheers, Martin On 15-08-07 11:53 AM, Lou Burnard wrote:

I think maybe the merge didnt work as I expected: anyway, I've added model.eventLike to the P5-Pure branch, so you should get it if you svn up

On 07/08/15 19:12, Martin Holmes wrote:

...

Just tried it, and there seems to be a file missing in that branch:

BUILD FAILED /home/mholmes/temp/antbuilder.xml:39: Fatal error during transformation using /home/mholmes/temp/Utilities/expand.xsl: I/O error reported by XML parser processing file:/home/mholmes/temp/Source/Specs/model.eventLike.xml: /home/mholmes/temp/Source/Specs/model.eventLike.xml (No such file or directory); SystemID: file:/home/mholmes/temp/Utilities/expand.xsl; Line#: 68; Column#: -1

Cheers, Martin

On 15-08-07 10:51 AM, Lou Burnard wrote:

...

I've now managed to get the P5-Pure branch of the P5 source up to date with the rest of the world, and have successfully run the P5 test suite (almost) to completion. *It would be very helpful if someone else could check this branch out and confirm that it works for them too!*

The main difference is that all datatypes are now expressed using the new dataSpec and dataRef elements we discussed approx 3 million years ago. This in turn means that the old data.foo macroSpecs are now redundant, (we now use dataSpecs called teidata.foo). I have however left the old macroSpecs in the build, since their absence obviously causes existing ODDs that reference them to fall over. It's a pain however, and I would really like to get data.foo deprecated as soon as possible. Only a few (like 2 ore 3) of our test ODDs actually refer to them, but I've no way of telling how many ODDs out there in the wild do, nor how upsetting their maintainers would find it to be told to turn into <dataRef key="teidata.foo"/> passim et seriatim

(If you're wondering, the "(almost)" above means that my poor little laptop gave up trying to compile the massive tei_allPlus.odd.processed into tei_allPlus.rng ... probably the heat.)

Huzzah! Did you also run all the test suite? On 07/08/15 20:37, Syd Bauman wrote:

Builds for me. I get no such errors; and there is no line 119462 column 85 in my p5.xml. From what part of the build process do these errors come?

...

I get a stack of errors in P5.xml, culminating in these:

/home/mholmes/temp/p5.xml:119462:85: error: text not allowed here /home/mholmes/temp/p5.xml:119463:14: error: text not allowed here /home/mholmes/temp/p5.xml:119463:33: error: text not allowed here /home/mholmes/temp/p5.xml:119463:54: error: text not allowed here /home/mholmes/temp/p5.xml:119463:75: error: text not allowed here /home/mholmes/temp/p5.xml:119463:98: error: text not allowed here /home/mholmes/temp/p5.xml:119474:59: error: text not allowed here /home/mholmes/temp/p5.xml:119475:35: error: text not allowed here /home/mholmes/temp/p5.xml:119482:22: error: text not allowed here

I ran (the equivalent of) $ make clean $ make validate $ make html-web $ make test $ make exemplars LB> Huzzah! Did you also run all the test suite? MH> It comes here: MH> ... MH> I'm running make with no target -- are you running a specific MH> target?

And I should mention that my builds with a local, up-to-date copy of the Stylesheets/ repo. MH> The failure comes on make validate for me.

Ah -- I'm forgetting I should be pointing at a fresh Stylesheets repo. I'll try that when I get a chance. Mine will be building with the usual installed deb. Cheers, Martin On 15-08-07 02:36 PM, Syd Bauman wrote:

And I should mention that my builds with a local, up-to-date copy of the Stylesheets/ repo.

MH> The failure comes on make validate for me.

I think moving on to phase 2 now (in the P5-Pure branch, of course) is a reasonable plan. I would tag the existing P5-Pure branch before delving in to such a huge task, though. How much of phase 2 do you plan to automate, and how much to do by hand? I'd be happy to chip in on either approach.

This means, of course, that this branch cannot be merged with the trunk until there has been a new stylesheet release and we have moved over to using it on our jinxes. Which is a pain, but not too great a one. In the circs, I think I might proceed to phase 2 of operation purity: namely, switching the content models in P5 to use pure ODD. It ought to All Just Work, but...

Another update on the purification of TEI content models... this seems to have worked fine, with just a couple of manual tweaks needed, EXCEPT THAT we have a problem with DTD generation which may take a while to sort out. At the moment the content models are spattered liberally with rng:group elements for no other reason than to generate parentheses in the DTD content models. Redundant rng:group elements are not carried forward during purification (if they were, downstream ISO schematron validation would fail) so DTDs are appearing without enough parentheses in them. One solution to the problem, as I so far understand it, is that the code which generates parameter entities when processing the class hierarchy needs to be a bit smarter. For example, the parameter entity corresponding with model.common should have the value "(%model.divPart; |%model.inter;)" rather than "%model.divPart; |%model.inter;" But this is definitely a four pipe problem. Any DTD magicians out there are welcome to chip in. I haven't checked in the specs with "purified" content models yet though. Some Council members may wish to take this opportunity to rehearse the arguments for dropping (or not) DTD support ... On 09/08/15 22:06, Lou Burnard wrote:

I hope to automate it all (cf the Scripts directory I just added to P5-Pure) ... and then spend ages picking up the pieces, as per usual. So far today I have identified two show stoppers

a) the content model of macro.anyXML

b) in impure ODDs DTD generation required massive numbers of redundant rng:group elements. I am hoping that these are no longer required in pure odd, since turning them all blindly into <sequence> causes various things to break.

If someone else would like to ponder what to do about (a) while I attack (b) I'd be grateful!

L

On 08/08/15 14:27, Syd Bauman wrote:

...

I think moving on to phase 2 now (in the P5-Pure branch, of course) is a reasonable plan. I would tag the existing P5-Pure branch before delving in to such a huge task, though.

How much of phase 2 do you plan to automate, and how much to do by hand? I'd be happy to chip in on either approach.

...

This means, of course, that this branch cannot be merged with the trunk until there has been a new stylesheet release and we have moved over to using it on our jinxes. Which is a pain, but not too great a one. In the circs, I think I might proceed to phase 2 of operation purity: namely, switching the content models in P5 to use pure ODD. It ought to All Just Work, but...

I *knew* we must have met this problem before! A Jinks job which tries to build from P5-Pure might well be a good idea, though I would prefer it not to kick off every time I make a modification in the source: that's something I can check locally. Is it possible to have one that I can start manually? On 08/08/15 17:40, Martin Holmes wrote:

Our Jenkins servers use the latest stylesheets build from the Stylesheets job, not the release version. We did this on purpose so that we wouldn't have to force stylesheets releases just to do P5 development.

Would you like me to set up a build job on Jenkins for P5-Pure?

Cheers, Martin

On 15-08-08 02:40 AM, Lou Burnard wrote:

...

This means, of course, that this branch cannot be merged with the trunk until there has been a new stylesheet release and we have moved over to using it on our jinxes. Which is a pain, but not too great a one. In the circs, I think I might proceed to phase 2 of operation purity: namely, switching the content models in P5 to use pure ODD. It ought to All Just Work, but...

On 07/08/15 22:39, Martin Holmes wrote:

...

Ah -- I'm forgetting I should be pointing at a fresh Stylesheets repo. I'll try that when I get a chance. Mine will be building with the usual installed deb.

Cheers, Martin

On 15-08-07 02:36 PM, Syd Bauman wrote:

...

And I should mention that my builds with a local, up-to-date copy of the Stylesheets/ repo.

MH> The failure comes on make validate for me.

On 15-08-08 10:39 AM, Martin Holmes wrote:

Yes, we could create one that you could start manually. I can give you a login for the Jinks server and permission to do that.

On the other hand, if all three of us can now build this locally, and you don't want to see fresh builds from every commit, is there a point? Cheers, Martin

Cheers, Martin

On 15-08-08 10:18 AM, Lou Burnard wrote:

...

I *knew* we must have met this problem before!

A Jinks job which tries to build from P5-Pure might well be a good idea, though I would prefer it not to kick off every time I make a modification in the source: that's something I can check locally. Is it possible to have one that I can start manually?

On 08/08/15 17:40, Martin Holmes wrote:

...

Our Jenkins servers use the latest stylesheets build from the Stylesheets job, not the release version. We did this on purpose so that we wouldn't have to force stylesheets releases just to do P5 development.

Would you like me to set up a build job on Jenkins for P5-Pure?

Cheers, Martin

On 15-08-08 02:40 AM, Lou Burnard wrote:

...

This means, of course, that this branch cannot be merged with the trunk until there has been a new stylesheet release and we have moved over to using it on our jinxes. Which is a pain, but not too great a one. In the circs, I think I might proceed to phase 2 of operation purity: namely, switching the content models in P5 to use pure ODD. It ought to All Just Work, but...

On 07/08/15 22:39, Martin Holmes wrote:

...

Ah -- I'm forgetting I should be pointing at a fresh Stylesheets repo. I'll try that when I get a chance. Mine will be building with the usual installed deb.

Cheers, Martin

On 15-08-07 02:36 PM, Syd Bauman wrote:

...

And I should mention that my builds with a local, up-to-date copy of the Stylesheets/ repo.

MH> The failure comes on make validate for me.

On 15-08-08 10:53 AM, Lou Burnard wrote:

On 08/08/15 18:49, Martin Holmes wrote:

...

On 15-08-08 10:39 AM, Martin Holmes wrote:

...

Yes, we could create one that you could start manually. I can give you a login for the Jinks server and permission to do that.

On the other hand, if all three of us can now build this locally, and you don't want to see fresh builds from every commit, is there a point?

No not really, tbh. Maybe we could use this as a way of encouraging more council members to figure out how to build locally? It can't be that hard....

I'm not sure what the value of that is, actually. The idea of Jenkins is that it provides a known, robust build environment. If you build locally and it fails, you have no idea whether it failed because your build environment is screwed up somehow, or whether it's really failed; and if it succeeds, you also don't know whether it succeeded because you have a local copy of a file that is missing from the repo, and when you commit it will fail anyway on Jinks. But it wouldn't be hard to put together a little wiki page on how to do it. Do we suggest that you must build your own rnv, or that you must install the TEI packages to get it? What if you're on a Mac -- does it come with rnv? Cheers, Martin

Hi Lou, On 15-08-08 01:39 PM, Lou Burnard wrote:

On 08/08/15 21:10, Martin Holmes wrote:

...

I'm not sure what the value of that is, actually. The idea of Jenkins is that it provides a known, robust build environment. If you build locally and it fails, you have no idea whether it failed because your build environment is screwed up somehow,

almost never happens in my experience: if it fails it's cos you changed something, and you know what you have changed.

...

or whether it's really failed;

"really" failed? as opposed to?

Failed because you have an old Saxon on your system, or because you accidentally deleted a file, or pointed at the wrong stylesheets, or called make targets in the wrong order, or the mrs joyful prize for rafia work toppled off the shelf onto your keyboard at the wrong moment.

...

and if it succeeds, you also don't know whether it succeeded because you have a local copy of a file that is missing from the repo, and when you commit it will fail anyway on Jinks.

that's a very specific problem, though admittedly quite a common one: jinks is VERY useful as a way of catching it though.

...

But it wouldn't be hard to put together a little wiki page on how to do it.

I think I started to do that once... will look.

If you find it, let me know. I'll contribute. The key thing is pointing at a local copy of the Stylesheets.

...

Do we suggest that you must build your own rnv, or that you must install the TEI packages to get it? What if you're on a Mac -- does it come with rnv?

Or shall we just abandon using rnv?

Hmm. I don't really like the notion of abandoning checks. Isn't RNV the only thing that's checking our .rnc products? Cheers, Martin

On the topic of creating documentation on building P5 locally ... On 8/8/15 3:46 PM, Lou Burnard wrote:

On 08/08/15 21:10, Martin Holmes wrote:

...

But it wouldn't be hard to put together a little wiki page on how to do it.

In the TEI SF repo there is a folder called Documents (same level as P5) which contains inter alia a document called howtomakep5.xml that does more or less this. I started working on some time ago... some way to go still (in particular it needs a mac person to fill in the appropriate section, syd) but it's a start

... I'll also point out http://www.tei-c.org/Guidelines/P5/get.xml , which has some documentation in related to this in two sections: "P5 Guidelines and Schema Development Package" and "Example: Installation of Stylesheets, P5, and Roma packages". I suggest revising this page rather than keeping this documentation buried in howtomakep5.xml, where it's unlikely to be discovered by those who need it. If anyone revises this, feel free to send a revised version of get.xml to web@tei-c.org so that Nick or I will update the page. Kevin

It's hard to see what the rnv validation adds to the onvdl validation, since the rnc we are using for the fornmer is generated from the rng used by the latter. Checking up on James Clark's code seems a little presumptuous. But I suppose the more checks the better was the probable motivation: certainly Sebastian has said in the past that the rnv test was dispensable with : if it's not there the Makefile wont fail.

Turns out there are two GitHub repos, one by the original author (which contains a bugfix from May) and another by the maintainer of the deb package (who is now no longer maintaining it because he moved to Gentoo, but who says he has some fixes from Fedora which haven't yet been applied). The ideal situation would be that these two coordinate and get all the fixes applied in a single repo (presumably the one belonging to the original author), and that we can then perhaps get distros to adopt the package again. I think the latter is unlikely, though, so I think we should consider maintaining our own package for the sake of convenience. Cheers, Martin On 15-08-09 10:26 AM, Martin Holmes wrote:

Sebastian confirms that the reasons were speed (it's C) and the desire to use a second piece of software to broaden the base.

I see rnv has recently been moved to GitHub, and I've written to the person who was the original maintainer of the Ubuntu package to see why it was dropped. I wouldn't mind seeing it back in Ubuntu; failing that I see no reason why we shouldn't build it assuming we're maintaining our own debs, and get the latest (there was a bugfix in May, but no binary release including it).

Cheers, Martin

On 15-08-09 03:44 AM, Lou Burnard wrote:

...

It's hard to see what the rnv validation adds to the onvdl validation, since the rnc we are using for the fornmer is generated from the rng used by the latter. Checking up on James Clark's code seems a little presumptuous.

But I suppose the more checks the better was the probable motivation: certainly Sebastian has said in the past that the rnv test was dispensable with : if it's not there the Makefile wont fail.

I always run make with no targets. This according to the current Makefile is the same as doing validate exemplars test html-web But I usually precede it with a "make clean" , which presumably forces validate to rebuild everything, as per P5-Test on Jenkins. On 08/08/15 18:23, Martin Holmes wrote:

Following Syd's make targets, I was still unable to build this branch, using a freshly-updated and built version of the Stylesheets. This was the output from the validate target:

-------------- mholmes@linux-2013:~/DiskStation/mholmes/WorkData/tei/temp$ make XSL=/home/mholmes/DiskStation/mholmes/WorkData/tei/gitrepo/release/xsl/xml/tei/stylesheet validate BUILD: Check validity with rnv if we have it command -v rnv && rnv -v p5odds.rnc p5.xml /usr/bin/rnv rnv version 1.7.8 p5odds.rnc:-1:-1: error: I/O error: No such file or directory

make: [valid] Error 1 (ignored) BUILD: Check validity with special-purpose XSL code, looking for bad links etc ANT_OPTS="-Xss2m -Xmx752m -Djava.awt.headless=true" ant -q -lib Utilities/lib/saxon9he.jar -f antbuilder.xml -Dverbose= -DXSL=/home/mholmes/DiskStation/mholmes/WorkData/tei/gitrepo/release/xsl/xml/tei/stylesheet validators [echo] Run Schematron script (normal part of Guidelines)

BUILD FAILED /home/mholmes/DiskStation/mholmes/WorkData/tei/temp/antbuilder.xml:186: stylesheet /home/mholmes/DiskStation/mholmes/WorkData/tei/temp/p5odds.isosch.xsl doesn't exist. --------------

The messages are correct: in the checked-out P5-Pure branch, there was no sign of p5odds.rnc or p5odds.isosch.xsl.

So I went back to the Jenkins config and found that the build targets there are:

clean deb dist

so I tried that. It now builds OK.

So Syd, I think at some point you must have run dist otherwise you couldn't have successfully run validate. If I run validate after the regular build it works.

What puzzles me a bit is that P5-Test on Jinks runs clean validate test. Hmm.

Cheers, Martin

On 15-08-08 09:40 AM, Martin Holmes wrote:

...

Our Jenkins servers use the latest stylesheets build from the Stylesheets job, not the release version. We did this on purpose so that we wouldn't have to force stylesheets releases just to do P5 development.

Would you like me to set up a build job on Jenkins for P5-Pure?

Cheers, Martin

On 15-08-08 02:40 AM, Lou Burnard wrote:

...

This means, of course, that this branch cannot be merged with the trunk until there has been a new stylesheet release and we have moved over to using it on our jinxes. Which is a pain, but not too great a one. In the circs, I think I might proceed to phase 2 of operation purity: namely, switching the content models in P5 to use pure ODD. It ought to All Just Work, but...

On 07/08/15 22:39, Martin Holmes wrote:

...

Ah -- I'm forgetting I should be pointing at a fresh Stylesheets repo. I'll try that when I get a chance. Mine will be building with the usual installed deb.

Cheers, Martin

On 15-08-07 02:36 PM, Syd Bauman wrote:

...

And I should mention that my builds with a local, up-to-date copy of the Stylesheets/ repo.

MH> The failure comes on make validate for me.

OK, now I'm confused. My local working copy of the P5 trunk says I've screwed up, model.eventLike.xml is not checked-in, and that model.persEventLike.xml & model.placeEventLike.xml are not checked into Defunct/: | (541) albus ~/Documents/P5 @ 14:29:11 ->svn info | Path: . | URL: https://svn.code.sf.net/p/tei/code/trunk/P5 | Repository Root: https://svn.code.sf.net/p/tei/code | Repository UUID: e5332ce4-a50f-0410-b94b-d658400b0204 | Revision: 13321 | Node Kind: directory | Schedule: normal | Last Changed Author: hcayless | Last Changed Rev: 13319 | Last Changed Date: 2015-08-06 16:17:39 -0400 (Thu, 06 Aug 2015) | | (542) albus ~/Documents/P5 @ 14:29:19 ->svn stat -u | egrep model\. | ? Source/Defunct/model.placeEventLike.xml | ? Source/Defunct/model.persEventLike.xml | ? Source/Specs/model.eventLike.xml HOWEVER, the file https://svn.code.sf.net/p/tei/code/trunk/P5/Source/Specs/model.eventLike.xml exists. And if I check out a fresh copy of https://svn.code.sf.net/p/tei/code/trunk/P5, I find that all the model.(pers|place)?eventLike files are where they belong. So are checked in and my working copy is messed up, or what? I'm a little lost, I think. (In branches/P5-Pure/ the two defunct ones are where they belong, but model.eventLike itself is not.)