August 2015 - Tei-council - lists.tei-c.org

Completed test migration of SourceForge FRs and bugs
by Raffaele Viglianti 17 Sep '15

17 Sep '15

Dear all, First of all apologies if you have just been spammed by GitHub notifications :( I've tested migrating to https://github.com/TEIC/Guidelines-TEST/issues Feature Requests and bugs as exported by Hugh on July 28. That's a total of 564 FRs and 764 bugs. I'm happy to report that all have been imported successfully. This was done using the perl script gosf2gh, which I've adjusted a bit to work with our requirements. The code of my fork is here: https://github.com/raffazizzi/gosf2github Feel free to contribute! A few comments: - All issues and comments were created by raffazizzi because I used my API token. When we do this for real, we should use a TEITechnicalCouncil token. - Even though all issues and comments will appear created by the same user, the script carries over the original creator / commentator in the body of the issue/comment. If we have a GitHub user for it, the GitHub user will appear, otherwise the old username will. - Referencing users: I've avoided adding explicit references to GitHub users (eg @raffazizzi) because the script I used warns that too much spam could cause my token to be banned. The advantage of referencing users directly is that issues can be searched by mentioned users, but we may have to give this one up. - Issue assignment: if the ticket was not assigned in SF, it won't be assigned in GH. If it was assigned to a user that is currently not a collaborator, it will be assigned to TEITechnicalCouncil. A label is also added with the old sourceforge assignee in the form of sf_assignee-USER. I'm now realizing that it may be more useful to have these labels for the original *creators* not the assignees. Thoughts? - Formatting of the text looks pretty good, but code blocks are a bit broken. The script guesses by matching ~~~~ blocks, but sometimes they're marked wit more tildes. I can try to fix this, but it's probably going to be a bit unreliable. Not a big deal, though, IMO. If you notice any other formatting problems, let me know! Raff

4 6

Update on build problems with P5
by Martin Holmes 01 Sep '15

01 Sep '15

HI all, Hugh and I have been wrestling with the very puzzling results of the P5 build on Jenkins: while TEIP5-Documentation works as expected, generating Guidelines based on the git version, TEIP5 seems to know nothing of git, and behaves as if we're still using svn. I think I've figured out the cause. The Makefile* includes targets "debversion" and "deb" (the latter depends on the former), and the latter seems to be shot through with explicit references to svn: @echo BUILD: make Debian tei-p5-database package (cd debian-tei-p5-database; debclean;debuild -eXSL=${XSL} -eINJENKINS=${INJENKINS} --no-lintian -nc -b -i.svn -I.svn -uc -us) The Documentation build does not try to build the deb packages, but the full P5 build calls: make* VCS=git clean deb dist and in the process of that happening, VCS is set back to svn somehow. If I call make* VCS=git clean dist everything works OK. I know nothing about building debian packages, so this is a mystery to me. I don't mind learning about it, and people assure me that it's not complicated, but it might be more than I can do in a couple of weeks, at least with any confidence. Does anyone else have familiarity with building deb packages? *I hate make. Cheers, Martin

4 9

GitHub Repo import
by Hugh Cayless 31 Aug '15

31 Aug '15

Finally finished and available at https://github.com/TEIC/Guidelines Took about a week to run, with one interruption over the weekend due to some network glitch. Given the degree of pain involved, if we think this looks good, I think we should run with it. So let's: a) stop committing to SF (at least for now, maybe forever). Maybe I should actually restrict commit access to it lest anyone not on this list push an update. b) Check the above repo over for warts/see if anything's missing. There are a couple of branches that are artifacts of the CVS -> SVN migration that weren't visible on Sourceforge. We can probably get rid of them, but I haven't yet. There's also an sf/trunk branch, which I think is just a copy of master. Thoughts?

5 31

TEI Simple vs TEI Minimal
by Martin Holmes 31 Aug '15

31 Aug '15

If you click on File / New in Oxygen, and look at the TEI P5 options, you'll see a bunch of templates called: Minimal Minimal with Internationalization Minimal with Math and so on. I believe that having TEI Simple alongside TEI Minimal (not to mention TEI Lite and TEI Tite as well) is going to be mighty confusing. I suggest that before we integrate Simple into this ecosystem we might want to reconsider whether we want these Minimal schemas or not. Cheers, Martin

2 2

Agenda and Hangout info
by Hugh Cayless 31 Aug '15

31 Aug '15

Agenda is here: https://docs.google.com/document/d/16XMTZsjq09GnwBRiLCpzXbXTyd5R9dAVhKS3ZH2… The Google Hangout link is in the agenda. Do feel free to add items!

1 0

let me share my pain
by Lou Burnard 31 Aug '15

31 Aug '15

You may be wondering how Pure ODD is progressing... Well, all over the place, there are bits of content model like this (expressed in pure ODD) <sequence minOccurs="0" maxOccurs="unbounded"> <classRef key="model.divBottomPart" /> <classRef key="model.global" minOccurs="0" maxOccurs="unbounded"/> </sequence> i.e. there may be nothing here at all, or there may be at least one member of the model.divBottomPart class possibly followed by some model.globals. This makes perfect sense when transformed to RelaxNG or even in XSD, but the DTD fragment we get for it is: (%model.divBottomPart;,(%model.global;)*)* which is not syntactically correct in DTD language. It needs to be ((%model.divBottomPart;),(%model.global;)*)* Alas, using the current oddtodtd, those extra parens only appear if the bare class reference is inside either a sequence or an alternation. But we have a schematron rule that says (reasonably enough) that a <sequence> must have two or more children. (We don't see this problem in the impure ODD, of course, because RelaxNG doesn't mind <group>s containing just a single pattern) This is not an isolated problem, either.... it's a favourite trick in our content models. Having spent most of today on this, I am reluctantly coming to one of the following conclusions: a) we should just abandon DTDs b) I could write a dirty hack to detect these things and post process the DTDs c) we should quietly drop the schematron rule d) someone (not me) should rewrite odd to dtd either to deal with these things properly or (more radically) to expand all parameter entities e) there is another silver bullet somewhere but i am too stupid to see it Any opinions? Recommendations?

4 22

constraintSpec
by Lou Burnard 31 Aug '15

31 Aug '15

The content model of <constraintSpec> is currently <rng:zeroOrMore> <rng:choice> <rng:ref name="model.glossLike"/> <rng:ref name="model.descLike"/> </rng:choice> </rng:zeroOrMore> <rng:optional> </rng:optional> or, in pure ODD, <alternate minOccurs="0" maxOccurs="unbounded"> <classRef key="model.glossLike"/> <classRef key="model.descLike"/> </alternate> <elementRef key="constraint" minOccurs="0"/> The fact that this gives rise to an invalid DTD content model (there needs to be a <rng:group> or <sequence> round te whole shebang) isn't really my question though. This is my question: What on earth is the point of a <constraintSpec> that doesn't contain a <constraint>? Or if the idea was to permit just documentation (but not a specification) for one, why isn't the model just <rng:zeroOrMore> <rng:choice> <rng:ref name="model.glossLike"/> <rng:ref name="model.descLike"/> <rng:ref name="constraint"/> </rng:choice> </rng:zeroOrMore> ? I think this is a corrigible error, and the <rng:optional>ity of <constraint> should be removed.

3 3

Abstract Model Enforcement
by Hugh Cayless 31 Aug '15

31 Aug '15

In my own copy of the Guidelines, which I’ll probably switch to a branch in the main repo soon, I’ve been playing with the new app. crit. capabilities. As you probably remember, we decided that allowing larger structures inside <lem> and <rdg> made sense, as long as there wasn’t any violation of the abstract model (e.g. using <rdg> to put a inside a ). I’ve been experimenting with adding Schematron rules that enforce abstract model concepts, such as: "no inside or <ab>", "no <div> inside " etc. These are general rules, nothing to do with the critical apparatus changes per se, but they should help. The commit with the changes is https://github.com/hcayless/TEI-Guidelines/commit/66d11fab1be737c0f17ad4521… <https://github.com/hcayless/TEI-Guidelines/commit/66d11fab1be737c0f17ad4521…> and you can experiment by grabbing a copy of https://github.com/hcayless/TEI-Guidelines/blob/appcrit/AppCrit/examples/Pr… <https://github.com/hcayless/TEI-Guidelines/blob/appcrit/AppCrit/examples/Pr…> and trying to cheat by putting, say, inside a <rdg> inside an <l>. If you do this in Oxygen, it should fuss at you and identify the problem at the correct spot. There are changes to prose that still need to be made, and there are probably more Schematron rules we could add, but I think this is workable. Thoughts?

2 1

gnuify-changelog.pl problem in Roma source may apply to P5
by Martin Holmes 30 Aug '15

30 Aug '15

Hi all, I did a quick bugfix for this ticket: <http://sourceforge.net/p/tei/bugs/724/> back in July, but I'm now dealing with the ramifications of getting it onto tei-c.org, and in the process, noticing that the project was moved from SF to GitHub without some key code changes being made to handle the changelog; I think the same issues will apply to P5, so I think we could use Roma as a good testcase for handling the change. When you run "make dist" or "make install" in the Roma source, one of the things that should happen is that the latest changes are added to the ChangeLog file. That's done by calling a PERL script called gnuify-changelog, which seems to be fairly widely used; but it is explicitly customized for the project and its contributors, all of whom are listed explicitly. What it does is to call svn log to get the latest commits, then converts them into a GNU-style changelog by substituting more detailed user info for user names. There are four problems with this: 1. gnuify-changelog.pl no longer exists in the Roma source, since it's been pulled out of the P5 tree. 2. We're now using git, not svn, so the log info is differently formatted. 3. The script requires a hard-coded list of contributors to be maintained. However, git log seems to be more friendly than svn log, in that it includes user name and email: --------------- $> git log -n 1 --name-status commit 18cc86ddcd79b409a9bfa406426fe4685c37fa8a Author: martindholmes <mholmes(a)uvic.ca> Date: Mon Jul 27 14:05:29 2015 -0700 Fix for http://sourceforge.net/p/tei/bugs/724/. I'm doing a string-replace which catches the specific bug reported (double quotes) in the title and author fields, but doesn't make any attempt to do more comprehensive escaping, since that would need a significant rewrite of large blocks of the code, I think. M VERSION M roma/romadom.php --------------- git log can also be massaged to output stuff in XML, by defining an XML format. So I'd like to suggest the following, to be piloted and tested with Roma, but then implemented for P5 (and any other projects which happen to be using gnuify-changelog.pl in our setup: Replace that call with an ant task which reads the existing XML log file (if it exists), calls git log to output anything subsequent to the last entry in XML, calls Saxon to XSLT it to whatever we want and insert it into the existing XML log file, then calls Saxon again to XSLT the now-updated XML log file into the GNU-preferred text format. Alternatively, if those of you who love PERL would like to simply update gnuify-changelog.pl so that it can use the git log output format, you're welcome to do that; it's a bit urgent because I don't think a build will work properly otherwise. Another option is to rely (for P5 builds) on having an svn checkout which can generate an svn log in the expected format, and using that; and then re-inserting gnuify-changelog.pl into the Roma tree, changing the Makefile to point to it, and doing other updates (adding and removing potential contributors) to both PERL files. I think the ant/XSLT solution is better in the long term, and removes one more PERL dependency, but I suspect I'm in a minority there. Cheers, Martin

2 2

Next Telecon
by Hugh Cayless 28 Aug '15

28 Aug '15

Hi all, A reminder that the Doodle poll for our August telecon needs your input, if you haven't already filled it out: http://doodle.com/5ks6ma5f3fez838c Right now, August 31 looks like the best date. I expect the agenda will focus mainly on the migration and on the logistics of a Fall release. It would be really nice to be able to discuss some tickets too though! All the best, Hugh

7 20