Lou had removed it, and the merge accidentally put it back. I’ll revert the file.

On Oct 2, 2015, at 8:35 , Syd Bauman wrote:

What? That's crazy. I never touched fvLib.xml, at least not by hand. It seems like the only time that file was altered in branch sydb-xenodata was 09-11 by Martin to change pointers to NVDL schemas (ostensibly in the xml-model PI, which wouldn't touch this), and on 09-29 when I merged changes that had occurred on master into that branch. Is that the merge you refer to, or when Hugh merged sydb-xenodata back into master yesterday?

Either way, I'm really worried that other problems may have been introduced. I have this sinking feeling in my git ...

If I understand correctly, predeclare=true means that the declaration for the structure (in this case, macro.anyXML) gets output not in the schema fragment in which it is declared (in this case 'tagdocs'), but rather in the 'tei' architecture schema fragment. This was needed because the new <xenoData> element, which uses macro.anyXML in its content model, is declared in the 'header' module. Schemas are loaded 'tei' first, and then the rest in alphabetical order. So header.rnc was referring to macro.anyXML which, since tagdocs.rnc had not been loaded yet, was not declared. Moving the declaration of macro.anyXML into the first schema loaded (along with most of the other macros) solves that.

AFAIK, this only affects use of the schemas when one wants to use them as fragments as we used to do when we constructed pizzas.[1] I don't even remember how to do that with RELAX NG, and AFAIK we provide no documentation to tell users how to do that with either RELAX NG or DTDs. And I've never heard of anyone doing it.

Notes ----- [1] Although logically it might affect flattened DTDs, too, I presume we test for that in the build process. Do we?

...

FWIW, you seem to have reintroduced a nonsensical content model

fvLib (prior to your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <ref name="model.featureVal"/> </zeroOrMore> </content>

fvLib (after your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <choice> <ref name="model.featureVal"/> </choice> </zeroOrMore> </content>

This doesnt affect schema production, and is thus benign. But, if for some reason we have to rerun the purification of content models script again before I merge in the pure ODD branch, this one will need hand fixing again.

And I don't understand (never have) what @preDeclare is doing. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Make that "Hugh accidentally put it back during the merge" :-) There might be any number of reasons why Syd’s branch didn’t pick that file up properly—all of them come down to human error. I don’t know whose, but it might well have been mine. Anyway, it’s easily fixed.

On Oct 2, 2015, at 8:55 , Lou Burnard wrote:

"the merge accidentally put it back"

this does not inspire confidence .:-)

On 02/10/15 13:45, Hugh Cayless wrote:

...

Lou had removed it, and the merge accidentally put it back. I’ll revert the file.

...

On Oct 2, 2015, at 8:35 , Syd Bauman wrote:

What? That's crazy. I never touched fvLib.xml, at least not by hand. It seems like the only time that file was altered in branch sydb-xenodata was 09-11 by Martin to change pointers to NVDL schemas (ostensibly in the xml-model PI, which wouldn't touch this), and on 09-29 when I merged changes that had occurred on master into that branch. Is that the merge you refer to, or when Hugh merged sydb-xenodata back into master yesterday?

Either way, I'm really worried that other problems may have been introduced. I have this sinking feeling in my git ...

If I understand correctly, predeclare=true means that the declaration for the structure (in this case, macro.anyXML) gets output not in the schema fragment in which it is declared (in this case 'tagdocs'), but rather in the 'tei' architecture schema fragment. This was needed because the new <xenoData> element, which uses macro.anyXML in its content model, is declared in the 'header' module. Schemas are loaded 'tei' first, and then the rest in alphabetical order. So header.rnc was referring to macro.anyXML which, since tagdocs.rnc had not been loaded yet, was not declared. Moving the declaration of macro.anyXML into the first schema loaded (along with most of the other macros) solves that.

AFAIK, this only affects use of the schemas when one wants to use them as fragments as we used to do when we constructed pizzas.[1] I don't even remember how to do that with RELAX NG, and AFAIK we provide no documentation to tell users how to do that with either RELAX NG or DTDs. And I've never heard of anyone doing it.

Notes ----- [1] Although logically it might affect flattened DTDs, too, I presume we test for that in the build process. Do we?

...

FWIW, you seem to have reintroduced a nonsensical content model

fvLib (prior to your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <ref name="model.featureVal"/> </zeroOrMore> </content>

fvLib (after your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <choice> <ref name="model.featureVal"/> </choice> </zeroOrMore> </content>

This doesnt affect schema production, and is thus benign. But, if for some reason we have to rerun the purification of content models script again before I merge in the pure ODD branch, this one will need hand fixing again.

And I don't understand (never have) what @preDeclare is doing. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

The anyXML thing is not quite as you describe it. if the macro is declared in the tagdocs module, as it currently is, then you won't get it unless you include that module in your schema. If you want to use it in xenodata, then you need to declare it in the infrastructure module. the @predeclare thing is not really relevant: it is a legacy piece of trickery whjich we should not be using. Just move the declaration to the infrastructure module and add the appropriate text into chapter ST would be my recommendation. On 02/10/15 13:35, Syd Bauman wrote:

What? That's crazy. I never touched fvLib.xml, at least not by hand. It seems like the only time that file was altered in branch sydb-xenodata was 09-11 by Martin to change pointers to NVDL schemas (ostensibly in the xml-model PI, which wouldn't touch this), and on 09-29 when I merged changes that had occurred on master into that branch. Is that the merge you refer to, or when Hugh merged sydb-xenodata back into master yesterday?

Either way, I'm really worried that other problems may have been introduced. I have this sinking feeling in my git ...

If I understand correctly, predeclare=true means that the declaration for the structure (in this case, macro.anyXML) gets output not in the schema fragment in which it is declared (in this case 'tagdocs'), but rather in the 'tei' architecture schema fragment. This was needed because the new <xenoData> element, which uses macro.anyXML in its content model, is declared in the 'header' module. Schemas are loaded 'tei' first, and then the rest in alphabetical order. So header.rnc was referring to macro.anyXML which, since tagdocs.rnc had not been loaded yet, was not declared. Moving the declaration of macro.anyXML into the first schema loaded (along with most of the other macros) solves that.

AFAIK, this only affects use of the schemas when one wants to use them as fragments as we used to do when we constructed pizzas.[1] I don't even remember how to do that with RELAX NG, and AFAIK we provide no documentation to tell users how to do that with either RELAX NG or DTDs. And I've never heard of anyone doing it.

Notes ----- [1] Although logically it might affect flattened DTDs, too, I presume we test for that in the build process. Do we?

...

FWIW, you seem to have reintroduced a nonsensical content model

fvLib (prior to your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <ref name="model.featureVal"/> </zeroOrMore> </content>

fvLib (after your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <choice> <ref name="model.featureVal"/> </choice> </zeroOrMore> </content>

This doesnt affect schema production, and is thus benign. But, if for some reason we have to rerun the purification of content models script again before I merge in the pure ODD branch, this one will need hand fixing again.

And I don't understand (never have) what @preDeclare is doing.

Sounds unlikely. I've just updated my local copy and will see if I get the same problem. On 02/10/15 14:30, Hugh Cayless wrote:

Is this why the builds are failing? It’s something to do with the PDF generation, but I haven’t been able to track it down yet.

...

On Oct 2, 2015, at 8:54 , Lou Burnard wrote:

The anyXML thing is not quite as you describe it. if the macro is declared in the tagdocs module, as it currently is, then you won't get it unless you include that module in your schema. If you want to use it in xenodata, then you need to declare it in the infrastructure module. the @predeclare thing is not really relevant: it is a legacy piece of trickery whjich we should not be using. Just move the declaration to the infrastructure module and add the appropriate text into chapter ST would be my recommendation.

On 02/10/15 13:35, Syd Bauman wrote:

...

What? That's crazy. I never touched fvLib.xml, at least not by hand. It seems like the only time that file was altered in branch sydb-xenodata was 09-11 by Martin to change pointers to NVDL schemas (ostensibly in the xml-model PI, which wouldn't touch this), and on 09-29 when I merged changes that had occurred on master into that branch. Is that the merge you refer to, or when Hugh merged sydb-xenodata back into master yesterday?

Either way, I'm really worried that other problems may have been introduced. I have this sinking feeling in my git ...

If I understand correctly, predeclare=true means that the declaration for the structure (in this case, macro.anyXML) gets output not in the schema fragment in which it is declared (in this case 'tagdocs'), but rather in the 'tei' architecture schema fragment. This was needed because the new <xenoData> element, which uses macro.anyXML in its content model, is declared in the 'header' module. Schemas are loaded 'tei' first, and then the rest in alphabetical order. So header.rnc was referring to macro.anyXML which, since tagdocs.rnc had not been loaded yet, was not declared. Moving the declaration of macro.anyXML into the first schema loaded (along with most of the other macros) solves that.

AFAIK, this only affects use of the schemas when one wants to use them as fragments as we used to do when we constructed pizzas.[1] I don't even remember how to do that with RELAX NG, and AFAIK we provide no documentation to tell users how to do that with either RELAX NG or DTDs. And I've never heard of anyone doing it.

Notes ----- [1] Although logically it might affect flattened DTDs, too, I presume we test for that in the build process. Do we?

...

FWIW, you seem to have reintroduced a nonsensical content model

fvLib (prior to your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <ref name="model.featureVal"/> </zeroOrMore> </content>

fvLib (after your merge)

<content> <zeroOrMore xmlns="http://relaxng.org/ns/structure/1.0"> <choice> <ref name="model.featureVal"/> </choice> </zeroOrMore> </content>

This doesnt affect schema production, and is thus benign. But, if for some reason we have to rerun the purification of content models script again before I merge in the pure ODD branch, this one will need hand fixing again.

And I don't understand (never have) what @preDeclare is doing. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

It worked pre-xenodata merge though, and works still if you revert to https://github.com/TEIC/TEI/commit/fb89c699b64610cbbfbf7c9c25c87d3a0f1248fc https://github.com/TEIC/TEI/commit/fb89c699b64610cbbfbf7c9c25c87d3a0f1248fc, so something weird is going on.

On Oct 2, 2015, at 11:26 , Lou Burnard wrote:

PDF generation appears to be failing because the cross references for element names are not being properly created. Am investigating.

I don't think this is caused by any recent change though.

On 02/10/15 14:30, Hugh Cayless wrote:

...

Is this why the builds are failing? It’s something to do with the PDF generation, but I haven’t been able to track it down yet.

...

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Yep, as I thought. the PDF build falls over in the middle (and therefore not all cross references get resolved) because there is an unescaped # character (which is Magic in Latex) floating around in one of the xenodata examples. I removed the # (which doesn't affect the sense of the example in any way that I can see) and the PDF is built just fine. On 02/10/15 16:51, Lou Burnard wrote:

Ah, found it.

Testing...

On 02/10/15 16:46, Hugh Cayless wrote:

...

It worked pre-xenodata merge though, and works still if you revert to https://github.com/TEIC/TEI/commit/fb89c699b64610cbbfbf7c9c25c87d3a0f1248fc https://github.com/TEIC/TEI/commit/fb89c699b64610cbbfbf7c9c25c87d3a0f1248fc, so something weird is going on.

...

On Oct 2, 2015, at 11:26 , Lou Burnard wrote:

PDF generation appears to be failing because the cross references for element names are not being properly created. Am investigating.

I don't think this is caused by any recent change though.

On 02/10/15 14:30, Hugh Cayless wrote:

...

Is this why the builds are failing? It’s something to do with the PDF generation, but I haven’t been able to track it down yet.

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

We’re in our freeze period, so bugfixes and typos should be corrected in master. Nothing else should be happening there.

On Oct 2, 2015, at 11:12 , Syd Bauman wrote:

Hmmm ... so you're suggesting that if we leave things as they are, then an ODD that had just

<schemaSpec> <moduleRef key="tei"/> <moduleRef key="header"/> <moduleRef key="core"/> <moduleRef key="textstructure"/> </schemaSpec>

would produce incorrect schemas because <xenoData> of 'header' wants macro.anyXML which will not be there?

Well, I just tried it, and you are absolutely right. (The good news is that the ODD processor does not generate an invalid schema. The bad news is that, as you predicted, it does not produce the right schema, either.)

I will fix this as soon as I hear from Hugh as to whether I should fix it in sydb-xenodata, master, or do something else.

...

The anyXML thing is not quite as you describe it. if the macro is declared in the tagdocs module, as it currently is, then you won't get it unless you include that module in your schema. If you want to use it in xenodata, then you need to declare it in the infrastructure module. the @predeclare thing is not really relevant: it is a legacy piece of trickery whjich we should not be using. Just move the declaration to the infrastructure module and add the appropriate text into chapter ST would be my recommendation. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

I do not think we need a show of hands. This is a corrigible error that should never see the light of day. It is most definitely a bug, and one that needs to be squashed. HOWEVER, it's not nearly so easy as it looks. In fact, it's a big problem. If I just do as you suggested, Lou, we run into a validation problem. Remember that there is a problem in the RELAX NG world with validating macro.anyXML when DTD compatibility mode is on. You get a conflicting ID-types for attribute "id" ... error, which indeed I get when running `make test` from both the Validate testbasic.xml against RELAXNG testbasic.rng and the Validate tei_svg.tei against RELAXNG tei_svg.rng steps. I will be looking into this in more detail as soon. (I need to eat first.) But in the meantime, do people want me to check in the change so they can play with the error, or try to fix it first?

So we need a quick show of hands as to whether we consider this[1] to be a bug or not[2].

[1] i.e. the fact that you won't get a satisfactory schema if you (a) select the header module and (b) do not select the tagdocs module and (c) expect to be able to use <xenodata>

[2] I think it is.

The Jing task for ant is documented here: http://www.thaiopensource.com/relaxng/jing-ant.html I believe (although I haven't tested it) that you can set the required param like this: <target name="validaterng"> <echo level="info">Validate ${testfile} against RELAXNG ${outputname}.rng</echo> <runjing rngfile="${outputname}.rng" file="${testfile}" checkid="false"/> </target> i.e. by adding @checkid="false" to the runjing task call on line 134. Hope this helps, Martin On 15-10-02 10:38 AM, Syd Bauman wrote:

So, here's the deal as far as I can figure out rapidly.

Basically, James Clark and MURATA Makoto did not intend us to use DTD compatibility mode to check our ID/IDREFs. They had expected there to be a part 6 to DSDL which would specify a specialized little language just for the purpose. But that never happened, there is no part 6 to DSDL. So we're kinda stuck.

I'm not sure what to do, but possibilities include:

1. Just not checking ID/IDREF on the two files that cause problems

2. Turning off ID/IDREF checking on all our RELAX NG validation, and adding a separate step to check ID/IDREF. This would be accomplished by either a) writing a Schematron rule or b) by tweaking the declaration of macro.anyXML.

3. Others?

My initial thought is that the best bet is #2a: - stop using DTD compatibility mode to check ID/IDREF -- which is what the rest of this mail is about - use Schematron to check ID/IDREFs -- which is what a future post will be about

The way you turn off ID/IDREF checking with `jing` on the commandline is to specify the '-i' switch. But I don't know how to do that in ant. The offending ant file is .../TEI/P5/Test/antruntest.xml, line 134 (defined on line 51). It causes problems twice: validating .../TEI/P5/Test/tei_svg.tei against .../TEI/P5/Test/tei_svg.rng validating .../TEI/P5/Test/testbare.xml against .../TEI/P5/Test/testbare.rng

Any thoughts?

...

I do not think we need a show of hands. This is a corrigible error that should never see the light of day. It is most definitely a bug, and one that needs to be squashed.

HOWEVER, it's not nearly so easy as it looks. In fact, it's a big problem. If I just do as you suggested, Lou, we run into a validation problem. Remember that there is a problem in the RELAX NG world with validating macro.anyXML when DTD compatibility mode is on. You get a conflicting ID-types for attribute "id" ... error, which indeed I get when running `make test` from both the Validate testbasic.xml against RELAXNG testbasic.rng and the Validate tei_svg.tei against RELAXNG tei_svg.rng steps.

I will be looking into this in more detail as soon. (I need to eat first.) But in the meantime, do people want me to check in the change so they can play with the error, or try to fix it first?

On 02/10/15 21:40, Syd Bauman wrote: Mr. Jenkins[1] is still complaining about not being able to use XeLaTeX to build PDFs, though. you need to lose the sharp sign in your xenodata as I said earlier

It's a pain, isn't it... (and you have to wonder what sort of an idiot thinks that it would be cool to make up a namespace like that) . But if you want to produce the Guidelines in PDF using LaTeX, either you have to ditch that example, or misrepresent the RDF namespace, or somehow find a way of tweaking the LaTeX code before it gets processed (in LaTeX, I believe, you would simply escape the character with a backslash) On 02/10/15 23:13, Syd Bauman wrote:

...

you need to lose the sharp sign in your xenodata as I said earlier Sorry, hadn't caught up with the PDF thread, yet.

...

because there is an unescaped # character (which is Magic in Latex) floating around in one of the xenodata examples ... which doesn't affect the sense of the example in any way that I can see Thanks for the catch, Lou. (Nicely done.) But how do I escape it?[1] (I don't think it makes sense to just remove it, because according to http://www.w3.org/TR/REC-rdf-syntax/#section-Namespace the RDF namespace is "http://www.w3.org/1999/02/22-rdf-syntax-ns#", not "http://www.w3.org/1999/02/22-rdf-syntax-ns". And since, according to http://www.w3.org/TR/REC-xml-names/#NSNameComparison, comparison of namespaces is simply a case-sensitive string comparison, not a resolve-URI-escapes-first comparison, we can't use "...-ns%23".)

Notes ----- [1] There is only 1 problematic '#', I think. There are only 4 <egXML> elements that contain <xenoData>, all in xenoData.xml. There are only 3 '#' signs in that entire file (and I checked for NCRs, too). One of them is part of an NCR ("…"), so it's not the problem. Another is part of a namespace declaration, so it's not the problem. The third is the last character of the RDF namespace, which is being given to the reader because the example uses that namespace.

Hi Syd,

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this.

Curses and more curses. This is what I've been dreading. If it's true that this is never overridden by an XSLT variable being passed in, then I was wrong that Jenkins is no longer dependent (as Sebastian and I believed) on the debs being installed; and that means our own build process is still dependent on the debs being up to date. Hugh, I believe you built a Jenkins that had no TEI debs installed on it, didn't you? Can it build the PDFs OK? Cheers, Martin On 15-10-02 05:22 PM, Syd Bauman wrote:

Egads. Ick.

But this doesn't make sense. Surely the character '#' occurs in content elsewhere in the Guidelines.

Just looked: yes, it does, 95 times. E.g.: <code>http://www.example.org/somewhere.xml#p1</code>

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>?

The difference *may* be that in the above case, the sharp is inside a teix:code, where is in the xenoData case, the sharp is inside a tei:code. But there is another example of a '#' inside tei:exemplum/tei:code, namely

<p>In the last case, the identifier includes a non-Unicode character which is defined elsewhere by means of a <gi>glyph</gi> or <gi>char</gi> element referenced here as <code>#sym</code>. </p>

which is in idno.xml. So I'm really confused.

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this. That file has three functions of interest: tei:escapeCharsVerbatim(), tei:escapeChars(), and tei:escapeCharsPartial(). Any one of these functions would convert a '#' to '\#'. So the question, of course, is why isn't one of them being called?

I need to turn my attention elsewhere for awhile, but hope to get back to this by tomorrow late morning (my time).

Ditching the example, at least temporarily, may be the best bet. But the idea that we can never have '#' an an example is chilling.

...

It's a pain, isn't it... (and you have to wonder what sort of an idiot thinks that it would be cool to make up a namespace like that) . But if you want to produce the Guidelines in PDF using LaTeX, either you have to ditch that example, or misrepresent the RDF namespace, or somehow find a way of tweaking the LaTeX code before it gets processed (in LaTeX, I believe, you would simply escape the character with a backslash)

Yeah, it still fails. This has to be because the Stylesheets aren’t properly escaping the hash in that one URL. '#' is completely legal and uncontroversial in URIs and has to be supported—and obviously is, mostly. This has somehow exposed a case where it isn’t...

On Oct 3, 2015, at 13:06 , Hugh Cayless wrote:

Yeah, I’ll have a go. It’s on a VM, so it’ll take me a little while to get it fired up again.

...

On Oct 3, 2015, at 12:25 , Martin Holmes wrote:

Hi Syd,

...

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this.

Curses and more curses. This is what I've been dreading. If it's true that this is never overridden by an XSLT variable being passed in, then I was wrong that Jenkins is no longer dependent (as Sebastian and I believed) on the debs being installed; and that means our own build process is still dependent on the debs being up to date.

Hugh, I believe you built a Jenkins that had no TEI debs installed on it, didn't you? Can it build the PDFs OK?

Cheers, Martin

On 15-10-02 05:22 PM, Syd Bauman wrote:

...

Egads. Ick.

But this doesn't make sense. Surely the character '#' occurs in content elsewhere in the Guidelines.

Just looked: yes, it does, 95 times. E.g.: <code>http://www.example.org/somewhere.xml#p1</code>

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>?

The difference *may* be that in the above case, the sharp is inside a teix:code, where is in the xenoData case, the sharp is inside a tei:code. But there is another example of a '#' inside tei:exemplum/tei:code, namely

<p>In the last case, the identifier includes a non-Unicode character which is defined elsewhere by means of a <gi>glyph</gi> or <gi>char</gi> element referenced here as <code>#sym</code>. </p>

which is in idno.xml. So I'm really confused.

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this. That file has three functions of interest: tei:escapeCharsVerbatim(), tei:escapeChars(), and tei:escapeCharsPartial(). Any one of these functions would convert a '#' to '\#'. So the question, of course, is why isn't one of them being called?

I need to turn my attention elsewhere for awhile, but hope to get back to this by tomorrow late morning (my time).

Ditching the example, at least temporarily, may be the best bet. But the idea that we can never have '#' an an example is chilling.

...

It's a pain, isn't it... (and you have to wonder what sort of an idiot thinks that it would be cool to make up a namespace like that) . But if you want to produce the Guidelines in PDF using LaTeX, either you have to ditch that example, or misrepresent the RDF namespace, or somehow find a way of tweaking the LaTeX code before it gets processed (in LaTeX, I believe, you would simply escape the character with a backslash) -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Yeah. I haven't visually inspected the output, but it doesn't error off. Sent from my phone.

On Oct 3, 2015, at 16:00, Martin Holmes wrote:

If it weren't for that sharp (i.e. if you just remove it temporarily), can you build the PDF without the debs being installed?

Cheers, Martin

...

On 15-10-03 10:20 AM, Hugh Cayless wrote: Yeah, it still fails. This has to be because the Stylesheets aren’t properly escaping the hash in that one URL. '#' is completely legal and uncontroversial in URIs and has to be supported—and obviously is, mostly. This has somehow exposed a case where it isn’t...

...

On Oct 3, 2015, at 13:06 , Hugh Cayless wrote:

Yeah, I’ll have a go. It’s on a VM, so it’ll take me a little while to get it fired up again.

...

On Oct 3, 2015, at 12:25 , Martin Holmes wrote:

Hi Syd,

...

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this.

Curses and more curses. This is what I've been dreading. If it's true that this is never overridden by an XSLT variable being passed in, then I was wrong that Jenkins is no longer dependent (as Sebastian and I believed) on the debs being installed; and that means our own build process is still dependent on the debs being up to date.

Hugh, I believe you built a Jenkins that had no TEI debs installed on it, didn't you? Can it build the PDFs OK?

Cheers, Martin

...

On 15-10-02 05:22 PM, Syd Bauman wrote: Egads. Ick.

But this doesn't make sense. Surely the character '#' occurs in content elsewhere in the Guidelines.

Just looked: yes, it does, 95 times. E.g.: <code>http://www.example.org/somewhere.xml#p1</code>

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>?

The difference *may* be that in the above case, the sharp is inside a teix:code, where is in the xenoData case, the sharp is inside a tei:code. But there is another example of a '#' inside tei:exemplum/tei:code, namely

<p>In the last case, the identifier includes a non-Unicode character which is defined elsewhere by means of a <gi>glyph</gi> or <gi>char</gi> element referenced here as <code>#sym</code>. </p>

which is in idno.xml. So I'm really confused.

Anyway, I just looked at the code, and got a bit queasy. From the .../TEI/P5/Makefile it looks like .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So you better have the Debian packages installed to do this. That file has three functions of interest: tei:escapeCharsVerbatim(), tei:escapeChars(), and tei:escapeCharsPartial(). Any one of these functions would convert a '#' to '\#'. So the question, of course, is why isn't one of them being called?

I need to turn my attention elsewhere for awhile, but hope to get back to this by tomorrow late morning (my time).

Ditching the example, at least temporarily, may be the best bet. But the idea that we can never have '#' an an example is chilling.

...

It's a pain, isn't it... (and you have to wonder what sort of an idiot thinks that it would be cool to make up a namespace like that) . But if you want to produce the Guidelines in PDF using LaTeX, either you have to ditch that example, or misrepresent the RDF namespace, or somehow find a way of tweaking the LaTeX code before it gets processed (in LaTeX, I believe, you would simply escape the character with a backslash) -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

On 15-10-03 03:15 PM, Lou Burnard wrote:

The debs have nothing to do with it. The problem is in the stylesheets, somewhere.

I know; I'm just trying to make sure that we don't have a dependence on the debs for our build servers. Separate issue. Cheers, Martin

On 03/10/15 23:14, Hugh Cayless wrote:

...

Yeah. I haven't visually inspected the output, but it doesn't error off.

Sent from my phone.

...

On Oct 3, 2015, at 16:00, Martin Holmes wrote:

If it weren't for that sharp (i.e. if you just remove it temporarily), can you build the PDF without the debs being installed?

Cheers, Martin

...

On 15-10-03 10:20 AM, Hugh Cayless wrote: Yeah, it still fails. This has to be because the Stylesheets aren’t properly escaping the hash in that one URL. '#' is completely legal and uncontroversial in URIs and has to be supported—and obviously is, mostly. This has somehow exposed a case where it isn’t...

...

On Oct 3, 2015, at 13:06 , Hugh Cayless wrote:

Yeah, I’ll have a go. It’s on a VM, so it’ll take me a little while to get it fired up again.

...

On Oct 3, 2015, at 12:25 , Martin Holmes wrote:

Hi Syd,

> Anyway, I just looked at the code, and got a bit queasy. From the > .../TEI/P5/Makefile it looks like > .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. > That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So > you better have the Debian packages installed to do this. Curses and more curses. This is what I've been dreading. If it's true that this is never overridden by an XSLT variable being passed in, then I was wrong that Jenkins is no longer dependent (as Sebastian and I believed) on the debs being installed; and that means our own build process is still dependent on the debs being up to date.

Hugh, I believe you built a Jenkins that had no TEI debs installed on it, didn't you? Can it build the PDFs OK?

Cheers, Martin

> On 15-10-02 05:22 PM, Syd Bauman wrote: > Egads. Ick. > > But this doesn't make sense. Surely the character '#' occurs in > content > elsewhere in the Guidelines. > > Just looked: yes, it does, 95 times. E.g.: > <code>http://www.example.org/somewhere.xml#p1</code> > > So what's different between the <code> in xenoData.xml which > contains > a sharp, and the one above? Or better yet, between it and > <code>personography.xml#MDH</code> > which also occurs inside an <exemplum>? > > The difference *may* be that in the above case, the sharp is > inside a > teix:code, where is in the xenoData case, the sharp is inside a > tei:code. But there is another example of a '#' inside > tei:exemplum/tei:code, namely > > <p>In the last case, the identifier includes a non-Unicode > character which is defined elsewhere by means of a <gi>glyph</gi> > or <gi>char</gi> element referenced here as <code>#sym</code>. > </p> > > which is in idno.xml. So I'm really confused. > > > Anyway, I just looked at the code, and got a bit queasy. From the > .../TEI/P5/Makefile it looks like > .../TEI/P5/Utilities/guidelines-latex.xsl is run from an ant file. > That file imports /usr/share/xml/tei/stylesheet/latex/latex.xsl. So > you better have the Debian packages installed to do this. That file > has three functions of interest: tei:escapeCharsVerbatim(), > tei:escapeChars(), and tei:escapeCharsPartial(). Any one of these > functions would convert a '#' to '\#'. So the question, of > course, is > why isn't one of them being called? > > I need to turn my attention elsewhere for awhile, but hope to get > back to this by tomorrow late morning (my time). > > Ditching the example, at least temporarily, may be the best bet. But > the idea that we can never have '#' an an example is chilling. > > >> It's a pain, isn't it... (and you have to wonder what sort of >> an idiot >> thinks that it would be cool to make up a namespace like that) . >> But if >> you want to produce the Guidelines in PDF using LaTeX, either >> you have >> to ditch that example, or misrepresent the RDF namespace, or >> somehow >> find a way of tweaking the LaTeX code before it gets processed (in >> LaTeX, I believe, you would simply escape the character with a >> backslash) -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

On 03/10/15 01:22, Syd Bauman wrote:

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>?

The problem isn't caused by the presence of an unescaped # in the ODD source which, as you rightly observe, happens all over the place. The problem is that the namespace declarations you have added in order to make your example code valid are getting copied into the output. Suppose, for example, you had some text <egXML xmlns="http://www.tei-c.org/ns/1.0" xmlns:foo ="http://www.example.foo"> <p>foo:whatever something </p> </egXML> The resulting example has to look like sometext <p xmlns:foo ="http://www.example.foo"> foo:whatever something </p> Whatever it is that copies the namespace declaration from the egXML element into the root element of the egXML content doesn't know to do the escaping. Indeed, it probably shouldn't. Regrettably, you don't have the option of simply removing the additional namespace declarations from the <exemplum> as your examples then look ill-formed. (And no, it makes no difference whether the declarations are on the exemplum or the egXML: they still have to be copied into the output) Hmmmm.

Just committed a fix, I think. We’ll see if Mr. Jenkins blows up. Local builds of the PDF Guidelines work again anyway, with the hash in the RDF namespace.

On Oct 3, 2015, at 19:14 , Lou Burnard wrote:

OK, I have now found what seems to be the relevant bit of the stylesheets (it's in common/verbatim.xsl)

There is a parameter which controls whether or not namespace declarations are copied into the start of a verbatim example. By default they are; however by setting the parameter "false" I can make this problem go away, at the small space of removing all such namespace declarations in the output.

Going to sleep on it now.

On 03/10/15 18:19, Lou Burnard wrote:

...

On 03/10/15 01:22, Syd Bauman wrote:

...

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>?

The problem isn't caused by the presence of an unescaped # in the ODD source which, as you rightly observe, happens all over the place. The problem is that the namespace declarations you have added in order to make your example code valid are getting copied into the output.

Suppose, for example, you had

some text <egXML xmlns="http://www.tei-c.org/ns/1.0" xmlns:foo ="http://www.example.foo"> <p>foo:whatever something </p> </egXML>

The resulting example has to look like

sometext <p xmlns:foo ="http://www.example.foo"> foo:whatever something </p>

Whatever it is that copies the namespace declaration from the egXML element into the root element of the egXML content doesn't know to do the escaping. Indeed, it probably shouldn't.

Regrettably, you don't have the option of simply removing the additional namespace declarations from the <exemplum> as your examples then look ill-formed.

(And no, it makes no difference whether the declarations are on the exemplum or the egXML: they still have to be copied into the output)

Hmmmm.

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

In the bad old days of subversion, if I were (as I am) in the situation where I had made some changes locally in files which I now wish to junk and replace with the changes that have meantime been made in the repo, I would simply delete my modified local versions, and then do "svn up". What hoops do I have to jump through to get the same effect with git? On 04/10/15 17:33, Lou Burnard wrote:

Thanks for doing the job properly !

Now, please remind me how to update my local branch... if I do "git pull" it says

From https://github.com/TEIC/Stylesheets 05c9755..2dca741 master -> origin/master Already up-to-date.

But I know this is not the case. I tried "git merge" in my local branch but got:

fatal: No commit specified and merge.defaultToUpstream not set.

(I have another copy of the Stylesheets directory, checked out with svn. I go there and type "svn up" and bingo, the two files you have changed get changed locally. Why does everything with git have to be so much more complicated?)

On 04/10/15 15:56, Hugh Cayless wrote:

...

Just committed a fix, I think. We’ll see if Mr. Jenkins blows up. Local builds of the PDF Guidelines work again anyway, with the hash in the RDF namespace.

...

On Oct 3, 2015, at 19:14 , Lou Burnard wrote:

OK, I have now found what seems to be the relevant bit of the stylesheets (it's in common/verbatim.xsl)

There is a parameter which controls whether or not namespace declarations are copied into the start of a verbatim example. By default they are; however by setting the parameter "false" I can make this problem go away, at the small space of removing all such namespace declarations in the output.

Going to sleep on it now.

On 03/10/15 18:19, Lou Burnard wrote:

...

...

So what's different between the <code> in xenoData.xml which contains a sharp, and the one above? Or better yet, between it and <code>personography.xml#MDH</code> which also occurs inside an <exemplum>? The problem isn't caused by the presence of an unescaped # in the ODD source which, as you rightly observe, happens all over the

On 03/10/15 01:22, Syd Bauman wrote: place. The problem is that the namespace declarations you have added in order to make your example code valid are getting copied into the output.

Suppose, for example, you had

some text <egXML xmlns="http://www.tei-c.org/ns/1.0" xmlns:foo ="http://www.example.foo"> <p>foo:whatever something </p> </egXML>

The resulting example has to look like

sometext <p xmlns:foo ="http://www.example.foo"> foo:whatever something </p>

Whatever it is that copies the namespace declaration from the egXML element into the root element of the egXML content doesn't know to do the escaping. Indeed, it probably shouldn't.

Regrettably, you don't have the option of simply removing the additional namespace declarations from the <exemplum> as your examples then look ill-formed.

(And no, it makes no difference whether the declarations are on the exemplum or the egXML: they still have to be copied into the output)

Hmmmm.

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

I think it is a bug too. You should be able to get a working schema by including all the modules that include all the elements you want. Cheers, Martin On 15-10-02 08:29 AM, Lou Burnard wrote:

So we need a quick show of hands as to whether we consider this[1] to be a bug or not[2].

[1] i.e. the fact that you won't get a satisfactory schema if you (a) select the header module and (b) do not select the tagdocs module and (c) expect to be able to use <xenodata>

[2] I think it is.

On 02/10/15 16:17, Hugh Cayless wrote:

...

We’re in our freeze period, so bugfixes and typos should be corrected in master. Nothing else should be happening there.

...

On Oct 2, 2015, at 11:12 , Syd Bauman wrote:

Hmmm ... so you're suggesting that if we leave things as they are, then an ODD that had just

<schemaSpec> <moduleRef key="tei"/> <moduleRef key="header"/> <moduleRef key="core"/> <moduleRef key="textstructure"/> </schemaSpec>

would produce incorrect schemas because <xenoData> of 'header' wants macro.anyXML which will not be there?

Well, I just tried it, and you are absolutely right. (The good news is that the ODD processor does not generate an invalid schema. The bad news is that, as you predicted, it does not produce the right schema, either.)

I will fix this as soon as I hear from Hugh as to whether I should fix it in sydb-xenodata, master, or do something else.

...

The anyXML thing is not quite as you describe it. if the macro is declared in the tagdocs module, as it currently is, then you won't get it unless you include that module in your schema. If you want to use it in xenodata, then you need to declare it in the infrastructure module. the @predeclare thing is not really relevant: it is a legacy piece of trickery whjich we should not be using. Just move the declaration to the infrastructure module and add the appropriate text into chapter ST would be my recommendation. -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived