It was precisely this kind of worry that made me not just merge it myself. I've not got time in the next couple days to attempt to do so, maybe at the weekend. -James On 18/08/15 23:58, Martin Holmes wrote:

I looked at this, but I'm by no means confident that I could assess the impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

...

Forwarding this reminder to tei-council to remind them that they need to keep a watching eye on such pull requests. (Once set up, we should get notifications sent to tei-council if possible)

-James

-------- Forwarded Message -------- Subject: Re: editing the TEI stylesheets Date: Wed, 19 Aug 2015 09:43:31 +1200 From: Stuart A. Yeates To: James.Cummings@it.ox.ac.uk CC: Sebastian Rahtz

OK, so I sent a pull request almost a week ago https://github.com/TEIC/Stylesheets/pull/114

Do I need to notify someone? Or has feedback gone somewhere I'm missing...

cheers stuart

-- ...let us be heard from red core to black sky

On Wed, Aug 12, 2015 at 10:01 PM, James Cummings mailto:James.Cummings@it.ox.ac.uk> wrote:

Hi Stuart,

The TEI Technical Council is currently reviewing how it uses GitHub as part of examining whether it is a suitable replacement for SourceForge in moving Guidelines development there. As you know, it moved the Stylesheets (and other software) building there some time ago. Although you (and various others) are members of the Stylesheets repository I believe the correct github etiquette is to fork, then issue a pull request. However, I think as long as you are flagging that you are making the change (or open an issue and then make the change as a result of that) it should probably be fine. (Though when you say 'regularise the display of attribute values' I'd be interested to know what you mean by regularise. There are some options and logic already in there.) The change in your forked version won't get looked at by Jenkins. We're also looking at whether some of the GitHub hosted alternative continuous integration systems might replace Jenkins. So, to be honest, you could do either, but fork+change+pull-request will ensure that it gets other eyes on it before going into the system. Jenkins uses the most up-to-date stylesheets when next building the Guidelines. I know it used to email when doing commits to the Guidelines SF SVN repository that broke something (or more accurately when someone committing around the same time as you broke something ;-) ) and I'm assuming that is still the case. I don't think it emailed just when it was running a job based on your commits...just when that job failed and your commit was one of those possible causes.

That is my understanding at least, some of our council github gurus may correct me.

-James

On 12/08/15 10:26, Stuart A. Yeates wrote:

I've got a change I'd like to make to the stylesheets used to generate the HTML version of the standard (a minor change to regularise the display of attribute values), but I'm not quite sure how to go about it.

Do I just push a change request to https://github.com/TEIC/Stylesheets ? Or fork it? Does that get tested by jenkins?

Jenkins used to send me emails when it ran a job based on my commits, but I've not seen one for a couple of recent commits, is that because I'm not on the council any more?

cheers stuart -- ...let us be heard from red core to black sky

-- Dr James Cummings, James.Cummings@it.ox.ac.uk mailto:James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

-- Dr James Cummings, James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

Hi Hugh, On 15-08-19 05:33 AM, Hugh Cayless wrote:

Yeah, we do need a protocol for this. I can't claim to understand the Stylesheets either, but I also don't understand our test infrastructure. For example, running "make test" in P5/ clearly runs some tests, but not all of them. I can do that and then check stuff in and have Jenkins yell at me a few minutes later. So how does one run a comprehensive suite of tests? Does it require a Jenkins install?

The Stylesheets project on Jenkins simply runs: make clean test deb dist The configuration for it is in our repo here: http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... So you did everything Jenkins did. However, after the build, Jenkins does something called Build Log Parsing, which uses a log file parsing configuration here: http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... to decide whether to treat specific things as errors that break the build or warnings that make it unstable, or to ignore other things. As a result, you see for example this: http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/ which (if there were errors in this particular build) would highlight them. To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I have to scroll back through the output to see that at least one test is failing—no surprise that it's related to making PDFs, which doesn't work outside our bespoke Linux environment.

LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests, but it's terrible, and terrifying(!) that they're not generally usable.

I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I would really like to get us into a place where normal humans (or even most Council members) could edit the Guidelines or Stylesheets and test their changes before pushing them. Jenkins is great, but it's a backstop (or feels to me like it should be, anyway).

What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it. We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible. Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time. Does anyone else have difficulty with this stuff?

I expect Stuart's pull request went past me last week while I was away at a conference. I've commented and asked him for some clarification.

On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes wrote:

...

I looked at this, but I'm by no means confident that I could assess the impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

...

Forwarding this reminder to tei-council to remind them that they need to keep a watching eye on such pull requests. (Once set up, we should get notifications sent to tei-council if possible)

-James

-------- Forwarded Message -------- Subject: Re: editing the TEI stylesheets Date: Wed, 19 Aug 2015 09:43:31 +1200 From: Stuart A. Yeates To: James.Cummings@it.ox.ac.uk CC: Sebastian Rahtz

OK, so I sent a pull request almost a week ago https://github.com/TEIC/Stylesheets/pull/114

Do I need to notify someone? Or has feedback gone somewhere I'm missing...

cheers stuart

-- ...let us be heard from red core to black sky

On Wed, Aug 12, 2015 at 10:01 PM, James Cummings mailto:James.Cummings@it.ox.ac.uk> wrote:

Hi Stuart,

The TEI Technical Council is currently reviewing how it uses GitHub as part of examining whether it is a suitable replacement for SourceForge in moving Guidelines development there. As you know, it moved the Stylesheets (and other software) building there some time ago. Although you (and various others) are members of the Stylesheets repository I believe the correct github etiquette is to fork, then issue a pull request. However, I think as long as you are flagging that you are making the change (or open an issue and then make the change as a result of that) it should probably be fine. (Though when you say 'regularise the display of attribute values' I'd be interested to know what you mean by regularise. There are some options and logic already in there.) The change in your forked version won't get looked at by Jenkins. We're also looking at whether some of the GitHub hosted alternative continuous integration systems might replace Jenkins. So, to be honest, you could do either, but fork+change+pull-request will ensure that it gets other eyes on it before going into the system. Jenkins uses the most up-to-date stylesheets when next building the Guidelines. I know it used to email when doing commits to the Guidelines SF SVN repository that broke something (or more accurately when someone committing around the same time as you broke something ;-) ) and I'm assuming that is still the case. I don't think it emailed just when it was running a job based on your commits...just when that job failed and your commit was one of those possible causes.

That is my understanding at least, some of our council github gurus may correct me.

-James

On 12/08/15 10:26, Stuart A. Yeates wrote:

I've got a change I'd like to make to the stylesheets used to generate the HTML version of the standard (a minor change to regularise the display of attribute values), but I'm not quite sure how to go about it.

Do I just push a change request to https://github.com/TEIC/Stylesheets ? Or fork it? Does that get tested by jenkins?

Jenkins used to send me emails when it ran a job based on my commits, but I've not seen one for a couple of recent commits, is that because I'm not on the council any more?

cheers stuart -- ...let us be heard from red core to black sky

-- Dr James Cummings, James.Cummings@it.ox.ac.uk mailto:James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Hi Hugh, On 15-08-19 06:01 AM, Hugh Cayless wrote:

Thanks for the info Martin. I guess I'm used to systems like Rake, where I run the test suite and get a nice little report in my console telling me what tests succeeded, what failed, and what raised an error.

That makes perfect sense. That's what we use Jenkins for, but perhaps if we have a Council that predominantly prefers to get a working build environment locally, and only commit when they're comfortable, then implementing our own build log parsing would make sense. In principle, anything that generates "error" or "warning" needs attention; but in practice, as you can see from the scale of the log parse rules, there are all sorts of things that need to be suppressed and ignored. For example, the line: No errors or warnings detected contains both "errors" and "warnings", so the build would fail if we didn't trap for it: ok /No errors or warnings detected/ Similarly, Exiting with failure status does not contain "error" or "warning", so it would go by unnoticed if we didn't trap for that: error /Exiting with failure status/ The issue of LaTeX is a very messy business; you have to install a great deal of stuff to get that to work, and it's probably more than any of us wants running on our laptops unless we're big LaTeX users.

The problem I have with Jenkins is this: I (thinking I'm done) commit something, it tells me a few minutes later that I broke the build for *everyone* and anything they commit subsequently will also fail until I fix the problem. I panic and rush to fix the problem. Personally, I'd vastly prefer to be able to run the test suite locally so that I know that I won't mess up everyone else's workflow when I commit stuff.

Maybe I'm just strange though...

No, I see that. How exactly does Rake produce its report? I'm not familiar with it. Cheers, Martin

On Wed, Aug 19, 2015 at 8:46 AM, Martin Holmes wrote:

...

Hi Hugh,

On 15-08-19 05:33 AM, Hugh Cayless wrote:

...

Yeah, we do need a protocol for this. I can't claim to understand the Stylesheets either, but I also don't understand our test infrastructure. For example, running "make test" in P5/ clearly runs some tests, but not all of them. I can do that and then check stuff in and have Jenkins yell at me a few minutes later. So how does one run a comprehensive suite of tests? Does it require a Jenkins install?

The Stylesheets project on Jenkins simply runs:

make clean test deb dist

The configuration for it is in our repo here:

< http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

So you did everything Jenkins did. However, after the build, Jenkins does something called Build Log Parsing, which uses a log file parsing configuration here:

< http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

to decide whether to treat specific things as errors that break the build or warnings that make it unstable, or to ignore other things. As a result, you see for example this:

http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/

which (if there were errors in this particular build) would highlight them.

To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I

...

have to scroll back through the output to see that at least one test is failing—no surprise that it's related to making PDFs, which doesn't work outside our bespoke Linux environment.

LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests,

...

but it's terrible, and terrifying(!) that they're not generally usable.

I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I

...

would really like to get us into a place where normal humans (or even most Council members) could edit the Guidelines or Stylesheets and test their changes before pushing them. Jenkins is great, but it's a backstop (or feels to me like it should be, anyway).

What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it.

We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible.

Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time.

...

Does anyone else have difficulty with this stuff?

I expect Stuart's pull request went past me last week while I was away at a

...

conference. I've commented and asked him for some clarification.

On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes wrote:

I looked at this, but I'm by no means confident that I could assess the

...

impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

...

Forwarding this reminder to tei-council to remind them that they need to keep a watching eye on such pull requests. (Once set up, we should get notifications sent to tei-council if possible)

-James

-------- Forwarded Message -------- Subject: Re: editing the TEI stylesheets Date: Wed, 19 Aug 2015 09:43:31 +1200 From: Stuart A. Yeates To: James.Cummings@it.ox.ac.uk CC: Sebastian Rahtz

OK, so I sent a pull request almost a week ago https://github.com/TEIC/Stylesheets/pull/114

Do I need to notify someone? Or has feedback gone somewhere I'm missing...

cheers stuart

-- ...let us be heard from red core to black sky

On Wed, Aug 12, 2015 at 10:01 PM, James Cummings mailto:James.Cummings@it.ox.ac.uk> wrote:

Hi Stuart,

The TEI Technical Council is currently reviewing how it uses GitHub as part of examining whether it is a suitable replacement for SourceForge in moving Guidelines development there. As you know, it moved the Stylesheets (and other software) building there some time ago. Although you (and various others) are members of the Stylesheets repository I believe the correct github etiquette is to fork, then issue a pull request. However, I think as long as you are flagging that you are making the change (or open an issue and then make the change as a result of that) it should probably be fine. (Though when you say 'regularise the display of attribute values' I'd be interested to know what you mean by regularise. There are some options and logic already in there.) The change in your forked version won't get looked at by Jenkins. We're also looking at whether some of the GitHub hosted alternative continuous integration systems might replace Jenkins. So, to be honest, you could do either, but fork+change+pull-request will ensure that it gets other eyes on it before going into the system. Jenkins uses the most up-to-date stylesheets when next building the Guidelines. I know it used to email when doing commits to the Guidelines SF SVN repository that broke something (or more accurately when someone committing around the same time as you broke something ;-) ) and I'm assuming that is still the case. I don't think it emailed just when it was running a job based on your commits...just when that job failed and your commit was one of those possible causes.

That is my understanding at least, some of our council github gurus may correct me.

-James

On 12/08/15 10:26, Stuart A. Yeates wrote:

I've got a change I'd like to make to the stylesheets used to generate the HTML version of the standard (a minor change to regularise the display of attribute values), but I'm not quite sure how to go about it.

Do I just push a change request to https://github.com/TEIC/Stylesheets ? Or fork it? Does that get tested by jenkins?

Jenkins used to send me emails when it ran a job based on my commits, but I've not seen one for a couple of recent commits, is that because I'm not on the council any more?

cheers stuart -- ...let us be heard from red core to black sky

-- Dr James Cummings, James.Cummings@it.ox.ac.uk mailto:James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

HI Hugh, On 15-08-19 08:33 AM, Hugh Cayless wrote:

It produces output like:

# Running tests:

..........................................................................................

Finished tests in 136.596000s, 0.6589 tests/s, 3.7922 assertions/s.

90 tests, 518 assertions, 0 failures, 0 errors, 0 skips

as a test runs, a "." is output if it succeeds, an "F" if it fails, or an "E" if it throws an exception.

I think I'd like better output than that -- I'd like to see which test failed, and if (for example) it's a validity test, where the error is in the file, and what the error message is.

Not saying we should re-write all our tests in Ruby,

No, don't say that.:-)

but it would be very nice to be able to run the test suite locally and get sensible outputs. Would be good too to be able to skip tests that are known to fail, like PDF generation. Incidentally, I *almost* have that working on my laptop...not sure I can remember all the steps to make it work though.

It's a bit outdated now, but the Jenkins build script should cover everything needed on Linux: http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... In particular, I think you need all the fonts, one of which (Hannom) has to be downloaded and unzipped from a SourceForge project; you need all the *tex* stuff; and there's probably a couple of other dependencies in there that aren't completely obvious. I think we ought to have a conversation about LaTeX in the build, though. I don't think we have any LaTeX experts on Council at the moment, do we? Cheers, Martin

On Wed, Aug 19, 2015 at 11:20 AM, Martin Holmes wrote:

...

Hi Hugh,

On 15-08-19 06:01 AM, Hugh Cayless wrote:

...

Thanks for the info Martin. I guess I'm used to systems like Rake, where I run the test suite and get a nice little report in my console telling me what tests succeeded, what failed, and what raised an error.

That makes perfect sense. That's what we use Jenkins for, but perhaps if we have a Council that predominantly prefers to get a working build environment locally, and only commit when they're comfortable, then implementing our own build log parsing would make sense. In principle, anything that generates "error" or "warning" needs attention; but in practice, as you can see from the scale of the log parse rules, there are all sorts of things that need to be suppressed and ignored. For example, the line:

No errors or warnings detected

contains both "errors" and "warnings", so the build would fail if we didn't trap for it:

ok /No errors or warnings detected/

Similarly,

Exiting with failure status

does not contain "error" or "warning", so it would go by unnoticed if we didn't trap for that:

error /Exiting with failure status/

The issue of LaTeX is a very messy business; you have to install a great deal of stuff to get that to work, and it's probably more than any of us wants running on our laptops unless we're big LaTeX users.

The problem I have with Jenkins is this: I (thinking I'm done) commit

...

something, it tells me a few minutes later that I broke the build for *everyone* and anything they commit subsequently will also fail until I fix the problem. I panic and rush to fix the problem. Personally, I'd vastly prefer to be able to run the test suite locally so that I know that I won't mess up everyone else's workflow when I commit stuff.

Maybe I'm just strange though...

No, I see that. How exactly does Rake produce its report? I'm not familiar with it.

Cheers, Martin

On Wed, Aug 19, 2015 at 8:46 AM, Martin Holmes wrote:

...

Hi Hugh,

...

On 15-08-19 05:33 AM, Hugh Cayless wrote:

Yeah, we do need a protocol for this. I can't claim to understand the

...

Stylesheets either, but I also don't understand our test infrastructure. For example, running "make test" in P5/ clearly runs some tests, but not all of them. I can do that and then check stuff in and have Jenkins yell at me a few minutes later. So how does one run a comprehensive suite of tests? Does it require a Jenkins install?

The Stylesheets project on Jenkins simply runs:

make clean test deb dist

The configuration for it is in our repo here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

So you did everything Jenkins did. However, after the build, Jenkins does something called Build Log Parsing, which uses a log file parsing configuration here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

to decide whether to treat specific things as errors that break the build or warnings that make it unstable, or to ignore other things. As a result, you see for example this:

http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/

which (if there were errors in this particular build) would highlight them.

To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I

...

have to scroll back through the output to see that at least one test is failing—no surprise that it's related to making PDFs, which doesn't work outside our bespoke Linux environment.

LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests,

...

but it's terrible, and terrifying(!) that they're not generally usable.

I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I

...

would really like to get us into a place where normal humans (or even most Council members) could edit the Guidelines or Stylesheets and test their changes before pushing them. Jenkins is great, but it's a backstop (or feels to me like it should be, anyway).

What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it.

We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible.

Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time.

...

Does anyone else have difficulty with this stuff?

I expect Stuart's pull request went past me last week while I was away at a

...

conference. I've commented and asked him for some clarification.

On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes wrote:

I looked at this, but I'm by no means confident that I could assess the

...

impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

> Forwarding this reminder to tei-council to remind them that they need > to > keep a watching eye on such pull requests. (Once set up, we should > get > notifications sent to tei-council if possible) > > -James > > > -------- Forwarded Message -------- > Subject: Re: editing the TEI stylesheets > Date: Wed, 19 Aug 2015 09:43:31 +1200 > From: Stuart A. Yeates > To: James.Cummings@it.ox.ac.uk > CC: Sebastian Rahtz > > > > OK, so I sent a pull request almost a week ago > https://github.com/TEIC/Stylesheets/pull/114 > > Do I need to notify someone? Or has feedback gone somewhere I'm > missing... > > cheers > stuart > > -- > ...let us be heard from red core to black sky > > On Wed, Aug 12, 2015 at 10:01 PM, James Cummings > mailto:James.Cummings@it.ox.ac.uk> > wrote: > > > Hi Stuart, > > The TEI Technical Council is currently reviewing how it uses > GitHub as part of examining whether it is a suitable > replacement for SourceForge in moving Guidelines development > there. As you know, it moved the Stylesheets (and other > software) building there some time ago. Although you (and > various others) are members of the Stylesheets repository I > believe the correct github etiquette is to fork, then issue a > pull request. However, I think as long as you are flagging > that you are making the change (or open an issue and then > make the change as a result of that) it should probably be > fine. (Though when you say 'regularise the display of > attribute values' I'd be interested to know what you mean by > regularise. There are some options and logic already in > there.) The change in your forked version won't get looked at > by Jenkins. We're also looking at whether some of the GitHub > hosted alternative continuous integration systems might > replace Jenkins. So, to be honest, you could do either, but > fork+change+pull-request will ensure that it gets other eyes > on it before going into the system. Jenkins uses the most > up-to-date stylesheets when next building the Guidelines. I > know it used to email when doing commits to the Guidelines SF > SVN repository that broke something (or more accurately when > someone committing around the same time as you broke > something ;-) ) and I'm assuming that is still the case. I > don't think it emailed just when it was running a job based > on your commits...just when that job failed and your commit > was one of those possible causes. > > That is my understanding at least, some of our council github > gurus may correct me. > > -James > > > > On 12/08/15 10:26, Stuart A. Yeates wrote: > > I've got a change I'd like to make to the stylesheets > used to generate the HTML version of the standard (a > minor change to regularise the display of attribute > values), but I'm not quite sure how to go about it. > > Do I just push a change request to > https://github.com/TEIC/Stylesheets ? Or fork it? Does > that get tested by jenkins? > > Jenkins used to send me emails when it ran a job based on > my commits, but I've not seen one for a couple of recent > commits, is that because I'm not on the council any more? > > cheers > stuart > -- > ...let us be heard from red core to black sky > > > > -- Dr James Cummings, James.Cummings@it.ox.ac.uk > mailto:James.Cummings@it.ox.ac.uk > Academic IT Services, University of Oxford > > > > > > -- > > tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

Hi Hugh,

I’m certainly not one. The thought of redoing the PDF generation in FOP fills me with terror though. I haven’t touched FOP in almost a decade…

Actually I'm very fond of XSL:FO, and FOP is looking pretty good these days; the only areas where it lags behind commercial alternatives like XEP are pretty obscure things that I don't think we would care about. I'd enjoy rewriting the PDF output with XSL:FO, but I think I'd need to be convinced that we actually need the PDF output.

Syd and I saw an interesting paper at Balisage that might point toward a third way: http://www.balisage.net/Proceedings/vol15/html/Wilm01/BalisageVol15-Wilm01.h... http://www.balisage.net/Proceedings/vol15/html/Wilm01/BalisageVol15-Wilm01.h...

This looks interesting. The approach goes back a long way; I seem to recall one of the original authors of Opera was working quite intensively on CSS for print output at one point. We could get a very long way with pure CSS for @media print, actually; support for paged media has been getting better and better, both in CSS specs and in browsers. Perhaps that would be the best way to start. Cheers, Martin

I see now that the trick is running the pdf-generation process more than once, so it picks up the cross-references. Seems to have done the trick, though I haven’t tried "make dist" again yet.

On 15-08-19 09:44 AM, Hugh Cayless wrote:

...

On Aug 19, 2015, at 11:51 , Martin Holmes wrote:

HI Hugh,

On 15-08-19 08:33 AM, Hugh Cayless wrote:

...

It produces output like:

# Running tests:

..........................................................................................

...

...

90 tests, 518 assertions, 0 failures, 0 errors, 0 skips

as a test runs, a "." is output if it succeeds, an "F" if it fails, or an "E" if it throws an exception.

I think I'd like better output than that -- I'd like to see which test failed, and if (for example) it's a validity test, where the error is in the file, and what the error message is.

It gives you more if there’s a failure or error. All of ours pass at the moment :-). The point is that you can see immediately if there’s a problem without having to dig. Perhaps we could wrap make test in a script that summarizes & logs the output...

...

...

Not saying we should re-write all our tests in Ruby,

No, don't say that.:-)

...

but it would be very nice to be able to run the test suite locally and get sensible outputs. Would be good too to be able to skip tests that are known to fail, like PDF generation. Incidentally, I *almost* have that working on my laptop...not sure I can remember all the steps to make it work though.

It's a bit outdated now, but the Jenkins build script should cover everything needed on Linux:

<http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...>

In particular, I think you need all the fonts, one of which (Hannom) has to be downloaded and unzipped from a SourceForge project; you need all

Finished tests in 136.596000s, 0.6589 tests/s, 3.7922 assertions/s. the *tex* stuff; and there's probably a couple of other dependencies in there that aren't completely obvious.

...

I think we ought to have a conversation about LaTeX in the build, though. I don't think we have any LaTeX experts on Council at the moment, do we?

I’m certainly not one. The thought of redoing the PDF generation in FOP fills me with terror though. I haven’t touched FOP in almost a decade…Syd and I saw an interesting paper at Balisage that might point toward a third way: http://www.balisage.net/Proceedings/vol15/html/Wilm01/BalisageVol15-Wilm01.h... http://www.balisage.net/Proceedings/vol15/html/Wilm01/BalisageVol15-Wilm01.h...

I see now that the trick is running the pdf-generation process more than once, so it picks up the cross-references. Seems to have done the trick, though I haven’t tried "make dist" again yet.

...

Cheers, Martin

...

On Wed, Aug 19, 2015 at 11:20 AM, Martin Holmes wrote:

...

Hi Hugh,

On 15-08-19 06:01 AM, Hugh Cayless wrote:

...

Thanks for the info Martin. I guess I'm used to systems like Rake, where I run the test suite and get a nice little report in my console telling me what tests succeeded, what failed, and what raised an error.

That makes perfect sense. That's what we use Jenkins for, but perhaps if we have a Council that predominantly prefers to get a working build environment locally, and only commit when they're comfortable, then implementing our own build log parsing would make sense. In principle, anything that generates "error" or "warning" needs attention; but in practice, as you can see from the scale of the log parse rules, there are all sorts of things that need to be suppressed and ignored. For example, the line:

No errors or warnings detected

contains both "errors" and "warnings", so the build would fail if we didn't trap for it:

ok /No errors or warnings detected/

Similarly,

Exiting with failure status

does not contain "error" or "warning", so it would go by unnoticed if we didn't trap for that:

error /Exiting with failure status/

The issue of LaTeX is a very messy business; you have to install a great deal of stuff to get that to work, and it's probably more than any of us wants running on our laptops unless we're big LaTeX users.

The problem I have with Jenkins is this: I (thinking I'm done) commit

...

something, it tells me a few minutes later that I broke the build for *everyone* and anything they commit subsequently will also fail until I fix the problem. I panic and rush to fix the problem. Personally, I'd vastly prefer to be able to run the test suite locally so that I know that I won't mess up everyone else's workflow when I commit stuff.

Maybe I'm just strange though...

No, I see that. How exactly does Rake produce its report? I'm not familiar with it.

Cheers, Martin

On Wed, Aug 19, 2015 at 8:46 AM, Martin Holmes wrote:

...

Hi Hugh,

...

On 15-08-19 05:33 AM, Hugh Cayless wrote:

Yeah, we do need a protocol for this. I can't claim to understand the > Stylesheets either, but I also don't understand our test > infrastructure. For example, running "make test" in P5/ > clearly runs some tests, but not all of them. I can do > that and then check stuff in and have Jenkins yell at me > a few minutes later. So how does one run a comprehensive > suite of tests? Does it require a Jenkins install? > > The Stylesheets project on Jenkins simply runs:

make clean test deb dist

The configuration for it is in our repo here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

> >

So you did everything Jenkins did. However, after the build, Jenkins does

...

...

...

...

...

something called Build Log Parsing, which uses a log file parsing configuration here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

> >

to decide whether to treat specific things as errors that break the build

...

or warnings that make it unstable, or to ignore other things. As a result, you see for example this:

http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/

which (if there were errors in this particular build) would highlight

...

them.

To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I

> have to scroll back through the output to see that at > least one test is failing—no surprise that it's related > to making PDFs, which doesn't work outside our bespoke > Linux environment. > > LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests,

> but it's terrible, and terrifying(!) that they're not > generally usable. > > I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I

> would really like to get us into a place where normal > humans (or even most Council members) could edit the > Guidelines or Stylesheets and test their changes before > pushing them. Jenkins is great, but it's a backstop (or > feels to me like it should be, anyway). > > What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it.

We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible.

Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time.

> Does anyone else have difficulty with this stuff? > >

I expect Stuart's pull request went past me last week while I was away at a

> conference. I've commented and asked him for some > clarification. > > > > On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes > wrote: > > I looked at this, but I'm by no means confident that I > could assess the > >> impact of a merge like this without investing a fair >> amount of time figuring out what it does and how it >> works. The Stylesheets are still fairly mysterious to >> me, except for the bits I happen to have looked at. >> >> I think what we actually need is a protocol for this. >> So, for instance, before accepting a pull request that >> is anything more than a typo fix, at least check out >> the branch and run all the regular tests; then perhaps >> create one more test that checks the bug that's being >> fixed; then go ahead and merge. >> >> Cheers, Martin >> >> >> On 15-08-18 02:52 PM, James Cummings wrote: >> >> >> >>> Forwarding this reminder to tei-council to remind >>> them that they need to keep a watching eye on such >>> pull requests. (Once set up, we should get >>> notifications sent to tei-council if possible) >>> >>> -James >>> >>> >>> -------- Forwarded Message -------- Subject: Re: >>> editing the TEI stylesheets Date: Wed, 19 Aug >>> 2015 09:43:31 +1200 From: Stuart A. Yeates >>> To: >>> James.Cummings@it.ox.ac.uk >>> CC: Sebastian Rahtz >>> >>> >>> >>> >>> OK, so I sent a pull request almost a week ago >>> https://github.com/TEIC/Stylesheets/pull/114 >>> >>> Do I need to notify someone? Or has feedback gone >>> somewhere I'm missing... >>> >>> cheers stuart >>> >>> -- ...let us be heard from red core to black sky >>> >>> On Wed, Aug 12, 2015 at 10:01 PM, James Cummings >>> >> mailto:James.Cummings@it.ox.ac.uk> wrote: >>> >>> >>> Hi Stuart, >>> >>> The TEI Technical Council is currently reviewing how >>> it uses GitHub as part of examining whether it is a >>> suitable replacement for SourceForge in moving >>> Guidelines development there. As you know, it moved >>> the Stylesheets (and other software) building there >>> some time ago. Although you (and various others) are >>> members of the Stylesheets repository I believe the >>> correct github etiquette is to fork, then issue a >>> pull request. However, I think as long as you are >>> flagging that you are making the change (or open an >>> issue and then make the change as a result of that) >>> it should probably be fine. (Though when you say >>> 'regularise the display of attribute values' I'd be >>> interested to know what you mean by regularise. There >>> are some options and logic already in there.) The >>> change in your forked version won't get looked at by >>> Jenkins. We're also looking at whether some of the >>> GitHub hosted alternative continuous integration >>> systems might replace Jenkins. So, to be honest, you >>> could do either, but fork+change+pull-request will >>> ensure that it gets other eyes on it before going >>> into the system. Jenkins uses the most up-to-date >>> stylesheets when next building the Guidelines. I know >>> it used to email when doing commits to the Guidelines >>> SF SVN repository that broke something (or more >>> accurately when someone committing around the same >>> time as you broke something ;-) ) and I'm assuming >>> that is still the case. I don't think it emailed just >>> when it was running a job based on your >>> commits...just when that job failed and your commit >>> was one of those possible causes. >>> >>> That is my understanding at least, some of our >>> council github gurus may correct me. >>> >>> -James >>> >>> >>> >>> On 12/08/15 10:26, Stuart A. Yeates wrote: >>> >>> I've got a change I'd like to make to the >>> stylesheets used to generate the HTML version of the >>> standard (a minor change to regularise the display of >>> attribute values), but I'm not quite sure how to go >>> about it. >>> >>> Do I just push a change request to >>> https://github.com/TEIC/Stylesheets ? Or fork it? >>> Does that get tested by jenkins? >>> >>> Jenkins used to send me emails when it ran a job >>> based on my commits, but I've not seen one for a >>> couple of recent commits, is that because I'm not on >>> the council any more? >>> >>> cheers stuart -- ...let us be heard from red core to >>> black sky >>> >>> >>> >>> -- Dr James Cummings, James.Cummings@it.ox.ac.uk >>> mailto:James.Cummings@it.ox.ac.uk Academic IT >>> Services, University of Oxford >>> >>> >>> >>> >>> >>> -- >>> >>> tei-council mailing list >> tei-council@lists.tei-c.org >> http://lists.lists.tei-c.org/mailman/listinfo/tei-council >> >> >> PLEASE NOTE: postings to this list are publicly archived >> >> -- >> > tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org mailto:tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

I wasn't, but it's entirely possible "make test" in P5/ was actually failing without my realizing it, because of the way the output reads. In the Stylesheets, because it's in a "Test" directory, I assumed just running "make" would do what I wanted... On Wed, Aug 19, 2015 at 2:15 PM, Lou Burnard wrote:

Let's not confuse running "make test" in the Stylesheets, with running "make test" in the P5 Guidelines directory.

________________________________________ From: tei-council-bounces@lists.tei-c.org [ tei-council-bounces@lists.tei-c.org] on behalf of Hugh Cayless [ philomousos@gmail.com] Sent: 19 August 2015 14:01 To: TEI Council Subject: Re: [tei-council] Fwd: Re: editing the TEI stylesheets

Thanks for the info Martin. I guess I'm used to systems like Rake, where I run the test suite and get a nice little report in my console telling me what tests succeeded, what failed, and what raised an error.

The problem I have with Jenkins is this: I (thinking I'm done) commit something, it tells me a few minutes later that I broke the build for *everyone* and anything they commit subsequently will also fail until I fix the problem. I panic and rush to fix the problem. Personally, I'd vastly prefer to be able to run the test suite locally so that I know that I won't mess up everyone else's workflow when I commit stuff.

Maybe I'm just strange though...

On Wed, Aug 19, 2015 at 8:46 AM, Martin Holmes wrote:

...

Hi Hugh,

On 15-08-19 05:33 AM, Hugh Cayless wrote:

...

Yeah, we do need a protocol for this. I can't claim to understand the Stylesheets either, but I also don't understand our test infrastructure. For example, running "make test" in P5/ clearly runs some tests, but not all of them. I can do that and then check stuff in and have Jenkins yell at me a few minutes later. So how does one run a comprehensive suite of tests? Does it require a Jenkins install?

The Stylesheets project on Jenkins simply runs:

make clean test deb dist

The configuration for it is in our repo here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

...

So you did everything Jenkins did. However, after the build, Jenkins does something called Build Log Parsing, which uses a log file parsing configuration here:

<

...

...

to decide whether to treat specific things as errors that break the build or warnings that make it unstable, or to ignore other things. As a result, you see for example this:

http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/

which (if there were errors in this particular build) would highlight

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... them.

...

To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I

...

have to scroll back through the output to see that at least one test is failing—no surprise that it's related to making PDFs, which doesn't work outside our bespoke Linux environment.

LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests,

...

but it's terrible, and terrifying(!) that they're not generally usable.

I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I

...

would really like to get us into a place where normal humans (or even

most

...

...

Council members) could edit the Guidelines or Stylesheets and test their changes before pushing them. Jenkins is great, but it's a backstop (or feels to me like it should be, anyway).

What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it.

We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible.

Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time.

...

Does anyone else have difficulty with this stuff?

I expect Stuart's pull request went past me last week while I was away at a

...

conference. I've commented and asked him for some clarification.

On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes wrote:

I looked at this, but I'm by no means confident that I could assess the

...

impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

...

Forwarding this reminder to tei-council to remind them that they need

to

...

keep a watching eye on such pull requests. (Once set up, we should get notifications sent to tei-council if possible)

-James

-------- Forwarded Message -------- Subject: Re: editing the TEI stylesheets Date: Wed, 19 Aug 2015 09:43:31 +1200 From: Stuart A. Yeates To: James.Cummings@it.ox.ac.uk CC: Sebastian Rahtz

OK, so I sent a pull request almost a week ago https://github.com/TEIC/Stylesheets/pull/114

Do I need to notify someone? Or has feedback gone somewhere I'm missing...

cheers stuart

-- ...let us be heard from red core to black sky

On Wed, Aug 12, 2015 at 10:01 PM, James Cummings mailto:James.Cummings@it.ox.ac.uk> wrote:

Hi Stuart,

The TEI Technical Council is currently reviewing how it uses GitHub as part of examining whether it is a suitable replacement for SourceForge in moving Guidelines development there. As you know, it moved the Stylesheets (and other software) building there some time ago. Although you (and various others) are members of the Stylesheets repository I believe the correct github etiquette is to fork, then issue a pull request. However, I think as long as you are flagging that you are making the change (or open an issue and then make the change as a result of that) it should probably be fine. (Though when you say 'regularise the display of attribute values' I'd be interested to know what you mean by regularise. There are some options and logic already in there.) The change in your forked version won't get looked at by Jenkins. We're also looking at whether some of the GitHub hosted alternative continuous integration systems might replace Jenkins. So, to be honest, you could do either, but fork+change+pull-request will ensure that it gets other eyes on it before going into the system. Jenkins uses the most up-to-date stylesheets when next building the Guidelines. I know it used to email when doing commits to the Guidelines SF SVN repository that broke something (or more accurately when someone committing around the same time as you broke something ;-) ) and I'm assuming that is still the case. I don't think it emailed just when it was running a job based on your commits...just when that job failed and your commit was one of those possible causes.

That is my understanding at least, some of our council github gurus may correct me.

-James

On 12/08/15 10:26, Stuart A. Yeates wrote:

I've got a change I'd like to make to the stylesheets used to generate the HTML version of the standard (a minor change to regularise the display of attribute values), but I'm not quite sure how to go about it.

Do I just push a change request to https://github.com/TEIC/Stylesheets ? Or fork it? Does that get tested by jenkins?

Jenkins used to send me emails when it ran a job based on my commits, but I've not seen one for a couple of recent commits, is that because I'm not on the council any more?

cheers stuart -- ...let us be heard from red core to black sky

-- Dr James Cummings, James.Cummings@it.ox.ac.uk mailto:James.Cummings@it.ox.ac.uk Academic IT Services, University of Oxford

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

-- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived -- tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived