Re: [tei-council] Fwd: Re: editing the TEI stylesheets

HI Hugh, On 15-08-19 08:33 AM, Hugh Cayless wrote:

It produces output like:

# Running tests:

..........................................................................................

Finished tests in 136.596000s, 0.6589 tests/s, 3.7922 assertions/s.

90 tests, 518 assertions, 0 failures, 0 errors, 0 skips

as a test runs, a "." is output if it succeeds, an "F" if it fails, or an "E" if it throws an exception.

I think I'd like better output than that -- I'd like to see which test failed, and if (for example) it's a validity test, where the error is in the file, and what the error message is.

Not saying we should re-write all our tests in Ruby,

No, don't say that.:-)

but it would be very nice to be able to run the test suite locally and get sensible outputs. Would be good too to be able to skip tests that are known to fail, like PDF generation. Incidentally, I *almost* have that working on my laptop...not sure I can remember all the steps to make it work though.

It's a bit outdated now, but the Jenkins build script should cover everything needed on Linux: http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/... In particular, I think you need all the fonts, one of which (Hannom) has to be downloaded and unzipped from a SourceForge project; you need all the *tex* stuff; and there's probably a couple of other dependencies in there that aren't completely obvious. I think we ought to have a conversation about LaTeX in the build, though. I don't think we have any LaTeX experts on Council at the moment, do we? Cheers, Martin

On Wed, Aug 19, 2015 at 11:20 AM, Martin Holmes wrote:

...

Hi Hugh,

On 15-08-19 06:01 AM, Hugh Cayless wrote:

...

Thanks for the info Martin. I guess I'm used to systems like Rake, where I run the test suite and get a nice little report in my console telling me what tests succeeded, what failed, and what raised an error.

That makes perfect sense. That's what we use Jenkins for, but perhaps if we have a Council that predominantly prefers to get a working build environment locally, and only commit when they're comfortable, then implementing our own build log parsing would make sense. In principle, anything that generates "error" or "warning" needs attention; but in practice, as you can see from the scale of the log parse rules, there are all sorts of things that need to be suppressed and ignored. For example, the line:

No errors or warnings detected

contains both "errors" and "warnings", so the build would fail if we didn't trap for it:

ok /No errors or warnings detected/

Similarly,

Exiting with failure status

does not contain "error" or "warning", so it would go by unnoticed if we didn't trap for that:

error /Exiting with failure status/

The issue of LaTeX is a very messy business; you have to install a great deal of stuff to get that to work, and it's probably more than any of us wants running on our laptops unless we're big LaTeX users.

The problem I have with Jenkins is this: I (thinking I'm done) commit

...

something, it tells me a few minutes later that I broke the build for *everyone* and anything they commit subsequently will also fail until I fix the problem. I panic and rush to fix the problem. Personally, I'd vastly prefer to be able to run the test suite locally so that I know that I won't mess up everyone else's workflow when I commit stuff.

Maybe I'm just strange though...

No, I see that. How exactly does Rake produce its report? I'm not familiar with it.

Cheers, Martin

On Wed, Aug 19, 2015 at 8:46 AM, Martin Holmes wrote:

...

Hi Hugh,

...

On 15-08-19 05:33 AM, Hugh Cayless wrote:

Yeah, we do need a protocol for this. I can't claim to understand the

...

Stylesheets either, but I also don't understand our test infrastructure. For example, running "make test" in P5/ clearly runs some tests, but not all of them. I can do that and then check stuff in and have Jenkins yell at me a few minutes later. So how does one run a comprehensive suite of tests? Does it require a Jenkins install?

The Stylesheets project on Jenkins simply runs:

make clean test deb dist

The configuration for it is in our repo here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

So you did everything Jenkins did. However, after the build, Jenkins does something called Build Log Parsing, which uses a log file parsing configuration here:

<

http://sourceforge.net/p/tei/code/HEAD/tree/trunk/Documents/Editing/Jenkins/...

...

to decide whether to treat specific things as errors that break the build or warnings that make it unstable, or to ignore other things. As a result, you see for example this:

http://teijenkins.hcmc.uvic.ca/job/Stylesheets/1963/parsed_console/

which (if there were errors in this particular build) would highlight them.

To reproduce this locally, you could redirect the build output to a file, and then grep it "error" or "warning" (case-insensitive).

Running the Stylesheet tests doesn't even give me a report at the end, I

...

have to scroll back through the output to see that at least one test is failing—no surprise that it's related to making PDFs, which doesn't work outside our bespoke Linux environment.

LaTeX (however you're supposed to capitalize it) is beyond me, so that aspect of the build seems frighteningly fragile. I'd prefer that we use XSL:FO and FOP for PDFs, but that of course is just my preference, and others would find LaTeX easier.

It's wonderful that there are tests,

...

but it's terrible, and terrifying(!) that they're not generally usable.

I don't think that's the case. There aren't enough tests, probably, but they're not complicated really.

I

...

would really like to get us into a place where normal humans (or even most Council members) could edit the Guidelines or Stylesheets and test their changes before pushing them. Jenkins is great, but it's a backstop (or feels to me like it should be, anyway).

What makes you so uneasy about letting Jenkins do it? I'm happy not to have to trawl through the build log, personally; I'd rather have an automated system doing it.

We could create a Jenkins instance pre-loaded with TEI stuff so you could run that locally before committing -- would that help? Alternatively, we could re-work all the Jenkins stuff on Travis if we move to GitHub, and perhaps make it more comprehensible.

Cheers, Martin

Or is this just me being an idiot? It certainly wouldn't be the first time.

...

Does anyone else have difficulty with this stuff?

I expect Stuart's pull request went past me last week while I was away at a

...

conference. I've commented and asked him for some clarification.

On Tue, Aug 18, 2015 at 6:58 PM, Martin Holmes wrote:

I looked at this, but I'm by no means confident that I could assess the

...

impact of a merge like this without investing a fair amount of time figuring out what it does and how it works. The Stylesheets are still fairly mysterious to me, except for the bits I happen to have looked at.

I think what we actually need is a protocol for this. So, for instance, before accepting a pull request that is anything more than a typo fix, at least check out the branch and run all the regular tests; then perhaps create one more test that checks the bug that's being fixed; then go ahead and merge.

Cheers, Martin

On 15-08-18 02:52 PM, James Cummings wrote:

> Forwarding this reminder to tei-council to remind them that they need > to > keep a watching eye on such pull requests. (Once set up, we should > get > notifications sent to tei-council if possible) > > -James > > > -------- Forwarded Message -------- > Subject: Re: editing the TEI stylesheets > Date: Wed, 19 Aug 2015 09:43:31 +1200 > From: Stuart A. Yeates > To: James.Cummings@it.ox.ac.uk > CC: Sebastian Rahtz > > > > OK, so I sent a pull request almost a week ago > https://github.com/TEIC/Stylesheets/pull/114 > > Do I need to notify someone? Or has feedback gone somewhere I'm > missing... > > cheers > stuart > > -- > ...let us be heard from red core to black sky > > On Wed, Aug 12, 2015 at 10:01 PM, James Cummings > mailto:James.Cummings@it.ox.ac.uk> > wrote: > > > Hi Stuart, > > The TEI Technical Council is currently reviewing how it uses > GitHub as part of examining whether it is a suitable > replacement for SourceForge in moving Guidelines development > there. As you know, it moved the Stylesheets (and other > software) building there some time ago. Although you (and > various others) are members of the Stylesheets repository I > believe the correct github etiquette is to fork, then issue a > pull request. However, I think as long as you are flagging > that you are making the change (or open an issue and then > make the change as a result of that) it should probably be > fine. (Though when you say 'regularise the display of > attribute values' I'd be interested to know what you mean by > regularise. There are some options and logic already in > there.) The change in your forked version won't get looked at > by Jenkins. We're also looking at whether some of the GitHub > hosted alternative continuous integration systems might > replace Jenkins. So, to be honest, you could do either, but > fork+change+pull-request will ensure that it gets other eyes > on it before going into the system. Jenkins uses the most > up-to-date stylesheets when next building the Guidelines. I > know it used to email when doing commits to the Guidelines SF > SVN repository that broke something (or more accurately when > someone committing around the same time as you broke > something ;-) ) and I'm assuming that is still the case. I > don't think it emailed just when it was running a job based > on your commits...just when that job failed and your commit > was one of those possible causes. > > That is my understanding at least, some of our council github > gurus may correct me. > > -James > > > > On 12/08/15 10:26, Stuart A. Yeates wrote: > > I've got a change I'd like to make to the stylesheets > used to generate the HTML version of the standard (a > minor change to regularise the display of attribute > values), but I'm not quite sure how to go about it. > > Do I just push a change request to > https://github.com/TEIC/Stylesheets ? Or fork it? Does > that get tested by jenkins? > > Jenkins used to send me emails when it ran a job based on > my commits, but I've not seen one for a couple of recent > commits, is that because I'm not on the council any more? > > cheers > stuart > -- > ...let us be heard from red core to black sky > > > > -- Dr James Cummings, James.Cummings@it.ox.ac.uk > mailto:James.Cummings@it.ox.ac.uk > Academic IT Services, University of Oxford > > > > > > -- > > tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived

--

tei-council mailing list tei-council@lists.tei-c.org http://lists.lists.tei-c.org/mailman/listinfo/tei-council

PLEASE NOTE: postings to this list are publicly archived