June 2018 - Indic-texts - lists.tei-c.org

Re: [Indic-texts] Pandit Project - Behavior change - Extract entity type
by Dominik Wujastyk 29 Jun '18

29 Jun '18

Thank you very much for this update, and for your continued work on this wonderful tool. Dominik -- Professor Dominik Wujastyk <http://ualberta.academia.edu/DominikWujastyk> , Singhmar Chair in Classical Indian Society and Polity , Department of History and Classics <http://historyandclassics.ualberta.ca/> , University of Alberta, Canada . South Asia at the U of A: sas.ualberta.ca On Thu, 28 Jun 2018 at 11:58, Amir Simantov <wawina(a)gmail.com> wrote: > Dear contributors, > > I write to you all as you are the active manuscript extracts contributors. > > The first months of your hard work helped us understand how we can improve > the behavior of the data entering. No need spotted for structure changes, > though. > > The behavior changes are: > > 1. You can add more than one extract of each kind for a manuscript. > The prompting link to add a new extract from within a manuscript is one: > "Add an extract". > 2. You can change the extract kind after creation. No need to > unpublish an extract and add another one instead. Thanks Jonathan Peterson > who helped us understand that this change is needed. > 3. You can add an extract for a manuscript that is *not* related to a > specific work. Use it when the extract text is manuscript-specific and not > to a work that is written in the manuscript. Thanks to Dominik Wujastyk for > his advice on this issue. > 4. You have to choose the work from the select list in the editing > form of the extract entity. If the extract is not specific for one work > then tick the checkbox "Not work-specific". > > Note there is no need to edit and change existing extracts; this email is > just to share with you what we did. > > Please report on any behavior you suspect to be a bug! > > We may also improve the *presentation* of the extracts in the manuscript > page if we get creative input on it from you :) > > Thanks, > Amir > (Pandit programmer) >

1 0

Google OCR for Sanskrit
by Andrew Ollett 17 Jun '18

17 Jun '18

Dear colleagues, After noticing results in Google Books from Dēvanāgarī editions of Sanskrit texts, I suspected that Google had some OCR technology for Dēvanāgarī. It turns out that they have had it for some time, but only relatively recently has an API been available to use it. Here is a comparison of Google OCR with Oliver Hellwig's SanskritOCR: https://sanskrit-coders.github.io/site/posts/ocr-comparison.html I think the conclusion is that SanskritOCR is slightly better in terms of quality, but Google OCR is faster and easier to use programmatically. I have tried out Google OCR and managed to get relatively good results: http://www.prakrit.info/blog/google-ocr-for-sanskrit/ I think this could be quite promising. If we manage to write a couple of tools for processing PDFs into txt files (and thence onto TEI if desired), then we could go from scanned document to machine-readable text in minutes. Of course things would need to be checked, and Google OCR seems to have particularly nasty problems with notes and smaller size text. My setup is a few local Python scripts that interact with the resources in a Google Cloud Storage "bucket" (where they need to be in order for Google OCR to work). Maybe others could come up with a more streamlined setup. Or maybe we could even create an app that allows users to upload PDF documents and get text data in return. Thoughts? Andrew

2 1

Fwd: on pādas (again)
by Andrew Ollett 16 Jun '18

16 Jun '18

I share Dan and Arlo's intuition regarding the equivalence of <l> and a pāda. The problem is just that we can't apply those intuitions consistently without a lot of extra work. This particularly inconsistency is probably not terrible (in processing we can always make a distinction between <lg> with two children and <lg> with four children) but if we wanted to solve it, I think a milestone element would do the trick. There are two problems regarding metrical and word boundary coincidence. The first is more general: since the pāda boundary often falls in the seam of a compound, you can't use <w> for compounds in such cases, because your metrical element will end before its child element, a word, is finished. The second is specific to Dravidian languages (which are "Indic" for our purposes): metrical structure and word structure are much more independent than in Sanskrit. We have ci // tram, vi // cāra-, en // dum, ā // ṟaneya, a // tta, usually between odd and even pādas, but sometimes also between even and odd pādas. I of course believe that word analysis is something that we definitely want. But I don't think it's something we can do programmatically at the moment, and even if we do it manually, I completely agree with Dan that standoff markup (i.e., have the unanalyzed text in one place, and the analyzed text linked to it in another place) is the only way to avoid really nasty problems of OHCO violations, vowel sandhi, and so on. In principle one could always find a solution (the BHELA conventions sometimes used on GRETIL are a nice way of specifying how vowel sandhi is to be "undone") but in general I don't think we should worry too much about word analysis at the stage of applying structural markup to texts. Here is my "alternate encoding" (using TEI) for the above verse, just to show that I have not entirely thrown word analysis to the wind: <s> <w lemma="ādisvarapada"> <w lemma="ādi" ana="#ibc">ādi</w> <w lemma="svara" ana="#ibc">svara</w> <w lemma="pada">padaṁ</w> </w> <w lemma="anta" ana="#sing #loc">antadoḷ</w> <w lemma="āgu" ana="#p #adjp">āda</w> <w lemma="eḍe" ana="#sing #loc">eḍeyoḷ</w> <w lemma="āgu" ana="#habit">akkuṁ</w> <w lemma="eraḍaneya" ana="#adn">eraḍaneya</w> <w lemma="vibhaktyādānapada"> <w lemma="vibhakti" ana="#ibc">vibhakti</w> <w lemma="ādāna" ana="#ibc">ādāna</w> <w lemma="pada" ana="#sing #nom">padaṁ</w> </w> <w lemma="dīrgha" ana="#sing #nom">dīrghaṁ</w> <w lemma="pādānta"> <w lemma="pāda" ana="#ibc">pāda</w> <w lemma="anta" ana="#sing #loc">antadoḷ</w> </w> <w lemma="uḻi" ana="#p #adjp">uḻida</w> <w lemma="tāṇa" ana="#sing #loc">tāṇadoḷ</w> <w lemma="svacchanda" ana="#sing #nom">svacchandaṁ</w> </s>

2 1

on pādas (again)
by Andrew Ollett 16 Jun '18

16 Jun '18

Dear colleagues, I am in the midst of a workshop in which we are attempting to encode texts in Old Javanese in TEI format, and the issue of encoding pādas has (once again) reared its head. We discussed this issue at length in the context of the SARIT project, and we came to the conclusion that <l> should be used for a pair of pādas, and the boundary between even and odd pādas should be represented by the <caesura/> element. Hence the following vasantatilaka verse: <lg n="3" met="vasantatilaka"> <l>vvantən mañumbana puḍak ginuritnya pārtha<caesura/> ndān susvasusvani kinolnya hanan liniṅliṅ</l> <l>rakryan vədinta tan akun ləvu paṅhavista<caesura/> heman kitābapa niragraha māsku liṅnya</l> </lg> This is somewhat contrary to what many people would expect, namely, that each pāda should correspond to a single <l> element, as follows: <lg n="3" met="vasantatilaka"> <l>vvantən mañumbana puḍak ginuritnya pārtha</l> <l>ndān susvasusvani kinolnya hanan liniṅliṅ</l> <l>rakryan vədinta tan akun ləvu paṅhavista</l> <l>heman kitābapa niragraha māsku liṅnya</l> </lg> My arguments for the use of <caesura/> involved (a) the practical necessity of encoding texts from printed editions, where the pādas are not separated typographically in all cases, especially in shorter verse forms, and thus (b) the requirement that <l> should mean the same thing for an anuṣṭubh verse as for (e.g.) śakvarī verse, i.e., it should not refer to a pādayuga in the first case, and a pāda in the second case; and (c) the frequent occurrence, in Sanskrit, of words that span the boundaries between odd and even pādas, and the undesirability of having structural elements like <l> overlap with the grammatical structure of the text (at least at the level of the word). The use of <caesura/> would be optional: it's not required (and often isn't marked typographically in shorter verse forms), but if it is present, the stylesheets will insert a space. But I can now think of counterarguments for all of these points, and in some ways, it might be easier if <l> always mean a "pāda." (<caesura/> also doesn't have a @type attribute in standard TEI, so it might be more difficult than I expected to differentiate this "pāda-boundary caesura" from the pāda-internal yati.) So I am asking everyone whether there are compelling reasons you've discovered for preferring one encoding solution over another. (Or if you have other suggestions altogether, including the use of <seg> or other such elements.) I know that there are some features that vary across Indic languages, such as the coincidence of metrical and grammatical (esp. lexical) boundaries: these structures always coincide in Old Javanese, and almost never in Kannada, so I am hoping to avoid the problem of overlapping hierarchies completely. Andrew

5 12

Re: [Indic-texts] on pādas (again)
by Charles Li 08 Jun '18

08 Jun '18

Hello, I'd just like to put in another vote for <caesura/>. In our extremely slow-going on-going edition of the Khaḍgaśataka, we've been using caesuras like so: <lg> <l> kalpātikrūrakālabhrukuṭikuṭilatākalpakalpāgramuṣṭiḥ <caesura/> krudhyatkālīkaṭākṣāṅkita iva kalayan kāntim ekāntakālīm | </l> <l> krīḍaty asyākrameṇa krakacakṛtikṛtī kumbhikumbhāgrakūṭa<caesura/>kroḍeṣūtkṛttakaṇṭhotthitarudhirakaṇākīrṇakoṇaḥ kṛpāṇaḥ || 1 || </l> </lg> As you can see, we put some sort of space before/after the caesura if it coincides with a word-break. It ends up being rendered like this: kalpātikrūrakālabhrukuṭikuṭilatākalpakalpāgramuṣṭiḥ krudhyatkālīkaṭākṣāṅkita iva kalayan kāntim ekāntakālīm | krīḍaty asyākrameṇa krakacakṛtikṛtī kumbhikumbhāgrakūṭa- kroḍeṣūtkṛttakaṇṭhotthitarudhirakaṇākīrṇakoṇaḥ kṛpāṇaḥ || 1 || ...with a hyphen automagically inserted when the caesura is within a word. Here's the XSLT code we use for that: <xsl:template match="caesura"> <xsl:variable name="pretext" select="preceding::text()[1]"/> <xsl:if test="normalize-space(substring($pretext,string-length($pretext))) != ''"> <span class="hyphen">-</span> </xsl:if> <xsl:element name="br"></xsl:element> </xsl:template> It looks for some kind of space character (including tabs, line breaks, etc.) preceding the caesura and hyphenates accordingly. The hyphen is wrapped in a <span> element so that it can be styled or disappeared as desired. You could also use a soft hyphen instead of a hard hyphen. With regards to there being no @type attribute for caesura -- maybe you should just go ahead and use it anyway, and wait for the TEI standard to catch up? I don't see any reason why it shouldn't be there... Have a great day, Charles

1 0