META-NORD – Uploading instructions (for first batch)
Deliverable 4.3
Introduction
This document is an instruction manual to assist META-NORD partners with the task of fulfilling their contractual obligations to the European Committee of providing open access to Language Technology resources. The ultimate goal is to make these resources and their metadata accessible through the META-SHARE network. One node of this network is located at UGOT and is available for direct uploading by META-NORD partners (see section 1 for more details). However, UGOT also provides a temporary solution to ward against incomplete functionality of the META-SHARE node (further details are in section 2).
1. The META-SHARE node at UGOT
Uploading directly to the META-SHARE node is done using the META-SHARE editor at this URL: http://demo.spraakdata.gu.se/ws/metashare/
The node currently contains the information submitted by META-NORD partners in the Deliverable 2.2 spreadsheet. This is insufficient information according to META-SHARE's minimal schema, but is a starting point. Partners can edit their entries directly in the editor to fill out the remaining mandatory fields so that each metadata file will validate against the aforementioned schemas.
In order to access the node and do more than just browse, you will need to register as a user on this node by clicking on the button “Register” in the upper right corner. You will receive an email from “” to confirm your registration, so you must be sure your email account does not filter it as spam. After clicking on the confirmation link you will receive another email with your log-in information.
NB:
· We kindly ask that partners only edit their own entries since the current version of the editor allows each user to have editing rights for all entries, and this will not be fixed until the next version release of the META-SHARE editor according to their Provider Manual.
· Furthermore, there is currently no way to attach the actual resource through the editor interface although it can be attached through the backend. To have this done, you must email the actual resource to stating who is the provider and the 64 character StorageObject Identifier seen at the top of the webpage when browsing in the editor (e.g. http://demo.spraakdata.gu.se/ws/metashare/browse/3/ )
· At this stage of the editor's life, we strongly suggest that caution be used when considering uploading resources that have restricted access, as we have not seen any evidence that there exists any restriction functionality in the editor.
For detailed instructions on how to use the editor, please refer to the META-SHARE editor manual, which can be found here: http://demo.spraakdata.gu.se/ws/metashare/site_media/documentation.pdf
For your convenience, here is a checklist of the tabs that contain mandatory fields:
n Identification
n Metadata
n Distribution
n Person
n Content
. NB, if MediaType is “text” then the tab “Text” becomes mandatory. This tab itself contains multiple tabs, of which the following have mandatory fields:
◦ Linguality
◦ Language
◦ Size
◦ TextFormat
◦ CharacterEncoding
◦ Domain
◦ Annotation
◦ TimeCoverage
◦ GeographicCoverage
2. The UGOT (temporary) repository
UGOT has dedicated server space for META-NORD partners' convenience. Uploading is done with a subversioning tool (svn) to the UGOT repository (see section 2.0 for instructions on how to get svn). From there the data will be imported to the META-SHARE node. The preferred file format of the metadata files is .xml, but spreadsheets are also acceptable (.ods .xls .xlsx).
This method allows you to edit data on your computer and update (or add new) files with a few simple commands (please do not be intimidated by the detailed instructions, the main ones are in 2.1. which will be the only steps necessary if everything is perfect). This also keeps a history of previous versions so you could revert back to older versions if need be.
An overview of the uploading process to UGOT's repository (details follow in next sections):
2.1. upload to the repository with the svn commit command
2.2. svn will respond with comments
2.3. if the upload is a spreadsheet then the repository converts it to xml
2.4. the repository checks the xml against the schemas whether it is valid
2.5. the repository sends an email to specified contact address(es)
2.6. the uploader must check the email message to see whether their upload was valid or invalid. If invalid, the message will provide information regarding the error(s).
Note that if the svn response was positive, then the upload to the repository was successful in the sense that a copy of the file exists in the repository. However, if the xml validation email reported errors, then the file can not be imported to the META-SHARE node.
In the following detailed sections there are examples of what to do if errors are encountered during this process.
2.0 Installing subversion tool (svn)
2.0.1. If you already have svn and know how to use it, skip to step 2.6.
2.0.2. First you will need svn software on your computer:
http://subversion.apache.org/packages.html
(For Windows users we suggest going here: http://tortoisesvn.net/downloads.html )
2.0.3. The following instructions are aimed at Linux users, but Windows users can take a look at this Quick Start guide for using TortoiseSVN: http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-qs-guide.html
(for full manual: http://tortoisesvn.net/docs/release/TortoiseSVN_en/index.html )
2.0.4. Using a terminal, go to (or create) the folder where the data that needs to be uploaded is stored and add the svn to that folder by typing the command:
svn checkout https://svn.spraakdata.gu.se/repos/metanord/
2.1 How to upload: first add file, then commit changes.
2.1.1. Adding a file to svn:
Using a terminal, open the folder on your local computer where the file is located, and type the command:
svn add filename.xml
2.1.2. Committing files to svn:
Note that it is possible to commit many additions (or changes) to the repository with one commit command, by typing:
svn commit -m “This is a comment about what is being committed.”
(NB. The comment must be between double quotation marks.)
2.2 Response from svn to the commit. If commit is valid, skip to step 2.6.
2.2.1. Valid examples:
If the commit to svn was successful, then the response will simply say “Done” and you can skip to step 2.6.
2.2.2. Invalid examples:
There are a number of errors that could come up, but are unlikely in this process. Check the svn help by typing the command:
svn help status
2.2.3. After errors have been resolved, it is necessary to re-commit (back to step 2.1.2)
2.3 These next few steps occur "behind the scenes". The repository checks whether the upload is an xml file or spreadsheet, and will convert the spreadsheet to xml. (To Be Completed)
2.4 The repository checks if the xml file is valid against the META-SHARE metadata schemas. (To Be Completed)
2.5 The repository sends an email (To Be Completed) with the result of the validation to specified email address(es). Partners will find a file in their repository folder named “email.txt” where they should simply enter the email address they wish to receive the xml validation email. For multiple recipients, enter one address per line.
2.6 The uploader must check the email response from the repository that was sent to the email address(es) in the email file. If the response indicated that there were no errors, then you are done.
2.6.1. Errors with xml validation:
How to resolve invalid xml validation, with examples. (To Be Completed)
2.6.2. After errors have been resolved, it is necessary to re-commit (back to step 2.1.2)