!wtKWYahxzQAvirVEJG:matrix.org

pycsw

26 Members
1 Servers

Load older messages


SenderMessageTime
24 Sep 2020
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter) https://nbs.csw.met.no/?service=CSW&version=2.0.2&request=GetRecordById&outputFormat=application%2Fxml&outputSchema=http%3A%2F%2Fwww.opengis.net%2Fcat%2Fcsw%2F2.0.2&elementsetname=full&id=S1A_EW_GRDH_1SDH_20181025T070541_20181025T070751_024289_02A88D_D41F 10:01:10
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter)works now10:01:48
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter) with https://nbs.csw.met.no/?service=CSW&version=2.0.2&request=GetRecordById&outputFormat=application%2Fxml&outputSchema=http://www.isotc211.org/2005/gmd&elementsetname=full&id=S1A_EW_GRDH_1SDH_20181025T070541_20181025T070751_024289_02A88D_D41F I get a 500 10:02:31
@gitter_epifanio:matrix.orgepifanio (Gitter)yes, I ghot the same - I was wondering about why that 500 - just to mnake sure I am not doing something wrong from my side10:03:56
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter)can you check the logs?10:09:36
@gitter_epifanio:matrix.orgepifanio (Gitter) yes, I got this:
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    | [2020-09-24 10:07:19 +0000] [14] [ERROR] Error handling request /?service=CSW&version=2.0.2&request=GetRecordById&outputFormat=application%2Fxml&outputSchema=http://www.isotc211.org/2005/gmd&elementsetname=full&id=S1A_EW_GRDH_1SDH_20181025T070541_20181025T070751_024289_02A88D_D41F
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    | Traceback (most recent call last):
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     self.handle_request(listener, req, client, addr)
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     respiter = self.wsgi(environ, resp.start_response)
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/pycsw/wsgi.py", line 77, in application
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     status, contents = csw.dispatch_wsgi()
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/pycsw/server.py", line 250, in dispatch_wsgi
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     return self.dispatch()
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/pycsw/server.py", line 561, in dispatch
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     self.response = self.iface.getrecordbyid()
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/pycsw/ogc/csw/csw2.py", line 1085, in getrecordbyid
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     self.parent.profiles['loaded'][self.parent.kvp['outputschema']].write_record(
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "/usr/lib/python3.8/site-packages/pycsw/plugins/profiles/apiso/apiso.py", line 402, in write_record
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |     return etree.fromstring(xml_blob, self.context.parser)
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 1777, in lxml.etree._parseDoc
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 1082, in lxml.etree._BaseParser._parseUnicodeDoc
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    |   File "<string>", line 1
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    | lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1
nbs_pycsw.1.lhp966cbbyyw@pycsw-prod    | 10.0.1.49 - - [24/Sep/2020:10:07:19 +0000] "GET /?service=CSW&version=2.0.2&request=GetRecordById&outputFormat=application%2Fxml&outputSchema=http://www.isotc211.org/2005/gmd&elementsetname=full&id=S1A_EW_GRDH_1SDH_20181025T070541_20181025T070751_024289_02A88D_D41F HTTP/1.1" 500 0 "-" "-"
10:13:17
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter) can you add a debug statement at https://github.com/geopython/pycsw/blob/master/pycsw/plugins/profiles/apiso/apiso.py#L401 to print out xml_blob ? 10:28:01
@freenode_sigabrt:matrix.orgsigabrtTitle: pycsw/apiso.py at master · geopython/pycsw · GitHub (at github.com)10:28:04
@gitter_epifanio:matrix.orgepifanio (Gitter)it is in a docker container, the csw code is not exposed as volume so I have to put down the service to do that - I will try :) 10:50:19
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter)are you able to test in a dev env?10:51:21
@gitter_epifanio:matrix.orgepifanio (Gitter)I will make a new service based on git master10:55:30
@gitter_epifanio:matrix.orgepifanio (Gitter)and expose the csw src as a volume10:55:41
@gitter_epifanio:matrix.orgepifanio (Gitter)I can do that pointing to the same pg db in the csw config I guess10:56:24
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter)

I managed to debug and install my bare-bones version of the datacite plugin. So far it produces results like this:

curl -G "http://localhost:8000/pycsw/csw.py?service=CSW&version=2.0.2&request=GetRecordById&ElementSetName=full&Id=urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f&outputSchema=http://datacite.org/schema/kernel-4"

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.5.dev0 -->

<csw:GetRecordByIdResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">

<datacite xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.3/metadata.xsd">

<identifier identifierType="DOI">urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f</identifier>

<creators><creator><creatorName nameType="organizational">creator</creatorName></creator></creators><titles><title>Lorem ipsum</title><subtitle titleType="Subtitle">A subtitle</subtitle></titles><publisher>EISCAT Scientific Association</publisher><publicationyear>2020</publicationyear><resourcetype><resourcetypegeneral>dataset</resourcetypegeneral></resourcetype></datacite></csw:GetRecordByIdResponse>

10:59:09
@freenode_sigabrt:matrix.orgsigabrtTitle: Lorem ipsum (at matrix.org)10:59:12
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter) Obviously, I have to extend on this once we have a proper database of metadata in our format. Just used the provided example for testing. Furthermore the above result is malformed. So next I will make sure that it conforms to the examples at https://schema.datacite.org/meta/kernel-4.3/ 11:06:11
@freenode_sigabrt:matrix.orgsigabrtTitle: DataCite Schema (at schema.datacite.org)11:06:13
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter) @cfenell great work ! To test:
  • add datacite to pycsw/plugins/outputschemas/__init__.py
  • start/restart server
  • run a GetCapabilities request. Does http://datacite.org/schema/kernel-4 appear in //ows:Parameter/@name=“outputSchema” ?
11:30:54
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter)

Yes, it looks like this:

<ows:Parameter name="outputSchema"><ows:Value>http://datacite.org/schema/kernel-4</ows:Value><ows:Value>http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/</ows:Value><ows:Value>http://www.interlis.ch/INTERLIS2.3</ows:Value><ows:Value>http://www.isotc211.org/2005/gmd</ows:Value><ows:Value>http://www.opengis.net/cat/csw/2.0.2</ows:Value><ows:Value>http://www.opengis.net/cat/csw/csdgm</ows:Value><ows:Value>http://www.w3.org/2005/Atom</ows:Value></ows:Parameter>

11:34:34
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter) However in order to make my changes appear I also have to
pip3 install --user .
in the top level pycsw directory before restarting the server. Maybe I can just remove the files from ~/.local/libs/python-x.y/... and run from the source directory, but this seems to work too.
11:41:20
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter)

I also modified oaipmh.py to allow this:
curl -G --data-urlencode "mode=oaipmh" --data-urlencode "verb=GetRecord" --data-urlencode "identifier=urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f" --data-urlencode "metadataPrefix=datacite" http://localhost:8000/pycsw/csw.py

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.5.dev0 -->

<oai:OAI-PMH xmlns:oai="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><oai:responseDate>2020-09-24T13:55:24Z</oai:responseDate><oai:request identifier="urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f" metadataprefix="datacite" verb="GetRecord">http://localhost/pycsw/csw.py?mode=oaipmh</oai:request><oai:GetRecord><oai:record><oai:header><oai:identifier>urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f</oai:identifier><oai:dateStamp/><oai:setSpec/></oai:header><oai:metadata>

<resource xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.3/metadata.xsd"><identifier identifierType="DOI">urn:uuid:19887a8a-f6b0-4a63-ae56-7fba0e17801f</identifier><creators><creator><creatorName nameType="organizational">creator</creatorName></creator></creators><titles><title>Lorem ipsum</title></titles><publisher/><publicationYear/><resourceType resourceTypeGeneral="Image">image/svg+xml</resourceType>
</resource>
</oai:metadata></oai:record></oai:GetRecord></oai:OAI-PMH>

11:57:10
@freenode_sigabrt:matrix.orgsigabrtTitle: Lorem ipsum (at matrix.org)11:57:12
@gitter_tomkralidis:matrix.orgTom Kralidis (Gitter) great. So as a next step you can run a GetRecords request with outputSchema=http://datacite.org/schema/kernel-4 12:07:06
@freenode_sigabrt:matrix.orgsigabrtTitle: DataCite - 404 (at datacite.org)12:07:09
@gitter_cfenell:matrix.orgCarl-Fredrik Enell (Gitter)

Ok, I can now also do this:

curl -G --data-urlencode "mode=oaipmh" --data-urlencode "verb=GetRecord" --data-urlencode "identifier=urn:uuid:9a669547-b69b-469f-a11f-2d875366bbdc" --data-urlencode "metadataPrefix=datacite" http://localhost:8000/pycsw/csw.py

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.5.dev0 -->

<oai:OAI-PMH xmlns:oai="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
<oai:responseDate>2020-09-24T14:40:12Z</oai:responseDate>
<oai:request identifier="urn:uuid:9a669547-b69b-469f-a11f-2d875366bbdc" metadataprefix="datacite" verb="GetRecord">http://localhost:8000/pycsw/csw.py?mode=oaipmh</oai:request>
<oai:GetRecord>
<oai:record>
<oai:header>
<oai:identifier>urn:uuid:9a669547-b69b-469f-a11f-2d875366bbdc</oai:identifier>
<oai:dateStamp/>
<oai:setSpec/>
</oai:header>
<oai:metadata>
<resource xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.3/metadata.xsd">
<identifier identifierType="DOI">urn:uuid:9a669547-b69b-469f-a11f-2d875366bbdc</identifier>
<creators>
<creator>
<creatorName nameType="organizational">creator</creatorName>
</creator>
</creators>
<titles>
<title>Ñunç elementum</title>
</titles>
<publisher/>
<publicationYear/>
<resourceType resourceTypeGeneral="Dataset"/>
</resource>
</oai:metadata>
</oai:record>
</oai:GetRecord>
</oai:OAI-PMH>

Some metadata here is wrong or missing. Anyhow this is a good starting point for the format we consider in order to provide metadata to B2SHARE.

12:43:14
@freenode_sigabrt:matrix.orgsigabrtTitle: Ñunç elementum (at matrix.org)12:43:16
25 Sep 2020
@freenode_sigabrt:matrix.orgsigabrt left the room.01:12:07
@freenode_sigabrt:matrix.orgsigabrt joined the room.01:12:39
@freenode_jmckenna:matrix.orgjmckenna left the room.01:18:22
@freenode_jmckenna:matrix.orgjmckenna joined the room.12:03:52

There are no newer messages yet.


Back to Room List