37 Members
Dataverse is open source research data repository software: http://dataverse.org Public logs at https://view.matrix.org/alias/%23dataverse:matrix.org2 Servers

Load older messages

8 Sep 2021
@philippc:matrix.orgPhilippYes, I have done that before. But when I checked today, they suddenly were there again!!15:00:50
@philippc:matrix.orgPhilippYes! I'll send a ticket to support. Thanks for your help!15:01:36
@pdurbin:matrix.orgpdurbinSure! Good luck!15:01:47
@oliver:bertuch.namepoikilothermIsn't there the new download API using HTML pages that might make it easy to download a branch of the tree via wet/curl/httpie?15:12:06
@pdurbin:matrix.orgpdurbin True. That might help. Virgile Jarrige ^^ 15:12:37
@oliver:bertuch.namepoikilothermIt might not be a perfect clicky solution but it's available15:12:48
@oliver:bertuch.namepoikilothermMost likely a UI solution will take a long time as UI/UX research team needs to be involved, running on short supplies I guess15:13:50
@philippc:matrix.orgPhilipp poikilotherm: Could you elaborate a little bit? I couldn't find any information about this in the API guide. 15:38:09
@pdurbin:matrix.orgpdurbinHere you go: https://guides.dataverse.org/en/5.6/api/native-api.html#view-dataset-files-and-folders-as-a-directory-index15:39:33
@oliver:bertuch.namepoikilothermpdurbin you have been 10 sec faster than me 😎15:40:25
@oliver:bertuch.namepoikilothermBut I can add this https://github.com/IQSS/dataverse/issues/794815:40:43
@oliver:bertuch.namepoikilothermCurrently it's a bit cumbersome to craft the API URL...15:41:13
@oliver:bertuch.namepoikilothermAnd I can add that this feature is available since 5.5 with a necessary fix in 5.6 IIRC15:41:44
@philippc:matrix.orgPhilippCould you help me include correctly the folder, version, original=true parameters into the API call: curl ${SERVER_URL}/api/datasets/:persistentId/dirindex?persistentId=doi:${PERSISTENT_ID} Thanks! 15:50:03
@pdurbin:matrix.orgpdurbin Philipp: here's an example: https://dataverse.no/api/datasets/:persistentId/dirindex?persistentId=doi:10.18710/AZT6UY 15:51:16
@pdurbin:matrix.orgpdurbinIf you click, you can get a subfolder, like this: https://dataverse.no/api/datasets/106017/dirindex/?version=1.0&folder=appendix15:52:11
@philippc:matrix.orgPhilippThanks! I think I understand how it works. But I still would need to click on each individual file to download all files in a folder? Or is there a way using API to download (all the files within) a given folder as a zip?15:54:50

The idea is to use a command line tool like wget. Scroll down in the guides and you should see this:

An example of a wget command line for crawling (“recursive downloading”) of the files and folders in a dataset:

@pdurbin:matrix.orgpdurbinA zip is not downloaded. Rather, individual files are downloaded.15:56:30
@pdurbin:matrix.orgpdurbinCreating a zip on the server size can be a bit expensive in terms of time and compute.15:59:46
@pdurbin:matrix.orgpdurbinPlus if you download individual files you can resume your download.15:59:59
@bricas:matrix.orgBrian Cassidy
In reply to @virgile.jarrige:matrix.org
Brian Cassidy: Hi
It depends on how you configure it. By default an authenticated user has the same rights as a unauthenticated user. Reading Only.
But then you can set permissions sub-dataverse per sub-dataverse.
If you use Shibboleth you can set permissions for a whole group.
Turns out it was just an incorrect assumption by our admin. Everything works as expected.
@philippc:matrix.orgPhilipp Thanks, pdurbin I'll have a closer look at it tomorrow :) 16:03:17
@lincoln55:matrix.orgLincolnHI i have a question . I would like to know the place in code file where "ingest of tab file" takes place in dataverse5.517:51:06
@pdurbin:matrix.orgpdurbin Lincoln: By tab I assume you mean tsv. Here's where that was added: https://github.com/IQSS/dataverse/pull/4854 17:53:21
@pdurbin:matrix.orgpdurbinOr do you mean tabular files in general, like Stata, etc.17:53:51
@lincoln55:matrix.orgLincolntabular files in general17:54:00
@pdurbin:matrix.orgpdurbinI guess I would suggest looking at saveAndAddFilesToDataset in IngestServiceBean.18:01:09

Show newer messages

Back to Room List