!AmypvmJtUjBesRrnLM:matrix.org

dataverse

59 Members
Dataverse is open source research data repository software: http://dataverse.org Public logs at https://view.matrix.org/alias/%23dataverse:matrix.org2 Servers

Load older messages


SenderMessageTime
21 Jan 2022
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
when you enabled direct uploads, did you enable CORS on the bucket?
* i enabled the CORS but while uploading via gui I am getting "Upload Unsuccessful (0:Network Error)"
12:18:45
@donsizemore:matrix.orgDon Sizemoreis your Ceph host reachable by the uploading client?12:20:49
@lincoln55:matrix.orgLincolnno direct access to client12:21:23
@donsizemore:matrix.orgDon SizemoreI believe that's the problem. To use upload- and download redirects, the storage must be directly accessible.12:21:52
@donsizemore:matrix.orgDon SizemoreJim Myers could tell you more, either on dataverse-users or in the community Slack12:22:30
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
is your Ceph host reachable by the uploading client?
just to be clear, do you mean User having PUT and GET access to buckets location?
12:24:07
@donsizemore:matrix.orgDon SizemoreMy understanding is that, in case of direct upload or download, Dataverse generates a pre-signed URL which it hands off to the client.12:24:36
@donsizemore:matrix.orgDon SizemoreSo if the client can't connect to the storage service, the request will fail.12:24:58
@lincoln55:matrix.orgLincolnthank you for the info... Any workarounds12:27:27
@lincoln55:matrix.orgLincolnviz: Reverse Proxy12:27:48
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
My understanding is that, in case of direct upload or download, Dataverse generates a pre-signed URL which it hands off to the client.
Also i as an admin am also cannot upload any files directly via S3
12:28:47
@donsizemore:matrix.orgDon SizemoreI vaguely remember mailing list chatter about such a solution, but it's been a while.12:28:59
@donsizemore:matrix.orgDon SizemoreIf you suspect trouble with the S3 service, can you manually list and put to Ceph via CLI as your Dataverse service account?12:29:22
@lincoln55:matrix.orgLincolni can manually list down buckets via CLI 12:30:43
@donsizemore:matrix.orgDon Sizemorecan you manually upload a test file/12:31:01
@lincoln55:matrix.orgLincolnwith upload-redirect = true in domain.xml file..... No orselse i have uploaded till 2GB of files on s3 (but via payara)12:31:48
@donsizemore:matrix.orgDon Sizemorewhat did you mean when you said "Also i as an admin am also cannot upload any files directly via S3"12:32:53
@donsizemore:matrix.orgDon Sizemoreyou'll want the payara/dataverse service account user to be able to list/read/write files, and also to update object tagging12:33:25
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
what did you mean when you said "Also i as an admin am also cannot upload any files directly via S3"
i mean when "upload-redirect= true" I cannot upload via gui to s3 as admin as well
12:35:46
@donsizemore:matrix.orgDon Sizemorebut, as the dataverse/payara service account user, you can manually upload a file?12:36:17
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
but, as the dataverse/payara service account user, you can manually upload a file?
.let me check again
12:39:48
@lincoln55:matrix.orgLincolnplease 12:40:18
@donsizemore:matrix.orgDon SizemoreI was just double-checking because of the "admin cannot upload" sentence, but it sounds like you'll want to experiment with a reverse proxy, or leave direct upload and download disabled.12:40:46
@lincoln55:matrix.orgLincolnthe idea is to enable direct upload-dwnload for all dataverse users12:41:49
@lincoln55:matrix.orgLincoln
In reply to @donsizemore:matrix.org
but, as the dataverse/payara service account user, you can manually upload a file?
sadly No
12:48:42
@donsizemore:matrix.orgDon SizemoreArizona had trouble like this with Dell S3 storage - they solved it by updating the firmware12:53:31
@lincoln55:matrix.orgLincolnoh12:56:30
24 Jan 2022
@nightowlaz:matrix.orgnightowlazhey there, just saw this (i'm from arizona) .. we got the dell s3 storage working without direct upload, but when we enable direct upload we get the same error (0network error) .. I believe that is because the s3 bucket is on a private subnet in the OneFS software .. waiting for the admins of the storage to add a public zone so we can test that theory .. would like to set up a reverse proxy to see if that works but not sure how to do that 03:45:51
@nightowlaz:matrix.orgnightowlazso we don't have to make the bucket public .. or something like CORS03:46:59
@lincoln55:matrix.orgLincoln
In reply to @nightowlaz:matrix.org
hey there, just saw this (i'm from arizona) .. we got the dell s3 storage working without direct upload, but when we enable direct upload we get the same error (0network error) .. I believe that is because the s3 bucket is on a private subnet in the OneFS software .. waiting for the admins of the storage to add a public zone so we can test that theory .. would like to set up a reverse proxy to see if that works but not sure how to do that
nightowlaz: thank you for the info...( i am also in the same scenario)
08:01:46

There are no newer messages yet.


Back to Room List