Guide for Integrating Storj with Dataverse

Storj Setup

After creating an account on Storj, you'll need to generate an access grant in order to get S3 credentials.

Generate S3 credentials

S3 credentials allow Dataverse to upload and download files from Storj as if it was using the S3 API. To create credentials:

Click 'Access' on the left menu
Select 'Create S3 Credentials'

Give your access grant a name such as 'My S3 Creds' and generate a passphrase. Your credentials will be created. These will be used to configure your Dataverse server.

Credentials will only appear once. Be sure to store them, including the Access Key and Secret Key, securely to prevent unwanted access to your files as well as allow you to configure your AWS credentials profile and to decrypt your files later.

Dataverse Setup

Configure AWS credentials profile

Dataverse will use the "AWS credentials profile file" that is configured for the user you run the Payara service with (e.g. /home/dataverse/.aws/config).

Enter the Storj S3 credentials using aws configure

aws configure
  AWS Access Key ID [None]: <Access Key>
  AWS Secret Access Key [None]: <Secret Key>
  Default region name [None]: us-1
  Default output format [None]:

aws configure
  AWS Access Key ID [None]: <Access Key>
  AWS Secret Access Key [None]: <Secret Key>
  Default region name [None]: us-1
  Default output format [None]:

Alternatively, edit the ~/.aws/config file directly.

[default]
aws_access_key_id = <Access Key>
aws_secret_access_key = <Secret Key>
region = us-1

[default]
aws_access_key_id = <Access Key>
aws_secret_access_key = <Secret Key>
region = us-1

If you have existing S3 credentials, create an alternative profile for Storj.

Create Bucket

If you choose a different bucket name, be sure to change it in the jvm-options below as well.

Create a bucket called dataverse to store your files.

aws --endpoint-url=https://gateway.storjshare.io s3 mb s3://dataverse

aws --endpoint-url=https://gateway.storjshare.io s3 mb s3://dataverse

Configure Storj option in Dataverse

Using asadmin (usually located at /usr/local/payara5/bin/asadmin) run the following commands.

./asadmin create-jvm-options "\-Ddataverse.files.storj.type=s3"
./asadmin create-jvm-options "\-Ddataverse.files.storj.label=Storj"
./asadmin create-jvm-options "\-Ddataverse.files.storj.custom-endpoint-url=gateway.storjshare.io"
./asadmin create-jvm-options "\-Ddataverse.files.storj.custom-endpoint-region=us-1"

./asadmin create-jvm-options "\-Ddataverse.files.storj.bucket-name=dataverse"
./asadmin create-jvm-options "\-Ddataverse.files.storj.profile=default"
./asadmin create-jvm-options "\-Ddataverse.files.storj.path-style-access=false"

./asadmin create-jvm-options "\-Ddataverse.files.storj.payload-signing=false"
./asadmin create-jvm-options "\-Ddataverse.files.storj.upload-redirect=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.chunked-encoding=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.download-redirect=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.min-part-size=67198864"
./asadmin create-jvm-options "\-Ddataverse.files.storj.url-expiration-minutes=120"

./asadmin create-jvm-options "\-Ddataverse.files.storj.type=s3"
./asadmin create-jvm-options "\-Ddataverse.files.storj.label=Storj"
./asadmin create-jvm-options "\-Ddataverse.files.storj.custom-endpoint-url=gateway.storjshare.io"
./asadmin create-jvm-options "\-Ddataverse.files.storj.custom-endpoint-region=us-1"

./asadmin create-jvm-options "\-Ddataverse.files.storj.bucket-name=dataverse"
./asadmin create-jvm-options "\-Ddataverse.files.storj.profile=default"
./asadmin create-jvm-options "\-Ddataverse.files.storj.path-style-access=false"

./asadmin create-jvm-options "\-Ddataverse.files.storj.payload-signing=false"
./asadmin create-jvm-options "\-Ddataverse.files.storj.upload-redirect=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.chunked-encoding=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.download-redirect=true"
./asadmin create-jvm-options "\-Ddataverse.files.storj.min-part-size=67198864"
./asadmin create-jvm-options "\-Ddataverse.files.storj.url-expiration-minutes=120"

(Optional) Make Storj the default

You'll need to delete the existing default storage driver and set a new one using jvm options:

./asadmin delete-jvm-options "-Ddataverse.files.storage-driver-id=file"
./asadmin create-jvm-options "-Ddataverse.files.storage-driver-id=storj"

./asadmin delete-jvm-options "-Ddataverse.files.storage-driver-id=file"
./asadmin create-jvm-options "-Ddataverse.files.storage-driver-id=storj"

Create new Dataverse using Storj

After you've configured your server, the Storj storage option will appear as a when creating a new Dataverse.