In this getting started guide we will go over what you need to do to get started with our object storage.
We will go through how you use is via the CLI and the GUI, we will also go through some of object storage functions.
What APIs do you support?
We support S3 and Swift, se the below list of endpoints. If you are working with the aws
CLI you are working against the S3 API, if you are using the openstack
CLI or any of our web interfaces Swift API is used.
In S3 a bucket is what stores your objects, in Swift that is called a container. Buckets/containers created with either API is available in the other one.
Where can I store my data?
Today we have object storage available in the europe-se-1 (eu-se-1) region. Object storage in different availability zones does not share any underlying storage platform.
We support both standalone object storage in each availability zone and a replicated object storage where data is replicated in all availability zones in europe-se-1 (eu-se-1)
If you are working against object storage in any of our web interface/GUI they are always using the standalone object storage in eu-se-1a, to work against any other availability zone you need to use an CLI or API.
Standalone object storage
Supports all the available storage policies. If no storage policy is explicitly selected the gp.recurring is used by default.
Region | Availability Zone | S3 endpoint | Swift endpoint |
europe-se-1 (eu-se-1) | europe-se-1a (eu-se-1a) | https://object-eu-se-1a.binero.cloud | https://object-eu-se-1a.binero.cloud/swift/v1/AUTH_%(tenant_id)s |
europe-se-1 (eu-se-1) | europe-se-1b (eu-se-1b) | https://object-eu-se-1b.binero.cloud | https://object-eu-se-1b.binero.cloud/swift/v1/AUTH_%(tenant_id)s |
Replicated object storage
Replicated object storage only supports the gp.recurring and hp.intensive storage policies. If no storage policy is explicitly selected the gp.recurring is used by default.
All buckets/containers and objects created or uploaded is replicated to all availability zones.
Most objects is replicated within 15 minutes, but sometimes replication can take longer. Replication is an asynchronous process meaning the objects are eventually replicated following the eventual consistency model.
Region | Availability Zone | S3 endpoint | Swift endpoint |
europe-se-1 (eu-se-1) | europe-se-1a (eu-se-1a) | https://object-eu-se-1a-rep.binero.cloud | https://object-eu-se-1a-rep.binero.cloud/swift/v1/AUTH_%(tenant_id)s |
europe-se-1 (eu-se-1) | europe-se-1b (eu-se-1b) | https://object-eu-se-1b-rep.binero.cloud | https://object-eu-se-1b-rep.binero.cloud/swift/v1/AUTH_%(tenant_id)s |
Storage policies
We have different storage policies, in the table below you can see what storage policy suits your use-case most.
If no storage policy is explicitly selected the gp.recurring is used by default.
hp.intensive | gp.recurring | gp.intermittent | gp.archive | |
Data type | Primary | Primary | Primary/Secondary | Secondary/Backups |
Data set size | Small to medium | Small to huge | Medium to huge | Large to huge |
Data accessed | Constantly | Constantly | Intermittently | Seldom |
Optimized | Fast access and low latency | Recurring access | Intermittent access | Long time storage |
Storage medium | SSD/Flash | HDD/Mechanical | HDD/Mechanical | HDD/Mechanical |
Data reliability | 99.999% | 99.999% | 99.999% | 99.9% |
Replication | X* | X* |
Prerequisites
To get started with CLI you need to install AWS CLI or similar. In this guide we will use the AWS CLI. The client is available for download herehttps://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
When you have installed the AWS CLI you can confirm that it works by running aws --version
To use openstack
commands with CLI you also need to have that installed, information about that is available here: https://docs.openstack.org/python-openstackclient
Getting started
Since we have the aws
and openstack
CLI installed we can start by creating our access and secret key credentials that is required to authenticate when working with the object storage. We do this with the command openstack ec2 credentials create
, we will then get an answer similar to this
+------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | access | 09876543210987654321098765432109 | | links | {u'self': u'http://auth.binero.cloud:5000/v3/users/01234567890123456789012345678901/credentials/OS-EC2/09876543210987654321098765432109'} | | project_id | abc123abc123abc123abc123abc123ab | | secret | cba321cba321cba321cba321cba321cb | | trust_id | None | | user_id | 01234567890123456789012345678901 | +------------+-------------------------------------------------------------------------------------------------------------------------------------------+
We need to save the access and secret in a configuration file to use the cli as an authenticated user, that file is saved here on linux ~/.aws/credentials
and must contain:
[default] aws_access_key_id=aws_secret_access_key=
We are now ready to create our first bucket/container. We can do this through Horizon (OpenStack Dashboard) or using the S3 API with the aws
CLI.
To create a Container in Horizon we go to Project -> Object Store -> Containers
then select +Container
. There you can name your container and choose a storage policy.
We can also create it with the aws
CLI, like shown below:
aws --endpoint=https://object-eu-se-1a.binero.cloud s3api create-bucket --bucket BUCKET_NAME --create-bucket-configuration LocationConstraint=europe-se-1:gp.recurring
After this you will se the newly created container/bucket on the Containers page in Horizon
Or using the aws
CLI
demo@binero:~$ aws --endpoint=https://object-eu-se-1a.binero.cloud s3 ls 2020-10-07 13:40:00 DEMO