Cloud storage 101 - Why you should avoid installing and maintaining databases by yourself

Edward Martins · Feb 20, 2020
about cloud storage and database as a service
about cloud storage and database as a service

Some time ago, cloud databases exploded the scene! It's so easy to turn it up that it began to rise up across most of the organizations empowering DevOps and DBAs. The disadvantages of maintaining a data cluster and automating server management in-house were so great that most businesses gave it little thought.

Manual management methods, like fine-tuning DBMS (Database Management Systems), tracking logs, applying patches and hotfixes for dozens of servers during a weekend or worse, in the middle of the night, now seem like unnecessary pain.

Today, the benefits of cloud databases far outweigh the costs of maintaining your own database servers, with an incredible array of additional services and solutions now available.

Growing with cloud database technology

Commonly, the design and construction of conventional systems, utilize data management and relational databases as their key building blocks.

Advanced queries expressed in SQL work well with the strict relationships that are required on information by relational databases. However, relational database technology was not initially designed or developed for use over distributed systems.

This issue has been addressed with the addition of clustering enhancements to the relational databases, although some basic tasks require complex and expensive protocols, such as with data synchronization.

What exactly is a cloud database?

A cloud database is by definition a database that typically runs on a database-as-a-service (DBaaS) solution. The DBaaS provider takes care of scalability and high availability of the database making the underlying software-stack transparent to the user.

DBaaS solution rises with the premises of a transparent and self-managed data flow that hides the data management complexity from the final user at the same time and that offers many benefits like:

Scalability: One of the most important key features of the DBaaS solution. Scalability allows adjusting operations upwards or downwards as IT requirements change. Scaling can be done quickly, cheaply, and efficiently, so even small businesses can find value in using the DBaaS.

Agility: Due to the fact that the majority of today’s DBaaS providers are based on Software as a Service (SaaS) architecture, there is a great deal of flexibility attached to accessing business-critical data in a DBaaS-powered cloud service. Data can be accessed quickly because the architecture where the data is located is fairly reliable. This also means that as the data structure needs shift, or geographical needs arise, the changes to data are incredibly easy to implement.

Cost-effectiveness over administrative flow: DBaaS solution doesn't eliminate the role of a database administrator, but it effectively can get rid of many features that typically consume a DBA’s time, allowing them to focus on more important tasks.

Cost-effectiveness over location issues: Solution providers and software developers in the DBaaS scenario can easily deploy their data delivering applications in a way that offsets many of the costs associated with managing and housing these complex data sets in-house. For instance, the DBaaS solution provider helps organizations save money. They can build them in such a way that makes it easy to change locations and organizational assets in a fluid way.

Big Data: The DBaaS solution model is expected to serve as a launching pad for the dynamic Business Intelligence (BI) and Big Data analytics markets. The BI market is fairly limited as a “build-driven” business model. It means that enterprise organizations merely license software so that they can build analytics on top of that software. DBaaS providers would enable companies to take account of features that integrate BI and analytics applications into their own architecture.

There's no doubt that any business can obtain impressive benefits from cloud databases.

How does Jexia provide a database as a service solution?

Jexia's DBaaS solution is unique in its kind because it provides both schema and schemaless database models and makes it possible to create relations between them. It's called a dataset!

Jexia dataset is a data storage provider where data is stored either using a strict schema or without it (schemaless). It allows manipulation of this data by creating, updating, fetching and deleting records and also to query them by using conditions like isEqualTo and isGreaterThan (full list available in FieldFilter API reference). By using this SQL-agnostic approach, you don't need to be an expert in SQL or in a specific database to be able to use Jexia dataset.

cloud database

Datasets REST APIs modules and Javascript SDK allows users to manipulate (create, read, update and delete) project data.

Let's get hands-on! - Create relations between datasets

Before we jump into it, if you don't have a Jexia account yet, you can create it here.

Okay, let's create a small banking system project that shows how it’s possible to see the relationship between the entities. See below the banking system example diagram that we're going to create:

Banking system use case

First thing first!

You have to create a project in the Jexia dashboard!

After login, create a project in a single click!

create project in jexia

After the project is created, it's possible to see the project ID and the project URL which is used to interact with data.

jexia project is created

Create datasets

In order to interact and perform CRUD operations on data, it is necessary to store data in a dataset. In order to do so, we need to create the dataset first.

jexia dataset

Let's add a dataset name "Customer", then click the button "Create dataset":

create datasets in jexia

Let's create more datasets for "Bank Employee", "Account", and "Funds":

create more datasets in jexia

Creating the dataset attributes (schema/schemaless)

Add fields to datasets or store data schemaless

Schema:

The next step is to add fields to datasets. In order to create a field, click the "Add field" button. In the same window, select name, type, and validation for the field. Also, it's possible to provide a default value for the field. Field name validation parameters can be changed via the edit field, but you can't change the field type. If it's necessary to change the type then it can only be by deleting it and creating a new field. However, by deleting the field the data stored in the field will also be lost. With the Schema approach in responding, there are specific types: String, Integer, Float, Date, DateTime, Boolean, JSON or UUID. Data will be validated against the validators added to the fields while inserting and updating records.

Schemaless:

To apply the schemaless approach, just insert JSON object into JEXIA without creating any fields inside Dataset. Data will be stored automatically with the proper type. Please, note that validations will not be applicable to schemaless data. After that, it’s possible to convert from Schemaless to Schema when data design for its project is stabilized. Jexia supports String, Integer, Float, Date, DateTime, Boolean, JSON and UUID as field types.

Validation:

Depends on field-type, validation parameters will be available. The most validators are available for String type, such as: Required, UpperCase, LowerCase, Alphanumeric, Numeric, Alpha, Min/ Max Length, RegEx.

For Float and Integer, there are Required, Min/Max Value.

When some validators are selected, another one might be unavailable. This is due to logical exclusion. For example, it is not logical to have Upper and Lower case validators at the same time. To reduce the possibility of human mistakes we decided to prevent selection for some combinations.

Adding fields to the datasets

Let's add fields to our "Customer" dataset:

adding fields to a dataset
edit fields on a dataset
edit fields on a dataset
edit fields on a dataset

Now let's add the fields to the "Bank_Employee" dataset:

add fields to the dataset
create field in dataset
edit fields on a dataset
edit fields on a dataset
edit fields on a dataset

Then let's add the fields to the "Account" dataset:

add fields on a dataset
edit fields on a dataset

Then we add fields to the "Funds" dataset:

add fields on a dataset
edit fields on a dataset
edit fields on a dataset

Creating datasets relations

Jexia allows you to relate two or more datasets and create one-to-many or many-to-many relationships between them. Relations are established under the hood, so don’t worry about specifying fields for relation.

While fetching the date, it's possible to get only parent record or parent + child records. For example: Post ← Comments ← User

Create a relation

There is a WYSIWIG editor that is used for this.

  • See Relations in the Services block in the side menu.
  • Then select a dataset and drag a line towards another dataset to add a relation to.
  • Set the type of relationship and that's it.

In order to create a relation between two datasets, head over to the "Relations" tab in the dashboard (see below):

Create a relation between "Customer" and "Account" datasets:

create relation between datasets
create relation between datasets

Now let's create a relation between "Bank Employee" and "Account" datasets:

create relations between datasets
create relations between datasets

Then we're creating a relation between "Funds" and "Account" datasets:

create relations between datasets
create relations between datasets

So, all relations will look like this:

Okay, they look beautiful!!

Working on the datasets

In order to access these datasets, it's necessary to create datasets Jexia API keys and Policies first. So, head over to the API keys tab in the dashboard, it looks like this:

create api keys in jexia

Then create an API key:

create api keys in jexia
create api keys in jexia
create api key in jexia

Then head over to the Policies tab in the dashboard to create policy. It looks like this:

create policies for API keys

Create policy:

create policies for API keys
create policies for API keys

To make things easy we will access the dataset endpoint directly by sending a JSON payload.

Getting the access token

POST - https://YOUR-PROJECT-ID.app.jexia.com/auth/

Heathers - Content-Type: application/json

Payload:

{
	"method": "apk", 
	"key": "YOUR-API-KEY", 
	"secret": "YOUR-SECRET-KEY"
}

Response:

{
    "access_token": "YOUR-ACCESS-TOKEN",
    "refresh_token": "YOUR-REFRESH-TOKEN"
}

Create Bank_Employee:

POST - https://YOUR-PROJECT-ID.app.jexia.com/ds/Bank_Employee

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Payload:

[
	{
		"Name": "Employee A",
		"Email":"ea@bankmail.com",
		"Birth_Date": "1982-10-12",
		"Agency":"123"
	}
]

Response:

[
    {
        "id": "a3246c44-34f2-48f4-b0ff-11e97e59a6ac",
        "created_at": "2020-01-24T14:25:07.293689Z",
        "updated_at": "2020-01-24T14:25:07.293689Z",
        "Name": "Employee A",
        "Email": "ea@bankmail.com",
        "Birth_Date": "1982-10-12",
        "Agency": 123
    }
]

Create both Account and Customer (nested payload):

POST - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Payload:

[
	{
		"Number": "001",
		"Customer":{
			"Name": "Customer A",
			"Email":"ca@mail.com",
			"Birth_Date": "1996-07-01" 
		}
	}
]

Response:

[
    {
        "id": "b177ab4c-e133-4e76-939f-696d4b31d8f6",
        "created_at": "2020-01-24T14:19:16.291117Z",
        "updated_at": "2020-01-24T14:19:16.291117Z",
        "Number": 1,
        "Customer": {
            "id": "88f2b4a9-9922-48a6-9e10-590e3b159390",
            "created_at": "2020-01-24T14:19:16.291117Z",
            "updated_at": "2020-01-24T14:19:16.291117Z",
            "Name": "Customer A",
            "Email": "ca@mail.com",
            "Birth_Date": "1996-07-01"
        }
    }
]

Attach Bank_Employee to the Customer Account:

PUT - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account?

outputs=["Bank_Employee"]

&action=attach

&cond=[{"field":"id"},"=","b177ab4c-e133-4e76-939f-696d4b31d8f6"]

&action_resource=Bank_Employee

&action_cond=[{"field":"id"},"=","a3246c44-34f2-48f4-b0ff-11e97e59a6ac"]

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Response:

[]

Get full Account information

GET - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account?

outputs=["Customer", "Bank_Employee"]

&cond=[{"field":"id"},"=","b177ab4c-e133-4e76-939f-696d4b31d8f6"]

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Response:

[
    {
        "id": "b177ab4c-e133-4e76-939f-696d4b31d8f6",
        "created_at": "2020-01-24T14:19:16.291117Z",
        "updated_at": "2020-01-24T14:19:16.291117Z",
        "Number": 1,
        "Customer": {
            "id": "88f2b4a9-9922-48a6-9e10-590e3b159390",
            "created_at": "2020-01-24T14:19:16.291117Z",
            "updated_at": "2020-01-24T14:19:16.291117Z",
            "Name": "Customer A",
            "Email": "ca@mail.com",
            "Birth_Date": "1996-07-01"
        },
        "Bank_Employee": {
            "id": "a3246c44-34f2-48f4-b0ff-11e97e59a6ac",
            "created_at": "2020-01-24T14:25:07.293689Z",
            "updated_at": "2020-01-24T14:25:07.293689Z",
            "Name": "Employee A",
            "Email": "ea@bankmail.com",
            "Birth_Date": "1982-10-12",
            "Agency": 123
        }
    }
]

Credit Fund

POST - https://YOUR-PROJECT-ID.app.jexia.com/ds/Funds

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Payload:

[
	{
		"Value": "100.10",
		"Operation_Type":"c"
	}
]

Response:

[
    {
        "id": "f4baae92-bc4b-424b-9ef2-86f9dc4aa502",
        "created_at": "2020-01-24T14:52:11.935828Z",
        "updated_at": "2020-01-24T14:52:11.935828Z",
        "Value": 100.1,
        "Operation_Type": "c"
    }
]

Attach Fund Operation to the Customer Account

PUT - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account?

outputs=["Funds"]

&action=attach

&cond=[{"field":"id"},"=","b177ab4c-e133-4e76-939f-696d4b31d8f6"]

&action_resource=Funds

&action_cond=[{"field":"id"},"=","f4baae92-bc4b-424b-9ef2-86f9dc4aa502"]

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Response:

[]

Debit Fund

POST - https://YOUR-PROJECT-ID.app.jexia.com/ds/Funds

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Payload:

[
	{
		"Value": "50.50",
		"Operation_Type":"c"
	}
]

Response:

[
    {
        "id": "87f1b0f7-9240-42c8-bc64-13b1af02bd80",
        "created_at": "2020-01-24T15:08:49.58834Z",
        "updated_at": "2020-01-24T15:08:49.58834Z",
        "Value": 50.5,
        "Operation_Type": "d"
    }
]

Attach Fund Operation to the Customer Account

PUT - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account?

outputs=["Funds"]

&action=attach

&cond=[{"field":"id"},"=","b177ab4c-e133-4e76-939f-696d4b31d8f6"]

&action_resource=Funds

&action_cond=[{"field":"id"},"=","87f1b0f7-9240-42c8-bc64-13b1af02bd80"]

Headers:

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Response:

[]

Get Full Account Funds Operations

GET - https://YOUR-PROJECT-ID.app.jexia.com/ds/Account?

outputs=["Customer", "Funds"]

&cond=[{"field":"id"},"=","b177ab4c-e133-4e76-939f-696d4b31d8f6"]

Headers: 

  • Content-Type: application/json
  • Authorization: Bearer YOUR-ACCESS-TOKEN

Response:

[
    {
        "id": "b177ab4c-e133-4e76-939f-696d4b31d8f6",
        "created_at": "2020-01-24T14:19:16.291117Z",
        "updated_at": "2020-01-24T14:19:16.291117Z",
        "Number": 1,
        "Customer": {
            "id": "88f2b4a9-9922-48a6-9e10-590e3b159390",
            "created_at": "2020-01-24T14:19:16.291117Z",
            "updated_at": "2020-01-24T14:19:16.291117Z",
            "Name": "Customer A",
            "Email": "ca@mail.com",
            "Birth_Date": "1996-07-01"
        },
        "Funds": [
            {
                "id": "f4baae92-bc4b-424b-9ef2-86f9dc4aa502",
                "created_at": "2020-01-24T14:52:11.935828Z",
                "updated_at": "2020-01-24T14:52:11.935828Z",
                "Value": 100.1,
                "Operation_Type": "c"
            },
            {
                "id": "87f1b0f7-9240-42c8-bc64-13b1af02bd80",
                "created_at": "2020-01-24T15:08:49.58834Z",
                "updated_at": "2020-01-24T15:08:49.58834Z",
                "Value": 50.5,
                "Operation_Type": "d"
            }
        ]
    }
]