Relational databases are easy to scale vertically, but not horizontally. So its hard to create distributed databases with these because its hard to adhere to ACID principles. This is why non relational was made.

Non-Relational Databases

Document

Document stores save data in document-like structures, typically JSON.

• flexible schemas different files can have different structures, so they are pretty easy to use (but could be sloppy)
• can have embedded arrays or documents, providing depth
• suitable for hierarchical data structure.
• designed with distributed databases in mind, so they scale easier than relational databases

Analogy: think about a file folder structure (instead of a table like in relational). There are files instead of rows.

BSON is encoded JSON in order to save space in binary form.

It's common to use outside libraries to perform checks on your document database, to enforce a schema. Like in MongoDB, you use Mongoose.

The most popular NoSQL database is definitely document database.

When to choose document over relational databases?

• If you don't need to access child data with the context of its parents or its relationships, you should use a document database. The relationship aspect is not needed. In return, you get speed.

Key-Value Stores

Key-Value Stores are simple database systems that use a dictionary-like structure.

• Suitable where data access patterns are predictable. coo

Data is stored as a set of unique keys, each associated with a specific value.

• This makes them optimized for rapid data retriever when using the key

Redis is an example of a key-value store. It's usually used for caching.

Graph Stores

Graph databases store data in nodes (entities) and edges (relationships).

• suited for highly connected data
• e.g., social networks (node = person, edge = relationship), recommendation engines, and complex relational data

Queries can traverse multiple nodes and relationships, fetching complex patterns.

• Perform higher order analysis of your data.

History: This concept dates back to the 18th century.

• Modern graph databases emerged in the 2000s, with the rise of social networks. The need to model complex relationships and interconnected data drove their adoption.
• Fraud detection (trace the money) and recommendation systems also grew its significance.

Used when you require a network for massive inter-connected data where you want to make a complex query.

Columnar Stores

Organize data by columns rather than rows.

This design is for efficient read operations

• compression techniques optimize storage and enhance query performance
• not optimized for frequent write or update operations

Used for analytics

• Looking at a particular attribute (like duration when considering analytics). Unlike rows, when you retrieve, you don't have to grab the other attributes.

Big Data

Big Data solutions must handle vast volumes, diverse formats, and span distributed storage.

• it's more than just about storage: we need efficient processing, analysis, and deriving insights.

As data grows exponentially, a need arises for solutions beyond even columnar databases.

Data warehouses are central repositories optimized for analytics and reporting.

• The data it has is cleaned, transformed, and structured for performance
• Example: Amazon Redshift, GoogleBigQuery, Snowflakes
• You typically have a daily database, but when you need to find pattern/business intelligence, then you dump it into a data warehouse. Then you use tools to analyze it.

Data Lakes are vast repository in its raw form.

• It accepts structured, semi-structured, or unstructured data
• You typically dump data in lakes, then clean it up and put it in warehouse

Map Reduce is a paradigm tailored for distributed data processing.

• Map: process chunks of data, producing key-value pairs
• Reduce: aggregates these pairs to derive meaningful results.
• Designed for handling vast datasets efficiently across distributed systems. Usually operates on the lakes.
• Popularized by Google
• Allows you to parallelize

Object Store

Object stores store data as objects with unique identifiers and metadata

• Scales horizontally and is optimized for retrieval and storage rather than complex queries.
• Examples: Multimedia, Images - use object store. But the identifier should be associated in the database.

What they're good for; database vs object store

• Databases: structured data + complex queries
• Object storage: focus on storing vast amounts of unstructured data

Notable database systems:

• wide-column stores: Cassandra, ScyllaDB. Designed for high availability and scalability across many nodes.