Scaling APIs

Handle increase load and demand, ensure availability, performance, and resilience of API services. Some strategies and solutions:

Vertical scaling: increases the capacity of a single server Horizontally scaling: distribute the load across multiple servers, often involving load balancing, replication, and partitioning. Load Management: includes caching, rate limiting, throttling, and load shedding. Not strictly a scaling strategy, but helps maintain service availability during high load.

Vertical Scaling

"Scaling up" an API involves increasing the resources of a single server to handle more load.

• Pros: easy to implement because you don't have to change the application and it can be an immediate performance boost.
• Cons: maximum limit to how much you can scale and it often takes some downtime to upgrade the hardware.

Compared to other techniques, its still considered very simple, so if this can achieve your requirements, go for this.

Horizontal Scaling

To "scale out," it involves adding more servers to distribute the load, enhancing the system's ability to handle more requests.

• Pros: handle more load by adding more servers, and failure in one server doesn't affect the others
• Cons: requires sophisticated setup and management (need load balancing) and it can be challenging to ensure data consistency across multiple servers.

Strategies include the following: Server replication:

• identical copies to distribute load and provide redundancy
• helps in distributing load and provides redundancy
• ensure continuous service availability in case of server failure

Every time you replicate, you need to load balance because you need to direct requests to different servers

Load balancing:

• distribute network traffic uniformly across multiple servers
• prevents any one server from being overwhelmed and optimizes resource use
• enhance application responsiveness and availability.

Not necessarily a 1:1 relationship, you can have many load balancer Think about using micro service architecture.

Partitioning APIs Divide server's duties and responsibilities, often by the type of request or data being handled. You can divide it by some criteria so that each server manages something different so that it improves efficiency and provides independence, which helps the system be more manageable and scalable. Some examples:

• It can be partitioned by resource one server manages everything about posts, one about comments, etc.
• Or maybe it's about domain, one manages post sharing, shopping cart, etc.
• Different API routes or endpoints can be handled by different servers that are optimized for that specific set of tasks or data.

This allows partitions to operate independently, improving efficiency, and making the system more manageable and scalable.

Application in APIs: different API routes or endpoints can be handled by different servers, each optimized for a specific set of tasks or data.

Requires redesigning of the entire application, takes a lot of work. Consider switching to managed service at this point.

API Gateway manages and handles API requests, acting as a reverse proxy to forward requests to the appropriate service

Microservice architecture: decomposing the API into smaller, independent service that can be scaled individually.

API Gateway

API Gateway acts as a reverse proxy (sits in front of the partitioned APIs and orchestrates how they work) to accept all API calls, aggregate the various services required to fulfill them, and return the appropriate result.

• Single entry point: From the client's perspective, there's one unified API entry point, and the gateway sends that response around. Enables the routing of requests to the appropriate downstream service.

• Request management: can manage and handle various tasks like request routing, security, analytics, monitoring, composition, etc.

• Micro-services architecture: beneficial in a micro-services architecture where it can route requests to appropriate micro-services

Different from a load balancer because it serves different purposes in architecture:

• api gateway is a reverse proxy for API calls. It provides a single entry so its job is not necessarily to balance the load, its job is to identify where a request goes because of its purpose. Typically, contains a load balancer.
• load balancer: distributes incoming network traffic across multiple servers to ensure high availability and scalability.

AWS API Gateway is a managed service that makes it easy for developers to create, deploy, secure, and monitor APIs at any scale.

• Use a managed service for ease. It's user friendly, enabling developers to focus on coding.
• Security and compliance: built-in security features like authentication and authorization enables secure request handling.
• Serverless integration: seamless integration with AWS Lambda creates serverless APIs, reducing server management overhead.
• Performance at scale: handles concurrent API calls, including traffic management, authorization, monitoring, and API version management.

Load Management Strategies

Strategies to improve availability.

Rate limiting: restrict the number of requests any user can make in a given time frame to prevent abuse and ensure availability for all users. E.g., 50x a minute is the limit of the rate you can hit the API.

• control traffic: helps control the traffic reaching the server so that it doesn't get overwhelmed by too many requests.
• maintain service availability: because the server will not be overwhelmed, the service can remain available to all users
• resource optimization: the server can optimize the allocation of its resources to handle each incoming request effectively.
• implemented typically using a token bucket algorithm or leaky bucket algorithm, configurable in the API gateway.

Throttling: adjust the speed at which a user can make requests, so that you can allocate resources fairly among all users during high traffic. It delays the handling of requests

• regulates request processing: helps in managing the server's load by delaying the processing of some incoming requests
• prevent server overload: when you slow down request processing, it ensures that server is not overwhelmed
• enhance system stability: maintain the stability and reliability of the system by avoiding resource exhaustion
• implementation approach: implemented by setting a delay in processing requests, and typically configured in the API gateway.

Load shedding: if the system is under heavy use. Better for some users to not access the application than to halt the whole system. Drop requests when the server is overwhelmed, allowing you to maintain availability during spikes in traffic.

• maintains system stability: helps during traffic spikes by rejecting excess requests
• protect critical resources: ensures critical system resources remain available for high-priority tasks
• enhance service resilience: contributes to the overall resilience of the system by preventing cascading failures due to resource exhaustion
• implementation approach: monitoring system load and defining thresholds at which requests will be dropped, configured the API Gateway.

Microservices

A microservices architecture involves developing a single application as a suite of small services, each running in its own process and communicating through APIs.

They should be

• independently deployable: allows for easier updates and maintenance
• single responsibility: performs a specific business function and has a single responsibility
• decoupled service: loosely coupled enables them to develop, deploy, and scale independently.

A service can be scaled independently, and services can be swapped out. Responsibilities can be reused in different applications.

Service communication in microservice. Services uses various mechanisms to interact, maintaining application coherence and reliability.

• varied mechanisms: choose the right API like HTTP/REST, GraphQL, etc. to interact effectively.
• asynchronous messaging: message brokers and event streams enable decoupled, scalable, asynchronous, inter-service communication
• protocol and format selection: choose the right communication protocol and data format is vital, affecting performance and interoperability.

Don't jump into microservices if your application is super small. This would be overengineering.

• that's why there are consultants, who use their years of experience to tell you how to architect it.
• you don't want to waste time if you don't need it to be that complex. The best is always the middle ground.

Microservice architecture aids in building scalable and resilient systems by allowing individual components to scale independently based on their resource requirements and usage patterns:

• independent scaling: each service scales depending on its resource utilization and what loads
• enhanced fault isolation: one service's fault doesn't impact others directly, improving system resilience
• complexity management: introduces complexity in terms of service interactions, deployment, and monitoring, requiring effective management strategies

Caching

Caching and API scaling. Caching reduces the load and contributes to efficient API scaling.

• reduce load: reduce the number of requests to server, minimizing resource utilization.
• enhanced response times: cached responses are faster (better UX and service efficiency)
• lowered database stress: by serving cached data, the number of read requests to the database is decreased, thereby preventing potential bottlenecks.

CDN enhances API scalability by distributing the load, serving static content close to the user, which reduces latency.

• distributed load: CDN distributes the load by serving content from the nearest location to the user, thus reducing stress on the origin server.
• reduced latency: serving content closer reduces latency, increasing UX.
• enhanced reliability: provides redundancy, enhancing reliability and availability in times of high traffic and failures.

AutoScaling optimizes server instances based on load, increasing API adaptability and efficiency.

• adaptive management: adjusts resources in real time for optimal utilization
• high availability: ensure service is available during demand spikes by allocating necessary resources
• cost-efficient: reduce cost by allocating resources only when needed, avoiding over-provisioning.