AWS SQS AND ITS USE CASES

SQS stands for simple queue service it is run and managed by Amazon.

(SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message-oriented middleware and empowers developers to focus on differentiating work.

What is a message queue?

It is a program that stores information/data/messages in a queue to be used by another program. Another program can catch messages and data from the message queue and processes and to their further task based on that information.

How does SQS work?

Let’s assume an architecture with three servers named as producer and three servers named as a consumer now the producer server contains the file that needs to be processed and they are received by the user and stored in producer server but the processing is done in consumer server and so how will the consumer know that the file is available at the producer server and they have to fetch it for processing here the SQS comes In-play SQS contains all the information/message generated by the producer server and adds in the queue from here the consumer server reads the message in the queue and gets to know that we have to fetch the file and start processing it. This makes the workflow smooth as easy as there is proper communication between the server through the SQS.

There are two types of queues :

· Standard Queues

· FIFO (First in First Out) Queues

What are the Retention Period, Visibility Period, and Dead Letter Queue?

· Retention Period:- This is the period in which SQS will have the Message/information within their storage even after the message is retrieved(by default)

· Visibility Timeout:- In the above architecture we have discussed the three servers of the consumer side when one of them will pick up the message for processing the other one will not be able to see the message in the queue this is known as Visibility Timeout it is 30 Seconds by default we can change it accordingly.

· Dead Letter Queue:- In the above architecture when the consumer picks up the message and is not able to processes it till visibility period after the visibility period message again comes back to the queue and when one message is not able to process multiple time it is transferred to the Dead Letter Queue.

Functionalities of AWS SQS

· Unlimited queues and messages: Create unlimited Amazon SQS queues with an unlimited number of message in any region

· Payload Size: Message payloads can contain up to 256KB of text in any format. Each 64KB ‘chunk’ of a payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests. To send messages larger than 256KB, you can use the Amazon SQS Extended Client Library for Java, which uses Amazon S3 to store the message payload. A reference to the message payload is sent using SQS.

· Batches: Send, receive, or delete messages in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost-effective for customers that use batching.

· Long polling: Reduce extraneous polling to minimize costs while receiving new messages as quickly as possible. When your queue is empty, long-poll requests wait up to 20 seconds for the next message to arrive. Long poll requests cost the same amount as regular requests.

· Retain messages in queues for up to 14 days.

· Send and read messages simultaneously.

· Message locking: When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again.

· Queue Sharing: Securely share Amazon SQS queues anonymously or with specific AWS accounts. Queue sharing can also be restricted by IP address and time of day.

· Server-side encryption (SSE): Protect the contents of messages in Amazon SQS queues using keys managed in the AWS Key Management Service (AWS KMS). SSE encrypts messages as soon as Amazon SQS receives them. The messages are stored in encrypted form and Amazon SQS decrypts messages only when they are sent to an authorized consumer.

Dead Letter Queues (DLQ): Handle messages that have not been successfully processed by a consumer with Dead Letter Queues. When the maximum receive count is exceeded for a message it will be moved to the DLQ associated with the original queue. Set up separate consumer processes for DLQs which can help analyze and understand why