Walking Through Azure || Part 9
If you are landing first time, this is the link to go for you. In this blog, we shall learn about accessing Microsoft Storage Queue Resource :-
Let’s talk about the Architecture, that we are going to discuss in this blog :-
- Users wants to upload the Videos to the Application.
- Application saves those videos into the DataStore.
- Module processes these videos from the DataStore.
Question → What are the problems associated with the afore-mentioned design & what’s the solution ?
Answer → If the Processing module crashes, then there is no way of keeping track of what all videos got processed and what all videos were not processed ? Therefore, the solution for this problem is to have an Storage-Queue in between of the following two components :-
- Video-Store.
- Video-Processing-Module → This module is the one, which actually process the video.
Question → Show the demo of using the Storage-Queue ?
Step #1.) We first open the Azure-Storage-Account :- In the left hand side pane, we can see the various options for Data-Storage within the Storage-Account :-
- Containers.
- File Shares.
- Queues.
- Tables.
Step #2.) We now go to the Queues option here and create a new queue with name as “appqueue” :-
Step #3.) We now go inside this Queues option and perform :-
- Add a new Message to this Queue.
- The Message sent to the Queue would expire in 7 days.
- The message sent is encoded in Base64 format.
Step #4.) We can now see that, the message is now successfully arrived into the Queue. We can also see the properties of the message which have been saved to the Queue.
Step #5.) Similarly, let’s push another message to the same Queue :-
Step #6.) Now, we can consume/dequeue the message from this Queue :- As soon as the message is being de-queued from the Queue, that message is gone forever.
After the dequeue operation is successful, we can see that, only one message is being left in the queue :-
Step #6.) Now, we can also clear the queue and this option shall finish all the messages from queue :
After we perform this operation, we can see that the queue is empty now :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code for pushing the messages to the Queue ?
Step #1.) We first go to our Visual-Studio-Code and Add New Project :-
Step #2.) We now create a Console-App :-
Step #3.) We now create the name of project as “storagequeue”.
Step #4.) We now add the package to our environment by this command :-
dotnet add package Azure.Storage.Queues
Step #4.) We now go to Azure Portal and take the ConnectionString for our Queue :-
Step #5.) We now can refer to the Azure documentation for pushing the messages to the Queue :-
Now, we write the code in C-Sharp language, which would connect to this Storage-Queue and push the messages to it :-
Step #6.) We now execute this code-piece & can see the messages being pushed to Queue :-
Step #7.) We now can verify from the Azure Portal that, our logs are getting pushed to the Azure Storage Queue :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for peeking (i.e. Just Reading) into the messages ?
Step #1.) We first write the code for Peeking the message from line no. 19 to 28 as shown below :-
Step #2.) Now, we execute our code and we can see that messages are being read from queue. Please note here that, these messages shall not be deleted from the Storage-Queue i.e. they would still be present into the Queue.
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for Reading & Deleting the messages ?
Step #1.) We first write the code for ReceiveMessage from the Queue. Here note that, we have not deleted the message after reading the same :-
Step #2.) We now execute this code and can see that, the messages are being read from the queue :-
Step #3.) Initially, there were around 5 messages in our Storage-Queue. We read the three messages, so we are now left with 2 messages in the queue :-
But note that, since we didn’t explicitly deleted the messages from the Queue, they shall re-appear into the queue after 30 seconds of Invisibility Timeout :-
This is because :-
- The code that we wrote above would only make the message Invisible for 30 seconds (default) so that no other consumer/instance should be able to read this message during this 30 second period.
- As a good practice, the consumer have to manually delete the message from the queue, after the message is being processed succesfully.
Step #4.) Now, we explicitly delete the message, after the processing is done successfully :-
Now, we re-run this code again :-
And we can see that, the messages which has been consumed shall not re-appear into the queue now. So, Message 2, 3, 4 are consumed successfully and Message 0, 1 are now remaining in the queue now.
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for knowing the length of the Queue ?
Step #1.) We first write the code for checking the length of the queue :-
Step #2.) If we execute this code now, we can see the output :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for pushing the object into the Queue ?
Step #1.) We first write the blueprint for Object (“Order”) that we wanted to push to the Queue :-
Step #2.) We now write the code to initialise the instances of this class and push to the queue :-
Step #3.) Now, if we execute this code, we can see, it is successfully pushing the messages into the Queue :-
Step #4.) We can also verify that, the messages are now visible into the Azure Dashboard :-
Here, we can see a sample message by clicking upon it :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for receiving the object from the Queue, using Azure Functions ?
Step #1.) We first create an Azure Function inside our Workspace :-
Next, We now create a new Project :-
Now, we select our Runtime as .NET :-
Here, we now select the : “Azure Queue Storage Trigger” :-
For this very purpose, we shall need a Storage Account and we shall be using the Azurite Emulator for Local Storage :-
Next, we specify the Queue Name :-
Step #2.) We now write the code for consuming from the Queue i.e. QueueTrigger. Note here that, we have consumed the message in the format of : QueueMessage :-
Step #3.) We now check the ConnectionString for our StorageAccount which we shall be configuring in our codebase for QueueTrigger :-
In below code-snippet, we configure the connection-url that we got from above dashboard :-
Step #4.) Now, let’s go ahead and consume from the Queue. Here we can see the logs which indicates something fishy. Our consumption have failed.
We can see that, there are no messages remaining in our queue :-
Also, these messages have automatically landed to an another queue which is also known as Poison-Queue. This Poison-Queue is something which got automatically created, because the messages could not be processed here :-
In order to debug this, let’s add logging to our code with mode as Trace :-
Now, if we try to consume the message again, we can see the error clearly that : The message decoding has failed.
Step #5.) Now, let’s go ahead and fix this issue at the time when the messages are being sent to the Queue. So, we perform the encoding of the message to Base64 format :-
Let’s run this code, so as to see whether messages are being sent to the queue :-
We can also verify the messages being received at the Azure dashboard :-
If we now re-run the QueueTrigger, we can see the messages consumed clearly :-
Also, we can see that, there are no messages remaining in our queue :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for receiving the object from the Queue, using Azure Functions , in the same format/Type as that of Object rather than Generic message format ?
Answer → Note that, we have already re-pushed the messages to this queue, before proceeding to consume from these messages :-
Step #1.) We first modify the code of our Azure Function, so as to receive the Object as it is, while consuming from the Queue :-
Step #2.) We now consume the message from this Queue and can see that, the messages started to be consumed :-
Question → Show the demo of using the Storage-Queue through the C-Sharp Code, for receiving the object from the Queue, using Azure Functions and writing the same into the Storage — Table ?
Step #1.) We first create a Table with name as : “Order” :-
Once the Table is created, we can see this on the dashboard :-
We can see that, as of now, there are no records in this Table :-
Step #2.) We now write the code of this Class, whose object shall be saved to the Table :-
Now, we write the code for consuming the messages from the Queue :-
Step #3.) We can see that, on line no. 20 above, we can see that, there is an error. Here, we install the Azure Extension for Tables using this command :-
dotnet add package Microsoft.Azure.Functions.Worker.Extensions.Tables
As we install the aforesaid extension, we can see that, this error is gone from the codebase :-
Step #4.) Finally, we can see that, while we execute the code of our Consumer again, we can see that the messages are being consumed successfully from the Queue and are being saved to the Table effectively.
We can also verify from the Azure Dashboard that, these records are now visible at the Azure Portal Dashboard as well :-
That’s all in this Blog. We shall see you in next blog. If you gained something from this blog, I encourage you to clap on this page.