The Azure Service Bus Queue feature offers easy to implement infrastructure for implementing brokered messaging communication.

Implementing a Message Queue requires the implementation of at least two application components: a Sender and a Receiver. The Sender will send messages to the Queue, while the Receiver will pull messages from the Queue to take action on. Both of these happen Asynchronously when working with Microsoft Azure Service Bus Queues.

Microsoft Service Bus Queues work on a “first in, first out” model. The first message into the Queue will be the first sent out when a message receiver checks the queue for messages.The above diagram is a basic representation of the various application components necessary to implement Azure Service Bus Queues. This article additionally describes what the benefits of using a Message Queue are, as well as the necessary .NET source code for implementing an Azure Service Bus Queue within an application.

Benefits of Message Queues

There are a few benefits that Message Queues offer when building larger enterprise applications / systems. While there is a little more design, implementation and maintenance overhead involved with using a message queue; the benefits can definitely outweigh and offer a much improved architecture. However, not all applications will see the full benefits of using a message queue, so application architecture needs to be fully evaluated before deciding to implement a message queues into any system.

Asynchronous Processing

When messages are sent to and stored within the Message Queue, they are reliably stored there until they are pulled from the queue. This allows for the various components of an enterprise system to be decoupled from relying on each other component from processing and returns the results of the messages in real time. This decoupling facilitates asynchronous processing of messages and event. This asynchronous processing additionally allows for one component to be taken down for maintenance without affecting the other from still functioning as normal.

Load Balancing and Leveling

The decoupling of the sender and receiver components of the system extends to additional benefits when it comes to leveling load spikes, as well as scaling up component instances to better handle system growth.

Load Balancing

If there happens to be an increase in traffic and/or load of a single component within the enterprise system using a Message Queue, then that individual component can be scaled up by adding more instances to better handle the amount of load being imposed on the system. This allows for a better utilization of resources since the specific Virtual Machines and Application Services associated with only the system component under high load stress need to be scaled up.

Load Leveling

The processing of events and messages within the system can be performed in a consistent pace across all hours of each day of the week. This eliminates some of the need to scale up and load balance application instances which can help reduce hosting costs by handling the application load with smaller Virtual Machines and/or Servers.

The load of the applications sending messaging to the Message Queue can be variable throughout the day. Then one or more message receivers can consistently pull and handle messages from the Message Queue at a constant rate. As long as the average of messages that are able to be handled per day is equal or greater than the average of messages being sent to the Message Queue, then there might not need to be any load balancing or application instance scaling necessary.

Azure Service Bus Nuget Package

The API’s for working with the Microsoft Azure Service Bus are contained within the “WindowsAzure.ServiceBus” Nuget package. Adding a reference to this Nuget package will easily pull in all necessary dependencies that will enable the application to communicate, manage and work with the Microsoft Azure Service Bus; including Service Bus Queues, Topics, Subscriptions and other features.

Once a reference to the Microsoft Azure Service Bus Nuget Package is added to the Visual Studio project, then everything will be ready to start coding against the Azure Service Bus Queue.

Service Bus Namespace

An Azure Service Bus Namespace must first be created before a Service Bus Queue can be created and managed. Once the Service Bus Namespace has been created, then a Shared Access Policy can be used to create, manage and work with a Queue on the Namespace. The “RootManagerSharedAccessKey” that is created by default when creating the Namespace is the simplest to use (as seen below), or a custom Shared Access Policy can be created if necessary.

All the Microsoft Azure Service Bus features reside within an Azure Service Bus Namespace. This includes Queues, Topics, Relays and Event Hubs. Below are code examples of working with an Azure Service Bus Queue.

Creating Service Bus Queue from .NET

The simplest way to create a Microsoft Azure Service Bus Queue is to just log into the Management Portal and create it through the UI. A couple more options would be to use either PowerShell or the Cross Platform CLI to create a new Service Bus Queue. Since this post is focusing on doing things from .NET code.

The “NamespaceManager” and other classes for managing Service Bus features are contained within the “Microsoft.ServiceBus” .NET namespace, so adding a using declaration to the code will help:

using Microsoft.ServiceBus;

Here’s an example of creating an Azure Service Bus Queue from .NET.

var serviceBusNamespace = "my-awesome-namespace";

var sharedAccessPolicyName = "RootManageSharedAccessKey";
var sharedAccessPolicyKey = "QK0axFFfn+lJgmJ8YzJRJSp/y/t3k8XU1+FlQU5BR/k=";

var serviceBusUri = ServiceBusEnvironment.CreateServiceUri(
  "sb",
  serviceBusNamespace,
  string.Empty);

var token = TokenProvider.CreateSharedAccessSignatureTokenProvider(
  sharedAccessPolicyName,
  sharedAccessPolicyKey);

// Get reference to the Service Bus Namespace
var namespaceManager = new NamespaceManager(
  serviceBusUri, token);

// Create the Service Bus Queue
namespaceManager.CreateQueue("SuperQueue");

Something that can be useful if the application always attempts to create the Azure Service Bus Queue before using it is to first check if it already exists. Here’s a short snippet that will check if the Service Bus Queue already exists before creating it; if so then it just doesn’t create it.

// Create the Service Bus Queue if it doesn't exist
if (!namespaceManager.QueueExists("SuperQueue"))
{
  namespaceManager.CreateQueue("SuperQueue");
}

Send Messages to the Queue

A message must be prepared before it can be sent to the Azure Service Bus Queue. The class object used to create messages is the “BrokeredMessage” class. The instance of the “BrokeredMessage” class created will contain the full message data

var messageBody = new MessageBodyObject();

var msg = new BrokeredMessage(messageBody);

msg.Label = "ImportSpreadsheet";

msg.Properties["FileName"] = "file.xlsx";

The “BrokeredMessage” has a couple things note on it’s properties. The Constructor accepts an argument that will make up the Body of the message, and can be any object that will get serialized so it can be transported. The “Label” and “Properties” Dictionary can be used to added metadata to the message that the message receiving code can read out.

The “BrokeredMessage” object is contained within the “Microsoft.ServiceBus.Messaging” namespace.

using Microsoft.ServiceBus.Messaging;

Once the message has been prepared, it can then be sent to the Azure Service Bus Queue. To send the message a “MessageSender” needs to be created and this can be done using a “MessagingFactory”.

var queueName = "SuperQueue";
var msgFactory = MessagingFactory.Create(serviceBusUri, token);
var msgSender = msgFactory.CreateMessageSender(queueName);

await msgSender.SendAsync(msg);
//msgSender.Send(msg);

There are both Synchronous (via “Send”) method and Asynchronous (via “SendAsync”) method when using async/await to call on the “MessageSender” class instance to send the message to the Queue.

Receive Messages from the Queue

Similarly to sending messages to the Azure Service Bus Queue, the “MessagingFactory” can be used to create a “MessageReceiver” that exposes both a Synchronous (via “Receive”) method and an Asynchronous (via “ReceiveAsync”) method when using async/await that can be used to retrieve messages from the Service Bus Queue.

var queueName = "SuperQueue";
var msgFactory = MessagingFactory.Create(serviceBusUri, token);
var msgReceiver = msgFactory.CreateMessageReceiver(queueName);

var msg = msgReceiver.ReceiveAsync();
// var msg = msgReceiver.Receive();

try
{
  ProcessMessage(msg);

  // mark message complete so it's removed from Queue
  msg.Complete();
}
catch (Exception ex)
{
  // log exception

  // Abandon message so it stays in Queue
  msg.Abandon();
}

It is important that the message handling / processing is wrapped within a try/catch so that the message can be marked Completed or Abandoned accordingly once processed. When the message is marked Completed by calling the “Complete” method on the BrokeredMessage object, it will be removed from the Queue. When marked Abandoned by calling the “Abandon” method, it will remain in the Queue so it can be processed accordingly. This allows for the Azure Service Bus Queue to remain reliable that all messages sent to the Queue can be processed successfully before being removed.

QueueClient to Send and Receiver Messages

Usually separate applications would be written to handle Sending and Receiving messages from the Azure Service Bus Queue, however this isn’t required. The QueueClient class can be used to both Send and Receiver messages from within the same code. This can help simplify the code if an application needs to both send and receive messages.

var queueName = "SuperQueue";
var msgFactory = MessagingFactory.Create(serviceBusUri, token);

// Create QueueClient to both Send and Receiver messages
var queueClient = msgFactory.CreateQueueClient(queueName);

// Send BrokeredMessage to the Queue
queueClient.Send(msgSent);

// Pull a BrokeredMessage from the Queue
var msg = queueClient.Receive();

try
{
  // Handle message
  ProcessMessage(msg);

  // mark message complete to it's removed from Queue
  msg.Complete();
}
catch(Exception ex)
{
  // log exception

  // Abandon message so it stays in Queue
  msg.Abandon();
}

Notes on Message Queue Use

The traditional communications medium within an application for a long time has been to either just build a monolithic application where the communication is purely internal, or to build things out as a Service Oriented Architecture (SOA). Both of these method rely on synchronous communication between the various components of a system. This can have huge performance implications as each component can’t finish processing a task until it’s dependent components finish and return a result back to it. The use of Message Queues decouples these to be Asynchronous, so that each component can fire off an event or message and then keep moving forward with what it needs to do; all with the system setup to make the handling of that event or message happen reliably behind the scenes.

As a result of the Asynchronous nature of Message Queues, it should be kept in mind that they work best only with events and/or messages that can be processed in an Asynchronous manner. This works well with submitting a PTO request, or a submitting a time sheet for billing, or even queuing up data edits to be processed and sync’d. However, it doesn’t work very well for a client search UI or the generation of website pages, since these are events that need to happen in real time with a user or other system waiting for the results. Basically, the querying of data doesn’t work well to be processed as events or messages in a Queue, but data edits, reports or other types of background tasks do work well.

Microservices Architectures have started becoming more popular in recent years; largely due to the broader adoption of cloud hosting providers such as Microsoft Azure. Using Azure Service Bus Queues with a Microservices Architectures does pair together fairly well, but not all applications will see the full potential benefits of using this type of system architecture.

It must be remembered that Message Queues are just another tool in the toolbox that can help when building large enterprise systems. The specific parts of the system that utilize the Message Queue still need to be fully evaluates before determining if it’s in fact the correct tool for the job; just as is done with all other tools available.

There are many application architecture scenarios that benefit from Message Queues, and the Microsoft Azure Service Bus Queue features are very easy and powerful to implement.

About the Author