In the previous post Processing Azure EventGrid events with NServiceBus, I showed how to process custom events emitted to EventGrid using NServiceBus.

In this post, I’ll focus on events emitted by EventGrid for Azure services such as Blob Storage and Resources. Note that there are other Azure resource providers that can raise EventGrid events which could be processed in the same manner.

To make processing simpler, I’ve extracted the logic for message conversion and contracts into its package, NServiceBus.AzureEventGrid.StorageQueues.

What’s EventGrid and why it’s important?

EventGrid helps with creation of event-based systems. The service manages routing of events from various source to the subscribers. Its high throughput and availability make it very appealing. Currently supports a few Azure Services but will change over time as the service is adopted by more and more other Azure services.

So, what makes it so important? Event-based development. No longer polling. Instead, it’s a push-based model where events are pushed to the subscribers. Think of Serverless applications. No longer need to poll for a storage blob or a queue. Instead, when a blob is created, or a message is received, an event is fired. If you have developed using NServiceBus in the past, event-driven development is not a new concept to you, and you’ll see that EventGrid and NServiceBus pub/sub are playing along nicely.

Update 2017-07-09: it appears none of the documentation was updated yet. ASB team is aware and it’s tracked here.

“What you pay is what you get.”
This age-old wisdom applied to cloud services even more than anywhere. Take Azure Service Bus for example. The service offers 3 tiers that provide a different level of service and commitment.

• Basic
• Standard
• Premium

I’m not going to focus on all there here. General information is available on the pricing page. What I am going to focus is on the distinction between the two most dominant tiers: Standard and Premium.

Azure Service Bus pub/sub is implemented using topics and subscriptions. Messages are published to topics and copied over to subscription queues with matching criteria. Criteria are declared using Rules. Each rule has a Filter. Filters help the broker decide if a message sent to a topic will be copied over to a subscription or not. Let’s dive into the world of filters to understand how they work. There are three types of filters supported by the broker:

Azure Service Bus can now integrate with Azure Event Grid. Great news! Currently, it’s only possible with Azure Service Bus premium tier, but soon standard tier will get the ability as well. Also, at the moment of writing this post, there’s no way to directly integrate EventGrid events with Azure Service Bus to post messages whenever there’s an event coming from EventGrid.

Non the less, the ability to connect EventGrid to Azure Functions opens up scenarios such as message processing without long polling by simplifying events consumption. An example would be an Azure Function that would be triggered whenever there’a new message and no listeners are registered. The Function would retrieve the message and will be able to hibernate again. Until the next message shows up. No long polling, no additional cost. The only problem I find with this approach is testing. To test that logic for EventGrid is working, one has to create an Azure Function. For some integration testing, creating a function sounds a little of an overhead. Using services like RequestBin to inspect HTTP requests is not ideal either. This is where Azure Relay (WCF Relay) can help.

Managed Server Identity (MSI)* is a feature of Azure Active Directory (AAD) to allow applications in Azure authenticate to cloud services without managing credentials in your code.

Integration with MSI presents an excellent opportunity to remove credentials from your code and no longer manage those pesky connections string. What’s even better, no more SAS keys to refresh in case those are re-generated. Azure Service Bus documentation has a quick tutorial with a linked sample, which is not as detailed as I’d like it to be. So here’s a more comprehensive walkthrough.

I always found it interesting that most of us start counting earlier than we can read or spell our own names. Almost as if we are predispositioned to count first. Yet numbers can become very difficult later in the game.

Azure Service Bus client seems to follow the same footsteps in its evolution. With the “old school” client (WindowsAzure.ServiceBus) reading message counts was a trivial exercise.

Recently, I’ve received a comment on an older post about message de-duplication that was somewhat peculiar.

An important note here - watch out for partitioning. I basically ran the same code, but de-duping did not work and I couldn’t understand why. Turns out that because I created the queue manually through the portal with partitioning enabled, and it screwed up the de-duping.

To recap, here’s what we have:

• A queue is created with native de-duplication enabled.
• It’s partitioned.
• A message with a calculated MessageId is sent to the queue.
• A message duplicate with the same MessageId is sent to the queue.
• Expected behavior: duplicate to be detected and removed
• Actual behavior: the message and its duplicate are found on the queue.

This makes no sense, right? Indeed it doesn’t.

ServiceBus.AttachmentPlugin is an Azure Service Bus plugin to help with messages exceeding the allowed maximum size. Sender would store the message body as a Azure Storage blob and receive would read blob content to re-hydrate the message body just before message is given to the consumer code. This assumes that both sender and receiver share the knowledge about Azure Storage account and both can access it. But what if that’s not the case?

For a very long time Azure Service Bus client WindowsAzure.ServiceBus/ was a black box. When it came to the customization of the message payload, it would allow choosing on serialization, and that’s pretty much it. Message IDs were always generated as random Guids and were required to be overwritten to comply with project requirements. Needed to secure the payload? You’d need to have that code to operate on the data before it would become BrokeredMessage payload.