Service Fabric would be a core tool in your computing arsenal for Microservices architecture, very popular among Microsoft and Azure enthusiast. I have covered Service Fabric extensively with in many blog posts. In this post, I would like to focus on Operational and Service Management aspect of the Azure Service Fabric.
Problem Definition
- Microservices are highly distributed and likely high volume transaction infrastructure, how to efficiently manage an aspect of service management such as Operational Logging, Application Logging, Security Logging, Log Shipping & Aggregation and Log Analytics & Alerting.
- Tracking Changes and System Updates effectively for vastly distributed logical and physical architecture.
- Setup comprehensive Incident Management or Outage Tracking for distributed applications.
- Covering for compliance around log retention and transactional tractability.
- Segregating logs based on functions and organisational visibility, i.e. finance, legal.
These problems mentioned above are a very common scenario (for Service Fabric or Microservices implementation) at any enterprise scale and business critical Microservices implementation. Clearly, the challenge is the scale. You could be dealing with 100+ logical services and 25-30 machine cluster in typical high-end microservices environments, or even bigger.
Microsoft Operations Management Suite would be an excellent fit if you are in Azure (also if you are not). Let us try to evaluate OMS based on following the business value of IT operations and management criteria.
- Cost Reduction
- Reduce Risk (Security)
- Visibility into IT Systems (through Application, Security and Availability)
- Automation and Orchestration of IT Operations
I would like to take a fresh approach for evaluation. This time I am going to share my thoughts and evaluation as we go along with implementation guide instead of spreadsheets and talks. The best way to evaluate a product is to be hands on, we never buy a car without a test drive, right! The only downside is that you need to read more article to get the complete picture.
Pre-requisite
Following are pre-requisites those you need to follow-up the step-by-step guide for an article.
Setup Service Fabric Cluster
You would need a Service Fabric Cluster to integrate with OMS, in a previous article, we discussed step-by-step guide to setup service fabric cluster in the Azure.
You can also use following templates to setup an Azure Service Fabric Cluster (including IaaSDiagnostics configured).
You can find Resource Manager Template and relevant scripts at JSON Template & PowerShell Script and Github/NilayParikh/AzureTemplates repository.
Setup Microsoft Operations Management Suite Workspace
In a previous article, we discussed step-by-step guide to create Microsoft OMS Workspace. You would need a OMS Workspace to enable Service Fabric Analytics.
Setup Service Fabric Analytics in Microsoft Operations Suite and Connect to Service Fabric Cluster
Setting up Service Fabric Analytics is very simple and straight-forward. You can setup through OMS Portal or Azure Portal.
Find Service Fabric Analytics among OMS Solutions
Search for Service Fabric Analytics solution and click on the card.
Add Service Fabric Analytics
You can add the solution by clicking the Add button. On setup, Service Fabric Analytics won’t search all Nodes or Auto Configure. You would require to connect the Service Fabric Cluster.
Connect Service Fabric Logs to OMS Analytics Solution
You can configure Service Fabric Logs (by connecting Storage Account) either using Azure Portal or PowerShell. Let us first configure the logs using Azure Portal. Find Service Fabric Analytics – Workspace Instance and open the configuration.
On right-hand side, you find Storage accounts log menu option, click on the menu item. And try to add a storage account. The IaaSDiagnostics configure storage accounts for logs. If you are not sure than you can validate in the ARM template. Configure Data Type and Source.
I have configured two sources Service Fabric Events and ETW Logs, respectively they provides logging for Service Fabric Events and ETW Events.
Service Fabric write to following tables (by default),
- WADServiceFabricReliableActorEventTable
- WADServiceFabricReliableServiceEventTable
- WADServiceFabricSystemEventTable
- WADETWEventTable
Log Analytics then reads Service Fabric framework events, including Reliable Service Events, Actor Events, Operational Events, and Custom ETW events.
Verify through Microsoft Operations Management Suite Portal
You can verify connected Azure Storage in OMS Portal. Visit Setting > Connected Source >> Azure Storage, it would show you connected storage accounts.
You can also verify if the service is importing Service Fabric events for you.
For advance use, Assess Azure Service Fabric applications and micro-services with PowerShell .
Summary & Business Value
- Simplified approach to connect and collect logs from every distributed VM instance.
- It can alert and highlight various Service Fabric event, as Service Fabric Analytics is platform aware solution and built for a same purpose.
- Platform Aware Data Query, you can query based on custom fields.
- Easy log retention through Storage Account while keep minimal set into Log Analytics.
- Provide Role Base Access to OMS Portal.
I personally enjoyed seamless Service Management and Operations experience with OMS and Service Fabric.