当前位置:网站首页>Build simple business monitoring Kanban based on Alibaba cloud log service

Build simple business monitoring Kanban based on Alibaba cloud log service

2020-11-08 13:48:53 Catcher8

Preface

Recently, Lao Huang has been making pairs 11 Relevant stuff , So blogs and github Not much updated , During this period, I also made a lot of things in the company .

Let's briefly share the business monitoring related .

Let's start with the background .

A business is in double 11 In the first wave, there was no real-time business kanban , During the summary meeting, technical colleagues were complained by relevant leaders and business personnel , It is said that there is no way to clearly understand the situation at that time , Can't adjust the corresponding strategy in time and effectively .

After that, Lao Huang learned that , That business is an old one , Resources are tight , Don't dare to access the database in real time ( Single point ), I'm afraid the database will hang up , The business is cold .

To avoid an embarrassing situation , I don't want to be criticized again , It can only be done .

Analyze the status quo

3 Applications ,.NET Framework Project , All are windows The server , No containerization .

From double 11 Only a few days , You can't change it too much , And deal with the new needs of the business unit .

The plan that came to mind at that time

  1. Business , Access prometheus, combination grafana
  2. Business development MQ, Consumption data to ES, Make a front panel
  3. Business , Access log service , Combined with the instrument panel

A general analysis

  • programme 1, The business team is right about prometheus Almost 0 cognition , It takes a lot of time to understand the relevant concepts ,pass
  • programme 2,MQ At present, Tencent cloud is used CMQ, I've been holed up 2 Time , Not very good either hold live ES,pass
  • programme 3, Log according to the internal standard , The business side only needs to add a line of corresponding log in the key place , And then leave it to logtail Go collect , Upload to log service

So in these three scenarios , Lao Huang chose the third plan .

First of all, the log service has been connected to various internal systems , I'm familiar with the team , Secondly, it will not affect the main business , Just bury it in key places , Add log .

Although it's intrusive to business code , But no doubt this is an optimal solution at this stage .

The overall implementation logic is as follows .

Business

Business , It's actually a very simple , And the most important step .

We have the corresponding log help class , So what we have to deal with here is just the content of the log .

SerilogHelper.Info($"[field1] [field2] [field3]", "metrics_name");

Lao Huang's agreement here is , The field content is placed in [] Inside , Each field should be separated by a space .

Then the log will be dropped to a specific directory , Waiting to be Logtail The log service has been collected .

Of course, there is a problem here , It's the format of the log file , The previous designation was UTF-8, The resulting file is UTF-8 with bom Format .

This will cause the first line of log not to be parsed correctly , So pay special attention to .

Data access and display

At the code level, the above step is done , The next thing to do is log access .

According to business scenarios , Now it's an application, an indicator , So we need to build three logstores , Select the storage time on demand , The default is permanent .

After the log database is built, you need to access the data source , Here Lao Huang chose Regular - Text log .

And then there's a bunch of regular configurations .

most important of all Logtail To configure This step .

Log path , Is the path of the program log output , such logtail To set the path collection .

Regularization is an important content ,logtail The log will be parsed according to this rule , Extract into fields , This will be more convenient for later query .

Next is the configuration of query analysis .

Here is to specify the statistical fields , Another is to turn off full-text indexing , Because in this scenario , It's not meaningful to open a full-text index , Waste money .

To this step , We can collect the data .

The last thing to do is to query the results . As long as it will be simple sql, Using log service to do statistics is certainly not a problem , The difficulty is relatively low .

Here are the specific effects .

It's a lot of coding , Rough it .

Because the console provides automatic refresh and full screen function , So when the hanging panel comes out, it can save manual intervention .

summary

I was sued Monday night , On Tuesday morning , On Tuesday afternoon , On Wednesday, the results , This wave of operation is really fierce .

Have to say , Alibaba cloud's log service really simplifies a lot of tedious operations .

But the way to extract logs needs to be improved , It's a bit awkward .

版权声明
本文为[Catcher8]所创,转载请带上原文链接,感谢