扫描日志文件数据的最后30分钟

问题描述

我必须写一个shell / perl脚本扫描日志文件最后30分钟有价值的数据。要求在cron来安排此脚本来运行，每30分钟并寻找错误字符串。

I have to write a shell/perl script to scan a log file for last 30 mins worth Data. The requirement is to schedule this script in Cron to run every 30 minutes and look for a error string.

OS: Solaris
Shell:Bash

我曾尝试下面的脚本，但它已成为太长，笨拙，做我们还有其他的方法来使它有点短？

I have tried below script, but it has become too long and clumsy, do we have other way to make it a bit shorter?

blogs=/opt/docs/datapower/prod/business.log
slogs=/opt/docs/datapower/prod/system.log


starttime=$(date +'%H')
currmin=$(date +'%M')
curdate=`date|cut -d' ' -f5`
echo $(date)

if [ $currmin -le 29 ] && [ $starttime -ne 00 ] ; then
starttime1=`echo "$(date +'%H') - 1" | bc`
logtime="$starttime1"
logtime="$logtime:[3-5][0-9]"
echo $logtime



elif [ $currmin -le 29 ] && [ $starttime -eq 00 ] ; then
logtime="23:[3-5][0-9]"
echo $logtime



else
logtime="$starttime"
logtime="$logtime:[0-2][0-9]"
echo $logtime

fi

if ( grep "$logtime" $slogs | egrep "AAA Authentication Failure|AAA Authorization Failure") > dptest 2>&1;then

       Do something

fi

更新：添加例如日志声明。
下面是日志语句的例子：
11月20日6时06分58秒的商业-LOG-STA [DP域分期] [0x80000001] [商务日志] [信息] MPGW（GenServiceMPG）：反式（31513092）[要求]：AAA验证失败/＆GT;

推荐答案

我觉得你这样做是倒退了一点 - 建立一个RE到grep的日期出日志文件。

I think you're doing it a little backwards - building an RE to grep a date out of a log file.

在Perl中处理这个我会寻找读取整个日志文件，tokenise它 - 提取时间戳 - 然后根据邮件内容提示。

Approaching this in perl I'd be looking to read the whole log file, tokenise it - to extract the time stamp - and then alert based on message content.

Perl有第一个部分一个不错的模块 - 时间::片。
它去有点像这样：

Perl has a nice module for the first part - Time::Piece.It goes a bit like this:

use strict;
use warnings;

use Time::Piece;

my $HALF_HOUR = 30 * 60;

while (<DATA>) {
    #extract timestamp via regular expression
    my ( $timestamp, $message ) = (m/\A(\w+\s+\d+\s+\d+:\d+:\d+) (.*)/);

    #convert text timestamp to 'unix time'.
    #need the year in here because your log doesn't include it.
    my $t = localtime();
    $t = $t->strptime( $timestamp . " " . $t->year, "%b %d %H:%M:%S %Y" );


    #skip if parsed time is more than half an hour ago.
    next if ( $t < time() - $HALF_HOUR );
    if (   $message =~ m/AAA Authentication failure/i
        or $message =~ m/AAA Authorization failure/i )
    {
        print "Alert: ( $t )  $message\n";
    }
}

__DATA__
Nov 20 13:46:58 business-log-sta [DP-Domain-STAGING][0x80000001][business-log][info] mpgw(GenServiceMPG): trans(31513092)[request]: AAA Authentication failure/>
Nov 20 13:00:58 business-log-sta [DP-Domain-STAGING][0x80000001][business-log][info] mpgw(GenServiceMPG): trans(31513092)[request]: AAA Authentication failure/>
Nov 20 10:06:58 business-log-sta [DP-Domain-STAGING][0x80000001][business-log][info] mpgw(GenServiceMPG): trans(31513092)[request]: AAA Authentication failure/>

跟帖问：

能否请您解释一下这句话呢，我的（$时间戳，$消息）=（M / \\ A（\\ w + \\ S + \\ D + \\ S + \\ D +：\\ D +：\\ D + ）（*）/）;

"Could you please explain what this statement does, my ( $timestamp, $message ) = (m/\A(\w+\s+\d+\s+\d+:\d+:\d+) (.*)/);"

这做了两件事：

• 一个在Perl的招数，就是你可以捕捉到一个普通的前pression的部分，通过将其放入括号内。因此， \\ A（\\ w + \\ S + \\ D + \\ S + \\ D +：\\ D +：\\ D +） - 将匹配从行的开头：
  
  
  
  • 一个或多个字字。
  
  
  • 一个或更多的'数字'
  
  
  • \\ D +：\\ D +：\\ D + 将捕获的时候。 （任意3冒号分隔数字）。
  
  
  • One of the tricks in Perl, is that you can capture parts of a regular expression, by putting it in brackets. So \A(\w+\s+\d+\s+\d+:\d+:\d+) - will match from the start of line:
    • One or more 'word' characters.
    • One or more 'digits'
    • \d+:\d+:\d+ will capture a time. (Any 3 colon separated numbers).

    的另一部分，当然​​，捕获'的其余部分。

    The other part, of course, captures 'the rest'.

    
    
    • 然后，我们分配的模式匹配返回的数组，到命名变量（数组 $时间戳和 $消息）。
    
    
    • Then, we assign the array returned by the pattern match, into an array of named variables ( $timestamp and $message).

    最终的结果是 - 定的行：

    Net result is - given the line:

     Nov 20  13:46:58       business-log-sta [DP-Domain-STAGING][0x80000001][business-log][info] mpgw(GenServiceMPG): trans(31513092)[request]: AAA Authentication failure/>
    (\w+ \d+ \d+:\d+:\d+)   (.*)
    

    我们的常规前pression返回两个'块'分开，然后我们把它们放入两个变量。

    Our regular expression returns the two 'chunks' separately, and then we put them into the two variables.

    这篇关于扫描日志文件数据的最后30分钟的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！