我有一些从Powershell导出到CSV的事件日志。如何获取和扩展消息字段，以便可以在Elasticsearch中使用其中的字段？我目前正在使用 Pandas 将数据发送到elasticsearch。
当前的拆分方法如下:
sec_events['action'] = sec_events.join(sec_events['message'].str.split('\\s\\s\\s\\s', 1, expand=True).apply(pd.Series))[0]
sec_events['message'] = sec_events.join(sec_events['message'].str.split('\\s\\s\\s\\s', 1, expand=True).apply(pd.Series))[1]
这输出我的方法，如:
0,mycompname.domm.f.f.a,4688,successaudit,microsoft-windows-security-auditing,3/7/2017 10:38:16 am,3/7/2017 10:38:16 am,NONE,"subject: security id: s-1-5-18 account name: mycompname$ account domain: domm logon id: 0x3e7 process information: new process id: 0x1a54 new process name: c:\windows\system32\ipconfig.exe token elevation type: %%1936 creator process id: 0x1b38 process command line: token elevation type indicates the type of token that was assigned to the new process in accordance with user account control policy. type 1 is a full token with no privileges removed or groups disabled. a full token is only used if user account control is disabled or if the user is the built-in administrator account or a service account. type 2 is an elevated token with no privileges removed or groups disabled. an elevated token is used when user account control is enabled and the user chooses to start the program using run as administrator. an elevated token is also used when an application is configured to always require administrative privilege or to always require maximum privilege, and the user is a member of the administrators group. type 3 is a limited token with administrative privileges removed and administrative groups disabled. the limited token is used when user account control is enabled, the application does not require administrative privilege, and the user does not choose to start the program using run as administrator.",a new process has been created.
1,mycompname.domm.f.f.f,4656,failureaudit,microsoft-windows-security-auditing,3/7/2017 10:38:05 am,3/7/2017 10:38:05 am,NONE,subject: security id: s-1-5-18 account name: mycompname$ account domain: f logon id: 0x3e7 object: object server: security object type: key object name: \registry\machine\system\controlset001\services\policyagent\parameters\cache handle id: 0x0 process information: process id: 0x1b14 process name: c:\windows\system32\reg.exe access request information: transaction id: {00000000-0000-0000-0000-000000000000} accesses: %%4432 %%4435 access reasons: - access mask: 0x9 privileges used for access check: - restricted sid count: 0,a handle to an object was requested.
如何从导出的csv将Windows事件日志的消息字段转换为elasticsearch / json样式格式的字段？
我正在寻找类似以下内容的json格式，以便可以将其纳入logstash:
{message: {
subject: ,
security id: s-1-5-18,
account name: mycompname$,
account domain: domm,
logon id: 0x3e7,
process information: ,
new process id: 0x1a54,
new process name: c:\windows\system32\ipconfig.exe,
token elevation type: %%1936,
creator process id: 0x1b38,
process command line: ,
process command line: ,
string1: token elevation type indicates the type of token that was assigned to the new process in accordance with user account control policy.
type 1 is a full token with no privileges removed or groups disabled.
a full token is only used if user account control is disabled or if the user is the built-in administrator account or a service account.
type 2 is an elevated token with no privileges removed or groups disabled.
an elevated token is used when user account control is enabled and the user chooses to start the program using run as administrator.
an elevated token is also used when an application is configured to always require administrative privilege or to always require maximum privilege, and the user is a member of the administrators group.
type 3 is a limited token with administrative privileges removed and administrative groups disabled.
the limited token is used when user account control is enabled, the application does not require administrative privilege, and the user does not choose to start the program using run as administrator.
action: a new process has been created.
}
}
我已经尝试了以下一些项目:
sec_events['message'] = sec_events.message.replace([':\s+,', '\[', '\]', ':\s+', ',\s+','\s\s\s'],['":"none","', '{"', '"}', '":"', '","','","'], regex=True)
我似乎无法正常工作。任何帮助将是惊人的谢谢。
我注意到以下情况，
3 exact white spaces = transition from field to field
6 exact white spaces = break between the accesses field
2 exact white spaces = transition from field name to data if the data is there.
最佳答案
问题最终是比赛之间的比赛没有拿起逗号。
例:
,Token elevation type: %%1936,
creator process id: 0x1b38,
process command line: C:\windows\system32\cmd.exe,
process command line: ,
匹配只会是
Key Value
Token elevation type %%1936
process command line C:\windows\system32\cmd.exe
关于python - Windows事件日志消息字段传递给elasticsearch，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/43034450/