在分布式系统中,应用数量众多,应用调用链复杂,常常使用ELK作为日志收集、分析和展示的组件。本篇文章将讲讲解如何部署ELK,然后讲解如何
使用Filebeat采集Spring Boot的日志输出到Logstash上,logstash再将日志输出到Elasticsearch上,最后展示到kibana上面。整个日志采集流程如下图:

在传统的日志采集只会用ELK,那么为什么需要使用filebeat呢,因为
logstash是java应用,解析日志是非的消耗cpu和内存,logstash安装在应用部署的机器上显得非常的影响应用的性能。最常见的做法是用filebeat部署在应用的机器上,logstash单独部署,然后由
filebeat将日志输出给logstash解析,解析完由logstash再传给elasticsearch。

安装计划
本文主要讲解如何部署ElasticSearch 集群,部署的ElasticSearch的版本为7.2,计划用三台机器组成一个ElasticSearch集群,从而组成高可用,机器分配如下:

| 节点 | 规则 | 数量 |
| -------- | -----: | :----: |
| 192.168.1.1 | 2核4G | 1 |
| 192.168.1.2 | 2核4G | 1 |
| 192.168.1.3 | 2核4G | 1 |

安装
下载安装执行以下命令:

1 # 下载elasticsearch-7.2.0-x86_64的rpm包
2 wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.0-x86_64.rpm
3 wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.2.0-x86_64.rpm.sha512
4 # shasum 检查版本信息
5 shasum -a 512 -c elasticsearch-7.2.0-x86_64.rpm.sha512
6 # rpm本地安装
7 sudo rpm --install elasticsearch-7.2.0-x86_64.rpm

安装成功ElasticSearch成功后,执行一下命令启动elasticSearch,并设置为开启自启动:

1 sudo systemctl daemon-reload
2 sudo systemctl enable elasticsearch.service
3 sudo systemctl start elasticsearch.service

elasticSearch的默认端口为9200,启动成功后,执行以下命令:

1 curl -X GET "localhost:9200/"

如果返回以下的信息,则证明安装成功:

 1 {
 2   "name" : "VM_0_5_centos",
 3   "cluster_name" : "elasticsearch",
 4   "cluster_uuid" : "gst98AuET6a648YmAkXyMw",
 5   "version" : {
 6     "number" : "7.2.0",
 7     "build_flavor" : "default",
 8     "build_type" : "rpm",
 9     "build_hash" : "508c38a",
10     "build_date" : "2019-06-20T15:54:18.811730Z",
11     "build_snapshot" : false,
12     "lucene_version" : "8.0.0",
13     "minimum_wire_compatibility_version" : "6.8.0",
14     "minimum_index_compatibility_version" : "6.0.0-beta1"
15   },
16   "tagline" : "You Know, for Search"
17 }

查看节点的健康状态,执行命令 curl localhost:9200/_cluster/health ,如果返回以下信息,则Elasticsearch则为监控状态

 1 {
 2   "cluster_name" : "elasticsearch",
 3   "status" : "green",
 4   "timed_out" : false,
 5   "number_of_nodes" : 1,
 6   "number_of_data_nodes" : 1,
 7   "active_primary_shards" : 0,
 8   "active_shards" : 0,
 9   "relocating_shards" : 0,
10   "initializing_shards" : 0,
11   "unassigned_shards" : 0,
12   "delayed_unassigned_shards" : 0,
13   "number_of_pending_tasks" : 0,
14   "number_of_in_flight_fetch" : 0,
15   "task_max_waiting_in_queue_millis" : 0,
16   "active_shards_percent_as_number" : 100.0
17 }

可以执行以下的命令,查看es的 journal:

1 sudo journalctl --unit elasticsearch

 

配置

以下的路径的配置文件可以配置es的java_home,es_config_home :

1 /etc/sysconfig/elasticsearch

es本身的一些配置在以下的路径,在这里可以配置elasticsearch的堆内存,数据保留天数等信息:

1 /etc/elasticsearch

所有的配置文件描述和路径如下表所示:

<div class="table-box"><table>
<thead>
<tr>
<th>配置类型</th>
<th align="right">描述</th>
<th align="center">路径</th>
</tr>
</thead>
<tbody>
<tr>
<td>home</td>
<td align="right">elasticsearch的home目录</td>
<td align="center">/usr/share/elasticsearch</td>
</tr>
<tr>
<td>bin</td>
<td align="right">elasticsearch的bin目录</td>
<td align="center">/usr/share/elasticsearch/bin</td>
</tr>
<tr>
<td>conf</td>
<td align="right">elasticsearch的配置文件</td>
<td align="center">/etc/elasticsearch</td>
</tr>
<tr>
<td>conf</td>
<td align="right">elasticsearch的环境变量配置</td>
<td align="center">/etc/sysconfig/elasticsearch</td>
</tr>
<tr>
<td>data</td>
<td align="right">elasticsearch的数据目录</td>
<td align="center">/var/lib/elasticsearch</td>
</tr>
<tr>
<td>logs</td>
<td align="right">elasticsearch的日志目录</td>
<td align="center">/var/log/elasticsearch</td>
</tr>
<tr>
<td>plugins</td>
<td align="right">elasticsearch的插件目录</td>
<td align="center">/usr/share/elasticsearch/plugins</td>
</tr>
</tbody>
</table></div>

12-27 16:08