Elastic Stack 6 : Install Filebeat2018/02/12 |
Install Filebeat that easily ships log file data to Elasticsearch or Logstash.
|
|
[1] | Install Filebeat. Configure Elasticsearch repository before it. |
[root@dlp ~]# yum -y install filebeat
|
[2] | Configure basic settings and start Filebeat. |
[root@dlp ~]#
vi /etc/filebeat/filebeat.yml # line 21: set items to collect data # if collect logs, turn to "enabled: true" # by default, /var/log/*.log files are shiped
- type: log
# Change to true to enable this prospector configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/*.log
#- c:\programdata\elasticsearch\logs\*
# Exclude lines. A list of regular expressions to match. It drops the lines thh
at are
# matching any regular expression from the list.
#exclude_lines: ['^DBG']
# Include lines. A list of regular expressions to match. It exports the lines
that are
# matching any regular expression from the list.
#include_lines: ['^ERR', '^WARN']
# Exclude files. A list of regular expressions to match. Filebeat drops the fii
les that
# are matching any regular expression from the list. By default, no files are
dropped.
#exclude_files: ['.gz$']
.....
.....
# line 117: if use Kibana, uncomment and specify output host # if SSL is enabled on Kibana, hostname should be the same with the hostname in certs
setup.kibana:
.....
host: "https://dlp.srv.world:5601"
# line 143: specify output host # the default is localhost's Elasticsearch # if output to Logstash, comment out Elasticsearch and uncomment logstash lines output.elasticsearch: # Array of hosts to connect to.l hosts: ["localhost:9200"] ..... ..... #output.logstash: # The Logstash hosts #hosts: ["localhost:5044"]
[root@dlp ~]#
vi /etc/filebeat/filebeat.reference.yml # line 15: set items to collect data - module: system # Syslog syslog: enabled: true ..... ..... # Authorization logs auth: enabled: true # line 1236: if use Kibana, uncomment and specify output host # if SSL is enabled on Kibana, uncomment ssl related lines # if your certs is self-siged one, ssl.verification_mode should be "none" setup.kibana: # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 host: "dlp.srv.world:5601" # Optional protocol and basic auth credentials. protocol: "https" #username: "elastic" #password: "changeme" # Optional HTTP Path #path: "" # Use SSL settings for HTTPS. Default is true. ssl.enabled: true # Configure SSL verification mode. If `none` is configured, all server hosts # and certificates will be accepted. In this mode, SSL based connections are # susceptible to man-in-the-middle attacks. Use only for testing. Default is # `full`. ssl.verification_mode: none[root@dlp ~]# systemctl start filebeat [root@dlp ~]# systemctl enable filebeat
|
[3] | Make sure the data has been collected normally. |
# index list [root@dlp ~]# curl localhost:9200/_cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open filebeat-6.2.1-2018.02.14 uzqg8... 3 1 29867 0 4.1mb 4.1mb yellow open packetbeat-6.2.1-2018.02.14 -O8vG... 3 1 1681 0 661.4kb 661.4kb yellow open sshd_fail-2018.02 ghhQe... 5 1 31 0 249.2kb 249.2kb yellow open test_index CIPjY... 5 1 1 0 6kb 6kb green open .kibana sV0Ds... 1 0 231 2 308kb 308kb yellow open metricbeat-6.2.1-2018.02.14 OhrZT... 1 1 25201 0 4.5mb 4.5mb # document list on the index [root@dlp ~]# curl localhost:9200/filebeat-6.2.1-2018.02.14/_search?pretty { "took" : 15, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 29867, "max_score" : 1.0, "hits" : [ { "_index" : "filebeat-6.2.1-2018.02.14", "_type" : "doc", ..... ..... |
[4] | If Kibana is running, it's possible to import data. |
[root@dlp ~]# filebeat setup --dashboards Loaded dashboards |
Sponsored Link |
|