elk-logstash-kibana

zhaojingyu zhaojingyu     2022-12-25     449

关键词:

一、修改logstash.yml

 unzip logstash-6.3.2

vim config/logstash.yml

#添加:检查所有ip

http.host: "0.0.0.0"

二、在logstash下的config下创建(根据自己log日志地址设置:安装nginx等)

[[email protected] config]$ cat syslog.conf 
input 
  syslog 
    type => "system-syslog"
    port => 10514
  

output 
  elasticsearch 
    hosts => ["139.199.124.136:9200"]  # 定义es服务器的ip
    index => "system-syslog-%+YYYY.MM.dd" # 定义索引
  

input 
  file   # 指定一个文件作为输入源
    path => "/home/elk/nginx/logs/elk_access.log"  # 指定文件的路径
    start_position => "beginning"  # 指定何时开始收集
    type => "nginx"  # 定义日志类型,可自定义
  

filter   # 配置过滤器
    grok 
        match =>  "message" => "%IPORHOST:http_host %IPORHOST:clientip - %USERNAME:remote_user [%HTTPDATE:timestamp] "(?:%WORD:http_verb %NOTSPACE:http_request(?: HTTP/%NUMBER:http_version)?|%DATA:raw_http_request)" %NUMBER:response (?:%NUMBER:bytes_read|-) %QS:referrer %QS:agent %QS:xforwardedfor %NUMBER:request_time:float"  # 定义日志的输出格式
    
    geoip 
        source => "clientip"
    

output 
    stdout  codec => rubydebug 
    elasticsearch 
        hosts => ["139.199.124.136:9200"]
        index => "nginx-test-%+YYYY.MM.dd"
  

nginx日志编译二进制:

nginx添加:

    log_format  main  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
                      ‘$status $body_bytes_sent "$http_referer" ‘
                      ‘"$http_user_agent" "$http_x_forwarded_for"‘;

    access_log  logs/access.log  main;

三、vim   kibana.yml

tar xf kibana-6.3.2-linux-x86_64.tar.gz

#vim kibana-6.3.2-linux-x86_64/config/kibana.yml
elasticsearch.url: "http://139.199.124.136:9200"    #ela地址
server.host: 0.0.0.0
server.port: 5601
logging.dest: /elk/logs/kibana.log  #日志存放目录,自定义 

四、配置kibana服务器的ip以及配置的监听端口:

vim /etc/rsyslog.conf

*.* @@192.168.77.130:10514

重启rsyslog

systemctl restart rsyslog

-----------启动服务查看

[[email protected] ~]# netstat -lntup|egrep ‘9600|5601‘

tcp        0      0 0.0.0.0:9600                0.0.0.0:*                   LISTEN      5076/java           
tcp        0      0 0.0.0.0:5601                0.0.0.0:*                   LISTEN      5073/node 

技术分享图片

五、启动服务汇总

#cat /etc/rc.local
/home/elk/nginx/sbin/nginx
su -c ‘/home/elk/elasticsearch-6.3.2/bin/elasticsearch -d >>/dev/null 2>&1 &‘ elk
#su -c ‘cd /home/elk/elasticsearch-head-master;grunt server >>/dev/null 2>&1 &‘ elk
su -c ‘/home/elk/kibana-6.3.2-linux-x86_64//bin/kibana  >>/dev/null 2>&1 &‘ elk
su -c ‘/home/elk/logstash-6.3.2/bin/logstash -f /home/elk/logstash-6.3.2/config/syslog.conf  >>/dev/null 2>&1 &‘ elk
#su -c ‘/home/elk/filebeat-6.3.2-linux-x86_64/filebeat &‘ elk