关键词：
1、安装JDK
rpm -ivh jdk-8u101-linux-x64.rpm Preparing... ########################################### [100%] 1:jdk1.8.0_101 ########################################### [100%] Unpacking JAR files... tools.jar... plugin.jar... javaws.jar... deploy.jar... rt.jar... jsse.jar... charsets.jar... localedata.jar...
检测java版本
java -version java version "1.8.0_101" Java(TM) SE Runtime Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
2、安装redis
yum install -y tcl gcc mkdir /usr/local/redis tar zxvf redis-2.8.20.tar.gz cp -rf redis-2.8.20/* /usr/local/redis/ cd /usr/local/redis make MALLOC=libc make install cd utils/ ./install_server.sh #所有选项默认
查看redis监控端口
netstat -tnlp |grep redis tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 1978/redis-server * tcp 0 0 :::6379 :::* LISTEN 1978/redis-server *
3、安装logstansh
rpm -ivh logstash-2.4.0.noarch.rpm Preparing... ########################################### [100%] 1:logstash ########################################### [100%] echo "PATH=$PATH:/opt/logstash/bin" >> /etc/profile source /etc/profile
测试logstash
logstash -e "input {stdin{}} output {stdout{}}"
hello
Settings: Default pipeline workers: 1
Pipeline main started
2016-09-18T09:10:32.369Z localhost.localdomain hello3.1、测试redis缓存（分两个终端运行b/c两步）
a、新建logstash配置文件
mkdir /opt/logstash/conf
vi output_redis.conf
input { stdin { } } #手动输入数据
output {
stdout { codec => rubydebug } #页面debug信息
redis {
host => ‘127.0.0.1‘
data_type => ‘list‘
key => ‘redis‘
}
}b、查看redis是否缓存数据
cd /usr/local/redis-2.8.20/src/ redis-cli monitor
c、启动logstansh(重启一个终端)
logstash -f output_redis.conf --verbose
hello
starting agent {:level=>:info}
starting pipeline {:id=>"main", :level=>:info}
Settings: Default pipeline workers: 1
Starting pipeline {:id=>"main", :pipeline_workers=>1, :batch_size=>125, :batch_delay=>5, :max_inflight=>125, :level=>:info}
Pipeline main started
{
"message" => "hello",
"@version" => "1",
"@timestamp" => "2016-09-18T09:14:55.288Z",
"host" => "localhost.localdomain"
}d、测试成功
redis-cli monitor
OK
1474190709.219548 [0 127.0.0.1:36399] "rpush" "redis" "{"message":"hello","@version":"1","@timestamp":"2016-09-18T09:25:07.911Z","host":"localhost.localdomain"}"四、安装elasticsearch
1、elasticsearch的安装
rpm -ivh elasticsearch-2.4.0.rpm warning: elasticsearch-2.4.0.rpm: Header V4 RSA/SHA1 Signature, key ID d88e42b4: NOKEY Preparing... ########################################### [100%] Creating elasticsearch group... OK Creating elasticsearch user... OK 1:elasticsearch ########################################### [100%] ### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using chkconfig sudo chkconfig --add elasticsearch ### You can start elasticsearch service by executing sudo service elasticsearch start
2、修改elasticsearch配置文件
vi /etc/elasticsearch/elasticsearch.yml network.host: 172.16.1.224
3、查看elasticsearch是否启动
netstat -tnlp |grep java tcp 0 0 ::ffff:172.16.1.224:9200 :::* LISTEN 1345/java tcp 0 0 ::ffff:172.16.1.224:9300 :::* LISTEN 1345/java
4、测试logstansh和elasticsearch是否能结合使用
a.新建logstansh配置文件elasticsearch.conf
cd /opt/logstash/conf/
vi elasticsearch.conf
input { stdin {} } #手动输入
output {
elasticsearch { hosts => "127.0.0.1" }
stdout { codec=> rubydebug } #页面debug信息
}b.启动elasticsearch.conf配置文件
logstash -f elasticsearch.conf --verbose
hello
starting agent {:level=>:info}
starting pipeline {:id=>"main", :level=>:info}
Settings: Default pipeline workers: 1
Using mapping template from {:path=>nil, :level=>:info}
Attempting to install template {:manage_template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"string", "index"=>"not_analyzed"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"float"}, "longitude"=>{"type"=>"float"}}}}}}}, :level=>:info}
New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["127.0.0.1"], :level=>:info}
Starting pipeline {:id=>"main", :pipeline_workers=>1, :batch_size=>125, :batch_delay=>5, :max_inflight=>125, :level=>:info}
Pipeline main started
{
"message" => "hello",
"@version" => "1",
"@timestamp" => "2016-09-18T09:41:44.603Z",
"host" => "localhost.localdomain"
}c.查看elasticsearch是否获取到了"hello elasticsearch"
curl http://localhost:9200/_search?pretty
{
"took" : 41,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2016.09.18",
"_type" : "logs",
"_id" : "AVc8rFYwCkn6K6s_ltue",
"_score" : 1.0,
"_source" : {
"message" : "hello",
"@version" : "1",
"@timestamp" : "2016-09-18T09:41:44.603Z",
"host" : "localhost.localdomain"
}
} ]
}
}4、安装elasticsearch插件
elasticsearch有很多插件：http://www.searchtech.pro/elasticsearch-plugins
elasticsearch-head插件安装
./plugin install mobz/elasticsearch-head -> Installing mobz/elasticsearch-head... Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ... Downloading .........DONE Verifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ... NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify) Installed head into /usr/share/elasticsearch/plugins/head
5、查看elasticsearch-head插件显示的页面
http://172.16.1.224:9200/_plugin/head/
五、kibana安装
1、安装kibana
rpm -ivh kibana-4.6.1-x86_64.rpm Preparing... ########################################### [100%] 1:kibana ########################################### [100%]
修改kibana配置文件，把下面这行改成elasticsearc的访问路径
vi /opt/kibana/config/kibana.yml elasticsearch.url: "http://172.16.1.224:9200"
2 启动kibana
/opt/kibana/bin/kibana& [2] 1441 [[email protected] elk]# log [18:06:27.275] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.324] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [18:06:27.387] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.400] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.407] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.412] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.420] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.425] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready log [18:06:27.432] [info][listening] Server running at http://0.0.0.0:5601 log [18:06:32.411] [info][status][plugin:[email protected]] Status changed from yellow to yellow - No existing Kibana index found log [18:06:35.448] [info][status][plugin:[email protected]] Status changed from yellow to green - Kibana index ready
3、测试kinaba
访问页面:http://172.16.1.224:5601/
elk学习笔记之kibana安装
Kibana安装： 安装地址：https://www.elastic.co/downloads/kibana安装：tar-zxvfkibana-5.6.1-linux-x86_64.tar.gz非常不开心的是，安装报错：只能重新下载安装5.5.2版本：https://www.elastic.co/downloads/past-releases/kibana-5-5-2安装运行Kibana 查看详情
elk学习笔记之logstash安装
Logstash安装： https://www.elastic.co/downloads/logstash下载解压：tar–zxvflogstash-5.6.1.tar.gz在/usr/local/logstash-5.6.1/bin下编辑conf：(因为配置了输出到es和console上，所以必须先启动es。)vilogstash-simple.confinput{stdin{}}ou 查看详情
elk学习笔记之elasticsearchhead插件安装
elasticsearchhead插件安装： 准备工作：安装nodejs和npm https://nodejs.org/en/download/ node-v6.11.2-linux-x64.tar.xz由于是xz压缩文件，所以要先安装yum-yinstallxz$xz-d***.tar.xz$tar-xvf ***.tar配置环境变量#setn 查看详情
elk配置笔记
filebeat安装配置1下载,安装wgethttps://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.4.3-linux-x86_64.tar.gzmvfilebeat-8.4.3-linux-x86_64.tar.gz/optcd/opttar-zxvffilebeat-8.4.3-linux-x86_64.tar.gzcd 查看详情
elk学习笔记安装elasticsearchkibanalogstash和x-pack
最近在学习ELK的时候踩了不少的坑，特此写个笔记记录下学习过程。日志主要包括系统日志、应用程序日志和安全日志。系统运维和开发人员可以通过日志了解服务器软硬件信息、检查配置过程中的错误及错误发生的原因。经常... 查看详情
elk笔记10--filebeat使用
...1filebeat介绍2filebeat使用案例2.1软件安装2.2采集数据到kafka2.3采集数据到es3使用技巧3.1filebeat将日志按照类别发送到不同kafkatopic3.2filebeat将日志按照类别发送到不... 查看详情
elk日志平台---老男孩教育笔记
...存储 2、日志收集（日志收集于某一处）3、日志展示安装过 查看详情
elk介绍及搭建elasticsearch分布式集群(代码片段)
...Elasticsearch分布式集群笔记日期：2018-03-0227.1ELK介绍27.2ELK安装准备工作27.3安装es27.4配置es27.5curl查看es集群情况ELK介绍需求背景：业务发展越来越庞大，服务器越来越多各种访问日志、应用日志、错误日志量越来越多，导致运维人... 查看详情
elk笔记16--聚合分析
elk笔记16--聚合分析1聚合概念2聚合类型2.1Bucketing2.1.1TermsAggregation2.2Metric2.2.1AvgAggregation2.3Pipeline2.3.1AvgBucketAggregation2.3.2MaxBucketAggregation2.3.3MinBucke... 查看详情
elk笔记8--index
elk笔记8--index1.index创建的几种方式1.1直接创建index1.2按照当前日期创建索引1.3创建带有rollover功能的索引2.索引的常见设置2.1基本设置2.2index为unassigned的常见处理方式... 查看详情
elk笔记13--queries-fulltextqueries
elk笔记13--Queries-fulltextqueries1fulltext查询简介2fulltext查询类型2.1intervalsquery2.2matchquery2.3match_bool_prefixquery2.4match_phrasequery2.5match_phrase_prefixquery2.6multi_m 查看详情
elk笔记9--跨集群搜索
elk笔记9--跨集群搜索1.跨集群搜索简介2.跨集群搜索配置3跨集群使用案例4说明1.跨集群搜索简介跨集群允许我们在一个或者多个远程集群上执行搜索任务，通常我们可以用跨集群搜索来过滤或... 查看详情
elk笔记22.2--通过api快速创建索引
elk笔记22.2--通过api快速创建索引1简介2功能实现2.1源码2.2测试3注意事项说明1简介本文基于elk笔记22–通过api快速创建索引继续通过api快速创建索引。本节将追加一个... 查看详情
elk笔记11--快照的使用
elk笔记11--快照的使用1快照介绍2快照使用2.1nfs作为存储仓库2.2hdfs作为存储仓库3使用技巧4说明1快照介绍快照是运行中es集群的一个备份，进行快照时候既可以全集群所有索引... 查看详情
elk学习笔记a
一、基本操作1、命令行运行 bin/logstash-e ‘input{stdin{}}output{stdout{codec=>rubydebug}}‘#bin/logstash -e ‘input{stdin{}}output{stdout{codec=>rubydebug}}‘Logstash startup compl 查看详情
elk学习笔记b
一、输入a、标准输入:input { stdin { add_field => {"key" =>"value"} 查看详情
elk笔记13--queries-term-levelqueries
elk笔记13--Queries-term-levelqueries1term-level查询简介2term-level查询类型2.1existsquery2.2fuzzyquery2.3idsquery2.4prefixquery2.5rangequery2.6regexpquery2.7termquery2.8term 查看详情
elk学习笔记之kibana入门使用
Kibana入门使用： 第一次导入索引：修改展示时间，不然查不到数据：点Discover，查阅数据:如果要添加新的index：点击Visualize,创建chart：点击Dashboard,布局：DashBoard完工啦！！！嘿嘿嘿 查看详情