ELK 設置定時清理腳本清理索引
1028
2025-04-03
1.ES 的Java API兩種方式
elasticsearch 的API 分為 REST Client API(http請求形式)以及 transportClient
API兩種。相比來說transportClient API效率更高,transportClient
是通過elasticsearch內部RPC的形式進行請求的,連接可以是一個長連接,相當于是把客戶端的請求當成
Elasticsearch 集群的一個節點,當然 REST Client API 也支持http
keepAlive形式的長連接,只是非內部RPC形式。但是從Elasticsearch 7 后就會移除transportClient
。主要原因是transportClient 難以向下兼容版本。
1.1 9300[TCP]
利用9300端口的是spring-data-elasticsearch:transport-api.jar,但是這種方式因為對應的SpringBoot版本不一致,造成對應的transport-api.jar也不同,不能適配es的版本,而且ElasticSearch7.x中已經不推薦使用了,ElasticSearch 8之后更是廢棄了,所以我們不做過多的介紹
1.2 9200[HTTP]
基于9200端口的方式也有多種
JsetClient:非官方,更新緩慢
RestTemplate:模擬發送Http請求,ES很多的操作需要我們自己來封裝,效率低
HttpClient:和上面的情況一樣
ElasticSearch-Rest-Client:官方的RestClient,封裝了ES的操作,API層次分明,易于上手。
JavaAPIClient 7.15版本后推薦
2.ElasticSearch-Rest-Client整合
2.1 創建檢索的服務
我們在商城服務中創建一個檢索的SpringBoot服務
添加對應的依賴:官方地址:https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-getting-started-maven.html#java-rest-high-getting-started-maven-maven
公共依賴不要忘了,同時我們在公共依賴中依賴了MyBatisPlus所以我們需要在search服務中排除數據源,不然啟動報錯
然后我們需要把這個服務注冊到Nacos注冊中心中,這塊操作了很多遍,不重復
添加對應的ElasticSearch的配置類
/** * ElasticSearch的配置類 */ @Configuration public class MallElasticSearchConfiguration { @Bean public RestHighLevelClient restHighLevelClient(){ RestClientBuilder builder = RestClient.builder(new HttpHost("192.168.56.100", 9200, "http")); RestHighLevelClient client = new RestHighLevelClient(builder); return client; } }
1
2
3
4
5
6
7
8
9
10
11
12
13
測試:
2.2 測試保存文檔
設置RequestOptions
我們就在ElasticSearch的配置文件中設置
保存數據
然后就可以結合官方文檔來實現文檔數據的存儲
package com.msb.mall.mallsearch; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.util.JSONPObject; import com.msb.mall.mallsearch.config.MallElasticSearchConfiguration; import lombok.Data; import org.elasticsearch.action.index.IndexRequest; import org.elasticsearch.action.index.IndexResponse; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.common.xcontent.XContentType; import org.json.JSONObject; import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; @SpringBootTest class MallSearchApplicationTests { @Autowired private RestHighLevelClient client; @Test void contextLoads() { System.out.println("--->"+client); } /** * 測試保存文檔 */ @Test void saveIndex() throws Exception { IndexRequest indexRequest = new IndexRequest("system"); indexRequest.id("1"); // indexRequest.source("name","bobokaoya","age",18,"gender","男"); User user = new User(); user.setName("bobo"); user.setAge(22); user.setGender("男"); // 用Jackson中的對象轉json數據 ObjectMapper objectMapper = new ObjectMapper(); String json = objectMapper.writeValueAsString(user); indexRequest.source(json, XContentType.JSON); // 執行操作 IndexResponse index = client.index(indexRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 提取有用的返回信息 System.out.println(index); } @Data class User{ private String name; private Integer age; private String gender; } }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
之后成功
2.3 檢索操作
參考官方文檔可以獲取到處理各種檢索情況的API
案例1:檢索出所有的bank索引的所有文檔
@Test void searchIndexAll() throws IOException { // 1.創建一個 SearchRequest 對象 SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("bank"); // 設置我們要檢索的數據對應的索引庫 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); /*sourceBuilder.query(); sourceBuilder.from(); sourceBuilder.size(); sourceBuilder.aggregation();*/ searchRequest.source(sourceBuilder); // 2.如何執行檢索操作 SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 3.獲取檢索后的響應對象,我們需要解析出我們關心的數據 System.out.println("ElasticSearch檢索的信息:"+response); }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
案例2:根據address全文檢索
@Test void searchIndexByAddress() throws IOException { // 1.創建一個 SearchRequest 對象 SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("bank"); // 設置我們要檢索的數據對應的索引庫 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // 查詢出bank下 address 中包含 mill的記錄 sourceBuilder.query(QueryBuilders.matchQuery("address","mill")); searchRequest.source(sourceBuilder); // System.out.println(searchRequest); // 2.如何執行檢索操作 SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 3.獲取檢索后的響應對象,我們需要解析出我們關心的數據 System.out.println("ElasticSearch檢索的信息:"+response); }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
案例3:嵌套的聚合操作:檢索出bank下的年齡分布和每個年齡段的平均薪資
/** * 聚合:嵌套聚合 * @throws IOException */ @Test void searchIndexAggregation() throws IOException { // 1.創建一個 SearchRequest 對象 SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("bank"); // 設置我們要檢索的數據對應的索引庫 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // 查詢出bank下 所有的文檔 sourceBuilder.query(QueryBuilders.matchAllQuery()); // 聚合 aggregation // 聚合bank下年齡的分布和每個年齡段的平均薪資 AggregationBuilder aggregationBuiler = AggregationBuilders.terms("ageAgg") .field("age") .size(10); // 嵌套聚合 aggregationBuiler.subAggregation(AggregationBuilders.avg("balanceAvg").field("balance")); sourceBuilder.aggregation(aggregationBuiler); sourceBuilder.size(0); // 聚合的時候就不用顯示滿足條件的文檔內容了 searchRequest.source(sourceBuilder); System.out.println(sourceBuilder); // 2.如何執行檢索操作 SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 3.獲取檢索后的響應對象,我們需要解析出我們關心的數據 System.out.println(response); }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
案例4:并行的聚合操作:查詢出bank下年齡段的分布和總的平均薪資
/** * 聚合 * @throws IOException */ @Test void searchIndexAggregation1() throws IOException { // 1.創建一個 SearchRequest 對象 SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("bank"); // 設置我們要檢索的數據對應的索引庫 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // 查詢出bank下 所有的文檔 sourceBuilder.query(QueryBuilders.matchAllQuery()); // 聚合 aggregation // 聚合bank下年齡的分布和平均薪資 AggregationBuilder aggregationBuiler = AggregationBuilders.terms("ageAgg") .field("age") .size(10); sourceBuilder.aggregation(aggregationBuiler); // 聚合平均年齡 AvgAggregationBuilder balanceAggBuilder = AggregationBuilders.avg("balanceAgg").field("age"); sourceBuilder.aggregation(balanceAggBuilder); sourceBuilder.size(0); // 聚合的時候就不用顯示滿足條件的文檔內容了 searchRequest.source(sourceBuilder); System.out.println(sourceBuilder); // 2.如何執行檢索操作 SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 3.獲取檢索后的響應對象,我們需要解析出我們關心的數據 System.out.println(response); }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
案例5:處理檢索后的結果
@Test void searchIndexResponse() throws IOException { // 1.創建一個 SearchRequest 對象 SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("bank"); // 設置我們要檢索的數據對應的索引庫 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); // 查詢出bank下 address 中包含 mill的記錄 sourceBuilder.query(QueryBuilders.matchQuery("address","mill")); searchRequest.source(sourceBuilder); // System.out.println(searchRequest); // 2.如何執行檢索操作 SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS); // 3.獲取檢索后的響應對象,我們需要解析出我們關心的數據 // System.out.println("ElasticSearch檢索的信息:"+response); RestStatus status = response.status(); TimeValue took = response.getTook(); SearchHits hits = response.getHits(); TotalHits totalHits = hits.getTotalHits(); TotalHits.Relation relation = totalHits.relation; long value = totalHits.value; float maxScore = hits.getMaxScore(); // 相關性的最高分 SearchHit[] hits1 = hits.getHits(); for (SearchHit documentFields : hits1) { /*"_index" : "bank", "_type" : "account", "_id" : "970", "_score" : 5.4032025*/ //documentFields.getIndex(),documentFields.getType(),documentFields.getId(),documentFields.getScore(); String json = documentFields.getSourceAsString(); //System.out.println(json); // JSON字符串轉換為 Object對象 ObjectMapper mapper = new ObjectMapper(); Account account = mapper.readValue(json, Account.class); System.out.println("account = " + account); } //System.out.println(relation.toString()+"--->" + value + "--->" + status); } @ToString @Data static class Account { private int account_number; private int balance; private String firstname; private String lastname; private int age; private String gender; private String address; private String employer; private String email; private String city; private String state; }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
數據的結果:
Elasticsearch Spring Boot
版權聲明:本文內容由網絡用戶投稿,版權歸原作者所有,本站不擁有其著作權,亦不承擔相應法律責任。如果您發現本站中有涉嫌抄襲或描述失實的內容,請聯系我們jiasou666@gmail.com 處理,核實后本網站將在24小時內刪除侵權內容。
版權聲明:本文內容由網絡用戶投稿,版權歸原作者所有,本站不擁有其著作權,亦不承擔相應法律責任。如果您發現本站中有涉嫌抄襲或描述失實的內容,請聯系我們jiasou666@gmail.com 處理,核實后本網站將在24小時內刪除侵權內容。