当前位置:网站首页>With all due respect, I just knew that elasticsearch condition update works like this

With all due respect, I just knew that elasticsearch condition update works like this

2020-11-06 01:29:47 Yin Jihuan

background

ElasticSearch It's becoming more and more popular , Many companies are using it . There are those who do log search , There are goods search , There are orders to search for .

Most of the usage scenarios are used to import data to ElasticSearch in , Or by CDC To build the index . In this case , Update data is a single update , such as ID=1 The data of has been modified , Then it will ElasticSearch in ID=1 Update this data of .

But in some scenarios, it is necessary to update multiple data simultaneously according to the conditions , It's like Mysql We use Update Table Set Name=XXX where Age=18 To update a batch of data .

It happened that a classmate asked me how to update in batch by wechat , Next, let's look at ElasticSearch How to update according to the condition in .

Single update

ElasticSearch The client is officially recommended to use elasticsearch-rest-high-level-client. So this article is also based on elasticsearch-rest-high-level-client To build code .

First, let's review how to update a single piece of data , The code is as follows :

  
  1. UpdateRequest updateRequest = new UpdateRequest(index, type, id);
  2. updateRequest.doc(documentJson, XContentType.JSON);
  3. restHighLevelClient.update(updateRequest, options);

structure UpdateRequest The index is specified , type ,ID Three fields , It's accurate to a certain piece of data , So the updated data is also this data .

Condition update

First of all, we prepare a few test data , as follows :

  
  1. {
  2. id: 1,
  3. title: "Java How to learn ",
  4. type: 1,
  5. userId: 1,
  6. tags: [
  7. "java"
  8. ],
  9. textContent: " I have to learn Java",
  10. status: 1,
  11. heat: 100
  12. }
  13. {
  14. id: 2,
  15. title: "Java How to learn ",
  16. type: 1,
  17. userId: 1,
  18. tags: [
  19. "java"
  20. ],
  21. textContent: " I have to learn Java",
  22. status: 1,
  23. heat: 100
  24. }

If our demand is to userId=1 Change all document data to invalid , That is to say status=0. If you don't have to update conditionally , You have to find out userId=1 All data for , And then update one by one , It's too slow .

Let's take a look at how conditional updates are used , as follows :

  
  1. POST http://47.105.66.210:9200/article_v1/doc/_update_by_query
  2. {
  3. "script": {
  4. "source":"ctx._source['status']=0;"
  5. },
  6. "query": {
  7. "term": {
  8. "userId": 1
  9. }
  10. }
  11. }

Conditional updates require the use of _update_by_query To carry out ,query Used to specify matching criteria for updating data ,script Logic for updating .

Use documentation in detail :

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-using.html

stay Java How to implement conditional update in code ?

  
  1. UpdateByQueryRequest request = new UpdateByQueryRequest("article_v1");
  2. request.setQuery(new TermQueryBuilder("userId", 1));
  3. request.setScript(new Script("ctx._source['status']=0;"));
  4. restHighLevelClient.updateByQuery(request, RequestOptions.DEFAULT);

Is it also very simple , It's almost the same as a single data update , Use UpdateByQueryRequest Build update objects , Then set the Query and Script That's all right. .

Conditional update array

For example, our requirement is to remove tags Medium java, as follows :

  
  1. POST http://47.105.66.210:9200/article_v1/doc/_update_by_query
  2. {
  3. "script": {
  4. "source":"ctx._source['tags'].removeIf(item -> item == 'java');"
  5. },
  6. "query": {
  7. "term": {
  8. "userId": 1
  9. }
  10. }
  11. }

In addition, you only need to add removeIf Change to add That's all right. .

  
  1. ctx._source['tags'].add('java');

If there is a special business logic ,Script You can also write a judgment to determine whether it needs to be modified .

  
  1. POST http://47.105.66.210:9200/article_v1/doc/_update_by_query
  2. {
  3. "script": {
  4. "source":"if(ctx._source.type == 11) {ctx._source['tags'].add('java');}"
  5. },
  6. "query": {
  7. "term": {
  8. "userId": 1
  9. }
  10. }
  11. }

Encapsulate general condition updates

The update in most scenarios is relatively simple , Update a value according to a field , Or to update multiple values . stay Java If you go to every place in the script , I repeat. , It's better to take a more general method to update .

Here's a simple list , There are many other points to consider , Like data types, I only deal with numbers , character string , and List, Others need to expand themselves .

  
  1. public BulkByScrollResponse updateByQuery(String index, QueryBuilder query, Map<String, Object> document) {
  2. UpdateByQueryRequest updateByQueryRequest = new UpdateByQueryRequest(index);
  3. updateByQueryRequest.setQuery(query);
  4. StringBuilder script = new StringBuilder();
  5. Set<String> keys = document.keySet();
  6. for (String key : keys) {
  7. String appendValue = "";
  8. Object value = document.get(key);
  9. if (value instanceof Number) {
  10. appendValue = value.toString();
  11. } else if (value instanceof String) {
  12. appendValue = "'" + value.toString() + "'";
  13. } else if (value instanceof List){
  14. appendValue = JsonUtils.toJson(value);
  15. } else {
  16. appendValue = value.toString();
  17. }
  18. script.append("ctx._source.").append(key).append("=").append(appendValue).append(";");
  19. }
  20. updateByQueryRequest.setScript(new Script(script.toString()));
  21. return updateByQuery(updateByQueryRequest, RequestOptions.DEFAULT);
  22. }
  23. public BulkByScrollResponse updateByQuery(UpdateByQueryRequest updateByQueryRequest, RequestOptions options) {
  24. Map<String, Object> catData = new HashMap<>(1);
  25. catData.put(ElasticSearchConstant.UPDATE_BY_QUERY_REQUEST, updateByQueryRequest.toString());
  26. return CatTransactionManager.newTransaction(() -> {
  27. try {
  28. return restHighLevelClient.updateByQuery(updateByQueryRequest, options);
  29. }catch (IOException e) {
  30. throw new RuntimeException(e);
  31. }
  32. }, ElasticSearchConstant.ES_CAT_TYPE, ElasticSearchConstant.UPDATE, catData);
  33. }

If there is such a way , Then use it as follows :

  
  1. @Test
  2. public void testUpdate5() {
  3. Map<String, Object> document = new HashMap<>();
  4. document.put("title", "Java");
  5. document.put("status", 0);
  6. document.put("tags", Lists.newArrayList("JS", "CSS"));
  7. kittyRestHighLevelClient.updateByQuery(elasticSearchIndexConfig.getArticleSaveIndexName(), new TermQueryBuilder("userId", 1), document);
  8. }

About author : Yin Jihuan , Simple technology enthusiasts ,《Spring Cloud Microservices - Full stack technology and case analysis 》, 《Spring Cloud Microservices introduction Actual combat and advanced 》 author , official account Ape world Originator .

I have compiled a complete set of learning materials , Those who are interested can search through wechat 「 Ape world 」, Reply key 「 Learning materials 」 Get what I've sorted out Spring Cloud,Spring Cloud Alibaba,Sharding-JDBC Sub database and sub table , Task scheduling framework XXL-JOB,MongoDB, Reptiles and other related information .

版权声明
本文为[Yin Jihuan]所创,转载请带上原文链接,感谢