1500字范文,内容丰富有趣,写作好帮手!
1500字范文 > Elasticsearch 分布式搜索引擎 -- 自动补全(拼音分词器 自定义分词器 自动补全

Elasticsearch 分布式搜索引擎 -- 自动补全(拼音分词器 自定义分词器 自动补全

时间:2020-11-18 06:04:08

相关推荐

Elasticsearch 分布式搜索引擎 -- 自动补全(拼音分词器 自定义分词器 自动补全

文章目录

1. 自动补全1.1 拼音分词器1.2.1 自定义分词器1.2.2 小结1.2 自动补全1.3 实现酒店搜索框自动补全1.3.1 修改酒店映射结构1.3.2 修改HotelDoc实体1.3.3 重新导入1.3.4 自动补全查询的 JavaAPI1.3.5 实现搜索框自动补全1.3.5 启动项目进行测试

本节代码和数据库文件

链接:/s/1N_bpjL7TZbZK287Y8xAg2A?pwd=3210 提取码:3210

1. 自动补全

当用户在搜索框输入字符时,我们应该提示出与该字符有关的搜索项,如图:

这种根据用户输入的字母,提示完整词条的功能,就是自动补全了。

因为需要根据拼音字母来推断,因此要用到拼音分词功能。

1.1 拼音分词器

拼音分词器

1.2.1 自定义分词器

默认的拼音分词器会将每个汉字单独分为拼音,而我们希望的是每个词条形成一组拼音,需要对拼音分词器做个性化定制,形成自定义分词器。

elasticsearch中分词器(analyzer)的组成包含三部分:

character filters:在tokenizer之前对文本进行处理。例如删除字符、替换字符tokenizer:将文本按照一定的规则切割成词条(term)。例如keyword,就是不分词;还有ik_smarttokenizer filter:将tokenizer输出的词条做进一步处理。例如大小写转换、同义词处理、拼音处理等

文档分词时会依次由这三部分来处理文档:

我们可以在创建索引库时,通过settings来配置自定义的analyzer(分词器)︰

/medcl/elasticsearch-analysis-pinyin # 拼音分词器

声明自定义分词器的语法示例如下:

PUT /test{"settings": {"analysis": {"analyzer": {// 自定义分词器"my_analyzer": {// 分词器名称"tokenizer": "ik_max_word","filter": "py"}},"filter": {// 自定义tokenizer filter"py": {// 过滤器名称"type": "pinyin", // 过滤器类型,这里是pinyin"keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"name": {"type": "text","analyzer": "my_analyzer","search_analyzer": "ik_smart"}}}}

示例:

我们往里面插入2条测试数据:

POST /test/_doc/1{"id":1,"name":"狮子"}POST /test/_doc/2{"id":2,"name":"虱子"}

现在去查询数据:

GET /test/_search{"query": {"match": {"name": "被狮子咬一口怎么办?"}}}

1.2.2 小结

1.2 自动补全

elasticsearch提供了Completion Suggester查询来实现自动补全功能。这个查询会匹配以用户输入内容开头的词条并返回。为了提高补全查询的效率,对于文档中字段的类型有一些约束:

参与补全查询的字段必须是completion类型。字段的内容一般是用来补全的多个词条形成的数组。

比如,一个这样的索引库:

PUT /test2{"mappings": {"properties": {"title": {"type": "completion"}}}}

然后插入下面的数据:

POST /test2/_doc{"title": ["Sony","WH-1000XM3"]}POST /test2/_doc{"title": ["SK-II","PITERA"]}POST /test2/_doc{"title": ["Nintendo","switch"]}

查询的DSL语句如下:

# 自动补全查询GET /test2/_search{"suggest": {"title_suggest": {"text": "s","completion": {"field": "title","skip_duplicates": true,"size": 10}}}}

查询结果:

{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 0,"relation" : "eq"},"max_score" : null,"hits" : [ ]},"suggest" : {"title_suggest" : [{"text" : "s","offset" : 0,"length" : 1,"options" : [{"text" : "SK-II","_index" : "test2","_type" : "_doc","_id" : "8AJHCIABJKdUE2v6zZ0E","_score" : 1.0,"_source" : {"title" : ["SK-II","PITERA"]}},{"text" : "Sony","_index" : "test2","_type" : "_doc","_id" : "7wJHCIABJKdUE2v6zJ3y","_score" : 1.0,"_source" : {"title" : ["Sony","WH-1000XM3"]}},{"text" : "switch","_index" : "test2","_type" : "_doc","_id" : "8QJHCIABJKdUE2v6zZ0U","_score" : 1.0,"_source" : {"title" : ["Nintendo","switch"]}}]}]}}

1.3 实现酒店搜索框自动补全

现在,我们的hotel索引库还没有设置拼音分词器,需要修改索引库中的配置。但是我们知道索引库是无法修改的,只能删除然后重新创建。

另外,我们需要添加一个字段,用来做自动补全,将brand、suggestion、city等都放进去,作为自动补全的提示。

因此,总结一下,我们需要做的事情包括:

修改hotel索引库结构,设置自定义拼音分词器修改索引库的name、all字段,使用自定义分词器索引库添加一个新字段suggestion,类型为completion类型,使用自定义的分词器给HotelDoc类添加suggestion字段,内容包含brand、business重新导入数据到hotel库

1.3.1 修改酒店映射结构

代码如下:

PUT /hotel{"settings": {"analysis": {"analyzer": {"text_anlyzer": {"tokenizer": "ik_max_word","filter": "py"},"completion_analyzer": {"tokenizer": "keyword","filter": "py"}},"filter": {"py": {"type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"id":{"type": "keyword"},"name":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart","copy_to": "all"},"address":{"type": "keyword","index": false},"price":{"type": "integer"},"score":{"type": "integer"},"brand":{"type": "keyword","copy_to": "all"},"city":{"type": "keyword"},"starName":{"type": "keyword"},"business":{"type": "keyword","copy_to": "all"},"location":{"type": "geo_point"},"pic":{"type": "keyword","index": false},"all":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart"},"suggestion":{"type": "completion","analyzer": "completion_analyzer"}}}}

1.3.2 修改HotelDoc实体

HotelDoc中要添加一个字段,用来做自动补全,内容可以是酒店品牌、城市、商圈等信息。按照自动补全字段的要求,最好是这些字段的数组。

因此我们在HotelDoc中添加一个suggestion字段,类型为List<String>,然后将brand、city、business等信息放到里面。

代码如下:

package cn.itcast.hotel.pojo;import lombok.Data;import lombok.NoArgsConstructor;import java.util.ArrayList;import java.util.Arrays;import java.util.Collections;import java.util.List;@Data@NoArgsConstructorpublic class HotelDoc {private Long id;private String name;private String address;private Integer price;private Integer score;private String brand;private String city;private String starName;private String business;private String location;private String pic;private Object distance;private Boolean isAD;private List<String> suggestion;public HotelDoc(Hotel hotel) {this.id = hotel.getId();this.name = hotel.getName();this.address = hotel.getAddress();this.price = hotel.getPrice();this.score = hotel.getScore();this.brand = hotel.getBrand();this.city = hotel.getCity();this.starName = hotel.getStarName();this.business = hotel.getBusiness();this.location = hotel.getLatitude() + ", " + hotel.getLongitude();this.pic = hotel.getPic();// 组装suggestionif (this.business.contains("/")) {// business有多个值,需要切割String[] arr = this.business.split("/");// 添加元素this.suggestion = new ArrayList<>();this.suggestion.add(this.brand);Collections.addAll(this.suggestion, arr);} else {this.suggestion = Arrays.asList(this.brand, this.business);}}}

1.3.3 重新导入

重新执行之前编写的导入数据功能,可以看到新的酒店数据中包含了suggestion:

GET /hotel/_search{"suggest": {"suggestions": {"text": "sd","completion": {"field": "suggestion","skip_duplicates": true,"size": 20}}}}

1.3.4 自动补全查询的 JavaAPI

而自动补全的结果也比较特殊,解析的代码如下:

1.3.5 实现搜索框自动补全

1)在cn.itcast.hotel.web包下的HotelController中添加新接口,接收新的请求:

@GetMapping("suggestion")public List<String> getSuggestions(@RequestParam("key") String prefix) {return hotelService.getSuggestions(prefix);}

2)在cn.itcast.hotel.service包下的IhotelService中添加方法:

List<String> getSuggestions(String prefix);

3)在cn.itcast.hotel.service.impl.HotelService中实现该方法:

@Overridepublic List<String> getSuggestions(String prefix) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().suggest(new SuggestBuilder().addSuggestion("suggestions",pletionSuggestion("suggestion").prefix(prefix).skipDuplicates(true).size(10)));// 3.发起请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析结果Suggest suggest = response.getSuggest();// 4.1.根据补全查询名称,获取补全结果CompletionSuggestion suggestions = suggest.getSuggestion("suggestions");// 4.2.获取optionsList<CompletionSuggestion.Entry.Option> options = suggestions.getOptions();// 4.3.遍历List<String> list = new ArrayList<>(options.size());for (CompletionSuggestion.Entry.Option option : options) {String text = option.getText().toString();list.add(text);}return list;} catch (IOException e) {throw new RuntimeException(e);}}

1.3.5 启动项目进行测试

Elasticsearch 分布式搜索引擎 -- 自动补全(拼音分词器 自定义分词器 自动补全查询 实现搜索框自动补全)

本内容不代表本网观点和政治立场,如有侵犯你的权益请联系我们处理。
网友评论
网友评论仅供其表达个人看法,并不表明网站立场。