当前位置:网站首页>Nacos for data synchronization

Nacos for data synchronization

2021-01-23 23:22:09 zhurd

The goal is

  • soul nacos Mode data synchronization principle and source code analysis

We were right in the last one Soul The gateway http Long polling A simple analysis of data synchronization mode is made , Got it http Long polling The basic process of synchronization . Let's take a look Soul The gateway nacos Data synchronization mode .

Soul Gateway on nacos Sync :

  • soul-bootstrap Add the following dependencies :

       <!--soul data sync start use nacos-->
        <dependency>
              <groupId>org.dromara</groupId>
               <artifactId>soul-spring-boot-starter-sync-data-nacos</artifactId>
               <version>2.2.1</version>
         </dependency>
  • application.yml Add related configuration

     soul :
         sync:
            nacos:
                 url: localhost:8848
                 namespace: 1c10d748-af86-43b9-8265-75f487d20c6c
                 acm:
                   enabled: false
                   endpoint: acm.aliyun.com
                   namespace:
                   accessKey:
                   secretKey:
     #url:  Configure it for your nacos Address , For cluster environment, please use (,) Separate .
     #  Other parameter configuration , Please refer to naocs Official website .

soul-admin To configure , Or in the soul-admin Set... In startup parameters --soul.sync.zookeeper='', Then restart the service

soul :
      sync:
         nacos:
              url: localhost:8848
              namespace: 1c10d748-af86-43b9-8265-75f487d20c6c
              acm:
                enabled: false
                endpoint: acm.aliyun.com
                namespace:
                accessKey:
                secretKey:

Source code analysis

soul-admin Data synchronization

soul-admin Data change notification for ,Soul Four data synchronization methods of gateway webscoket、zookeeper、http Long polling 、nacos The principle is the same , It's just that different data synchronization configurations have different event processors , Before zookeeper Data synchronization has been analyzed , I'm not going to go over it here ..

  • nacos Listener source code analysis

It's similar to the previous analysis ,NacosDataChangedListener Class is DataChangedListener The concrete implementation of the interface , To modify Selector For example :

public void onSelectorChanged(final List<SelectorData> changed, final DataEventTypeEnum eventType) {
        //getConfig  adopt  configService  Get configuration information 
        updateSelectorMap(getConfig(SELECTOR_DATA_ID));
        switch (eventType) {
            case DELETE:
                ...
                break;
            case REFRESH:
            case MYSELF:
                ...
                break;
            default:
                changed.forEach(selector -> {
                    List<SelectorData> ls = SELECTOR_MAP
                            .getOrDefault(selector.getPluginName(), new ArrayList<>())
                            .stream()
                            .filter(s -> !s.getId().equals(selector.getId()))
                            .sorted(SELECTOR_DATA_COMPARATOR)
                            .collect(Collectors.toList());
                    ls.add(selector);
                     // Replace with the latest selector information 
                    SELECTOR_MAP.put(selector.getPluginName(), ls);
                });
                break;
        }
        // Release data 
        publishConfig(SELECTOR_DATA_ID, SELECTOR_MAP);
    }

 private void updateSelectorMap(final String configInfo) {
        JsonObject jo = GsonUtils.getInstance().fromJson(configInfo, JsonObject.class);
        // At present  SELECTOR_MAP  all  key
        Set<String> set = new HashSet<>(SELECTOR_MAP.keySet());
        for (Entry<String, JsonElement> e : jo.entrySet()) {
            set.remove(e.getKey());
            List<SelectorData> ls = new ArrayList<>();
            e.getValue().getAsJsonArray().forEach(je -> ls.add(GsonUtils.getInstance().fromJson(je, SelectorData.class)));
            // Put the obtained configuration information into SELECTOR_MAP
            SELECTOR_MAP.put(e.getKey(), ls);
        }
        // Why do you have to remove?set Already in for In circulation remove It's empty , Think about it ?
        SELECTOR_MAP.keySet().removeAll(set);
    }

thus ,soul-admin The data transmission has been completed .

soul-bootstrap Gateway data synchronization

Turn on nacos Sync , Need to be in soul-bootstrap Introduction in soul-spring-boot-starter-sync-data-nacos, Find the corresponding customization in the project spring-boot-starter, Found out NacosSyncDataService Configuration class .

@Configuration
@ConditionalOnClass(NacosSyncDataService.class)
@ConditionalOnProperty(prefix = "soul.sync.nacos", name = "url")
@Slf4j
public class NacosSyncDataConfiguration {

    /**
     * Nacos sync data service.
     *
     * @param configService     the config service
     * @param pluginSubscriber the plugin subscriber
     * @param metaSubscribers   the meta subscribers
     * @param authSubscribers   the auth subscribers
     * @return the sync data service
     */
    @Bean
    public SyncDataService nacosSyncDataService(final ObjectProvider<ConfigService> configService, final ObjectProvider<PluginDataSubscriber> pluginSubscriber,
                                           final ObjectProvider<List<MetaDataSubscriber>> metaSubscribers, final ObjectProvider<List<AuthDataSubscriber>> authSubscribers) {
        log.info("you use nacos sync soul data.......");
        return new NacosSyncDataService(configService.getIfAvailable(), pluginSubscriber.getIfAvailable(),
                metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
    }

    /**
     * Nacos config service config service.
     *
     * @param nacosConfig the nacos config
     * @return the config service
     * @throws Exception the exception
     */
    @Bean
    public ConfigService nacosConfigService(final NacosConfig nacosConfig) throws Exception {
        Properties properties = new Properties();
        if (nacosConfig.getAcm() != null && nacosConfig.getAcm().isEnabled()) {
            properties.put(PropertyKeyConst.ENDPOINT, nacosConfig.getAcm().getEndpoint());
            properties.put(PropertyKeyConst.NAMESPACE, nacosConfig.getAcm().getNamespace());
            properties.put(PropertyKeyConst.ACCESS_KEY, nacosConfig.getAcm().getAccessKey());
            properties.put(PropertyKeyConst.SECRET_KEY, nacosConfig.getAcm().getSecretKey());
        } else {
            properties.put(PropertyKeyConst.SERVER_ADDR, nacosConfig.getUrl());
            properties.put(PropertyKeyConst.NAMESPACE, nacosConfig.getNamespace());
        }
        return NacosFactory.createConfigService(properties);
    }

    /**
     * Http config http config.
     *
     * @return the http config
     */
    @Bean
    @ConfigurationProperties(prefix = "soul.sync.nacos")
    public NacosConfig nacosConfig() {
        return new NacosConfig();
    }
}

With Selector For example , to glance at NacosSyncDataService Class listening Selector The logic of data change :

protected void updateSelectorMap(final String configInfo) {
        /*if(configInfo == null){
            return;
        }*/
        try {
            //configInfo  It's empty , Lead to  json  Deserialization failure 
            List<SelectorData> selectorDataList = GsonUtils.getInstance().toObjectMapList(configInfo, SelectorData.class).values().stream().flatMap(Collection::stream).collect(Collectors.toList());
            selectorDataList.forEach(selectorData -> Optional.ofNullable(pluginDataSubscriber).ifPresent(subscriber -> {
                // Remove cached data 
                subscriber.unSelectorSubscribe(selectorData);
                // Save cached data 
                subscriber.onSelectorSubscribe(selectorData);
            }));
        } catch (JsonParseException e) {
            log.error("sync selector data have error:", e);
        }
    }

above unSelectorSubscribe(selectorData)、onSelectorSubscribe(selectorData) Methods for updating cached data , The specific implementation classes are CommonPluginDataSubscriber, This is the same as the last one webscoket、zookeeper The call to update the cached data is the same .

nacos How to update data and webscoket、zookeeper The biggest difference :nacos Every time it's a full update , and webscoket、zookeeper Only one full update at startup , Other times are incremental updates .

problem

soul-bootstrap start-up NPE An error caused the startup to fail

Caused by: java.lang.NullPointerException: null
    at org.dromara.soul.sync.data.nacos.handler.NacosCacheHandler.updatePluginMap(NacosCacheHandler.java:90) ~[classes/:na]
    at org.dromara.soul.sync.data.nacos.handler.NacosCacheHandler.watcherData(NacosCacheHandler.java:167) ~[classes/:na]
    at org.dromara.soul.sync.data.nacos.NacosSyncDataService.start(NacosSyncDataService.java:56) ~[classes/:na]
    at org.dromara.soul.sync.data.nacos.NacosSyncDataService.<init>(NacosSyncDataService.java:49) ~[classes/:na]
    at org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration.nacosSyncDataService(NacosSyncDataConfiguration.java:66) 

Through error messages , Go to code

protected void updateSelectorMap(final String configInfo) {
        /*if(configInfo == null){
            return;
        }*/
        try {
            //configInfo  It's empty , Lead to  json  Deserialization failure 
            List<SelectorData> selectorDataList = GsonUtils.getInstance().toObjectMapList(configInfo, SelectorData.class).values().stream().flatMap(Collection::stream).collect(Collectors.toList());
            selectorDataList.forEach(selectorData -> Optional.ofNullable(pluginDataSubscriber).ifPresent(subscriber -> {
                // Remove cached data 
                subscriber.unSelectorSubscribe(selectorData);
                // Save cached data 
                subscriber.onSelectorSubscribe(selectorData);
            }));
        } catch (JsonParseException e) {
            log.error("sync selector data have error:", e);
        }
    }

stay soul-admin Change plug-ins in the background 、 Selectors and rule configurations , Start again , The anomaly disappeared . Doubt and nacos It's about the handling mechanism of , Every update is a full update , At the first initialization nacos No data , But after updating the data, it will trigger publishConfig Release data .

stay soul Of issues It's been mentioned in the book issue, It should be fixed soon .

thus ,nacos Data synchronization source code analysis completed .

版权声明
本文为[zhurd]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/01/20210123232137698t.html

随机推荐