如何高效地将对象添加到hazelcast中的分布式队列中？

问题描述

当队列存储在与执行代码的那个节点不同的节点上（并且设置为0时），以hazelcast将项目添加到分布式队列中的速度非常慢（读取：66个项目/秒；这正常吗？）在此队列的配置中）。有什么方法可以从所有者节点添加项目？

Adding items to a distributed queue in hazelcast is incredibly slow (read: 66 items/sec; is that normal?) when the queue is stored on a different node than the one where the code is executed ( and is set to 0 in the config for this queue). Is there any way to add the items from the owner node? Is there anything fundamentally wrong with my approach using Hazelcast?

此操作大约需要15秒钟：

This operation takes around 15 seconds:

public static void main(String[] args) throws ExecutionException {
    HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
    IQueue<String> configs = hazelcastInstance.getQueue("configs");  
    for(int i = 0; i<1000;i++) {
        configs.add("Some string"+i);
    }
}

更改值和配置（请参见下文）对执行速度没有任何影响。我假设增加将阻止插入操作，而增加将不会（实际上，循环应像#add操作在本地队列中一样快地运行）。但是，执行for循环的时间相同。即使我都将两个值都设置为0。我是否遗漏了某些东西？

Changing the values and in the config (see below) doesn't have any influence on the execution speed. I'd assume that increasing would block the insert operations and increasing would not (actually the loop should be run through as quickly as if the #add operation was on a local queue). However, the time executing the for-loop is the same. Even if i set both values to 0. Am I missing something?

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation=
  "http://www.hazelcast.com/schema/config hazelcast-config-3.7.xsd"
  xmlns="http://www.hazelcast.com/schema/config"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <network>
        <port auto-increment="true" port-count="20">5701</port>
        <join>
            <multicast enabled="false">
        </multicast>
        <tcp-ip enabled="true">
            <member>172.105.66.xx</member> 
        </tcp-ip>
        </join>
    </network>

    <queue name="configs">
        <statistics-enabled>false</statistics-enabled>
        <max-size>0</max-size>
        <backup-count>0</backup-count>
        <async-backup-count>1</async-backup-count>
        <empty-queue-ttl>-1</empty-queue-ttl>
    </queue>
</hazelcast>

推荐答案

有几种方法可以优化队列。队列可以二进制或对象形式存储您的条目。最重要的是，您可以定义备份数量，备份越多，速度越慢。默认的复制备份数为1。这意味着对于每个pu，除了put之外，您还将有一个复制。使用二进制形式的条目存储时，可能最大程度地提高性能的是序列化。 Java序列化非常慢。

there are couple of ways you can optimize the Queue. The queue can store your entries in a binary or in object form. On top you can define the number of backups, the more backups the slower would it be. The default number of replicated backups is 1. This mean that for every pu you will have one replication besides the put. Probably the thing that increases the performance the most when using binary form of entry storage is the serialization. Java serialization is very slow. Use DataSerializeable or IdentifiedDataserializeable instead.

您需要aso来检查某些慢速网络的网络延迟，而对于仅用于1000次插入的网络通信，您将只有1-2MS延迟。 1000-2000 ms，这是只有1-2秒的网络等待时间。

You need aso to check your network latency for some slow networks you would have 1-2MS latency only for network comunication for 1000 inserts this would be 1000 - 2000 ms which is 1-2 second only network waiting.

对于批处理，只要插入顺序不重要，就可以并行进行插入。

For batch iperations you can do the inserts in paralel as long as the order of the inserts is not important.

并且Queue可以具有异步备份，这将再次提高性能。

And Queue can have asynchroneus backups which would again incease a bit the performance.

更新：
在应用程序的正常运行期间，您很少会遇到单一场景的情况。因此，不要一一插入这1000个值，请执行以下操作：

List arrayList = new ArrayList（）;

UPDATE:During normal running of the application you will rarely have a single theaded scenario. So instead of inserting these 1000 values one by one do the following:
List arrayList = new ArrayList();

for(int i=0;i<1000;++i) {
   arrayList.add("myString " + i);
}

arrayList.paralelstream().forEach(t->configs.put(t));

现在您要在paralel中插入。

Now you are inserting in paralel .

这篇关于如何高效地将对象添加到hazelcast中的分布式队列中？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！