This article continues the series that I have started featuring Hazelcast, a distributed, in-memory database. If one has not read the first post, please click here.
Distributed Collections
Hazelcast has a number of distributed collections that can be used to store data. Here is a list of them:
- IList
- ISet
- IQueue
IList
IList is a collection that keeps the order of what is put in and can have duplicates. In fact, it implements the java.util.List interface. This is not thread safe and one must use some sort of mutex or lock to control access by many threads. I suggest Hazelcast’s ILock.
ISet
ISet is a collection that does not keep order of the items placed in it. However, the elements are unique. This collection implements the java.util.Set interface. Like ILists, this collection is not thread safe. I suggest using the ILock again.
IQueue
IQueue is a collection that keeps the order of what comes in and allows duplicates. It implements the java.util.concurrent.BlockingQueue so it is thread safe. This is the most scalable of the collections because its capacity grows as the number of instances go up. For instance, lets say there is a limit of 10 items for a queue. Once the queue is full, no more can go in there unless another Hazelcast instance comes up, then another 10 spaces are available. A copy of the queue is also made. IQueues can also be persisted via implementing the interface QueueStore.
What They Have in Common
All three of them implement the ICollection interface. This means one can add an ItemListener to them. This lets one know when an item is added or removed. An example of this is in the Examples section.
Scalablity
As scalability goes, ISet and IList don’t do that well in Hazelcast 3.x. This is because the implementation changed from being map based to becoming a collection in the MultiMap. This means they don’t partition and don’t go beyond a single machine. Striping the collections can go a long way or making one’s own that are based on the mighty IMap. Another way is to implement Hazelcast’s spi.
Examples
Here is an example of an ISet, IList and IQueue. All three of them have an ItemListener. The ItemListener is added in the hazelcast.xml configuration file. One can also add an ItemListener programmatically for those inclined. A main class and the snippet of configuration file that configured the collection will be shown.
CollectionItemListener
I implemented the ItemListener interface to show that all three of the collections can have an ItemListener. Here is the implementation
package hazelcastcollections;
import com.hazelcast.core.ItemEvent;
import com.hazelcast.core.ItemListener;
/**
*
* @author Daryl
*/
public class CollectionItemListener implements ItemListener {
@Override
public void itemAdded(ItemEvent ie) {
System.out.println(“ItemListener – itemAdded: ” + ie.getItem());
}
@Override
public void itemRemoved(ItemEvent ie) {
System.out.println(“ItemListener – itemRemoved: ” + ie.getItem());
}
}
ISet
Code
package hazelcastcollections.iset;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.ISet;
/**
*
* @author Daryl
*/
public class HazelcastISet {
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
HazelcastInstance instance2 = Hazelcast.newHazelcastInstance();
ISet set = instance.getSet(“set”);
set.add(“Once”);
set.add(“upon”);
set.add(“a”);
set.add(“time”);
ISet set2 = instance2.getSet(“set”);
for(String s: set2) {
System.out.println(s);
}
System.exit(0);
}
}
Configuration
hazelcastcollections.CollectionItemListener
IList
Code
package hazelcastcollections.ilist;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IList;
/**
*
* @author Daryl
*/
public class HazelcastIlist {
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
HazelcastInstance instance2 = Hazelcast.newHazelcastInstance();
IList list = instance.getList(“list”);
list.add(“Once”);
list.add(“upon”);
list.add(“a”);
list.add(“time”);
IList list2 = instance2.getList(“list”);
for(String s: list2) {
System.out.println(s);
}
System.exit(0);
}
}
Configuration
hazelcastcollections.CollectionItemListener
IQueue
Code
I left this one for last because I have also implemented a QueueStore. There is no call on IQueue to add a QueueStore. One has to configure it in the hazelcast.xml file.
package hazelcastcollections.iqueue;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IQueue;
/**
*
* @author Daryl
*/
public class HazelcastIQueue {
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
HazelcastInstance instance2 = Hazelcast.newHazelcastInstance();
IQueue queue = instance.getQueue(“queue”);
queue.add(“Once”);
queue.add(“upon”);
queue.add(“a”);
queue.add(“time”);
IQueue queue2 = instance2.getQueue(“queue”);
for(String s: queue2) {
System.out.println(s);
}
System.exit(0);
}
}
QueueStore Code
package hazelcastcollections.iqueue;
import com.hazelcast.core.QueueStore;
import java.util.Collection;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
import java.util.TreeSet;
/**
*
* @author Daryl
*/
public class QueueQStore implements QueueStore {
@Override
public void store(Long l, String t) {
System.out.println(“storing ” + t + ” with ” + l);
}
@Override
public void storeAll(Map<Long, String> map) {
System.out.println(“store all”);
}
@Override
public void delete(Long l) {
System.out.println(“removing ” + l);
}
@Override
public void deleteAll(Collection clctn) {
System.out.println(“deleteAll”);
}
@Override
public String load(Long l) {
System.out.println(“loading ” + l);
return “”;
}
@Override
public Map<Long, String> loadAll(Collection clctn) {
System.out.println(“loadAll”);
Map<Long, String> retMap = new TreeMap<>();
return retMap;
}
@Override
public Set loadAllKeys() {
System.out.println(“loadAllKeys”);
return new TreeSet<>();
}
}
Configuration
Some mention needs to be addressed when it comes to configuring the QueueStore. There are three properties that do not get passed to the implementation. The binary property deals with how Hazelcast will send the data to the store. Normally, Hazelcast stores the data serialized and deserializes it before it is sent to the QueueStore. If the property is true, then the data is sent serialized. The default is false. The memory-limit is how many entries are kept in memory before being put into the QueueStore. A 10000 memory-limit means that the 10001st is being sent to the QueueStore. At initialization of the IQueue, entries are being loaded from the QueueStore. The bulk-load property is how many can be pulled from the QueueStore at a time.
10
hazelcastcollections.CollectionItemListener
hazelcastcollections.iqueue.QueueQStore
false 10000 500
Conclusion
I hope one has learned about distributed collections inside Hazelcast. ISet, IList and IQueue were discussed. The ISet and IList only stay on the instance that they are created while the IQueue has a copy made, can be persisted and its capacity increases as the number of instances increase. The code can be seen here.
References
The Book of Hazelcast: www.hazelcast.com
Hazelcast Documentation (comes with the hazelcast download)
Solving Interesting Problems by Technical Means 