Files replication framework
The
Replicator allows replicating files between a server and client(s). Producers publish
revisions and consumers update to the latest revision available.
ReplicationClient is a helper utility for performing the update operation. It can
be invoked either
manually or periodically by
starting an update thread.
HttpReplicator can be used to replicate revisions by consumers that reside on
a different node than the producer.
The replication framework supports replicating any type of files, with built-in support for a single search index as
well as an index and taxonomy pair. For a single index, the application should publish an
IndexRevision and set
IndexReplicationHandler on the client. For an index and taxonomy pair, the
application should publish an IndexAndTaxonomyRevision and set
IndexAndTaxonomyReplicationHandler on the client.
When the replication client detects that there is a newer revision available, it copies the files of the revision and
then invokes the handler to complete the operation (e.g. copy the files to the index directory, fsync them, reopen an
index reader etc.). By default, only files that do not exist in the handler's
current revision files are copied,
however this can be overridden by extending the client.
An example usage of the Replicator:
// ++++++++++++++ SERVER SIDE ++++++++++++++ //
IndexWriter publishWriter; // the writer used for indexing
Replicator replicator = new LocalReplicator();
replicator.publish(new IndexRevision(publishWriter));
// ++++++++++++++ CLIENT SIDE ++++++++++++++ //
// either LocalReplictor, or HttpReplicator if client and server are on different nodes
Replicator replicator;
// callback invoked after handler finished handling the revision and e.g. can reopen the reader.
Callable<Boolean> callback = null; // can also be null if no callback is needed
ReplicationHandler handler = new IndexReplicationHandler(indexDir, callback);
SourceDirectoryFactory factory = new PerSessionDirectoryFactory(workDir);
ReplicationClient client = new ReplicationClient(replicator, handler, factory);
// invoke client manually
client.updateNow();
// or, periodically
client.startUpdateThread(100); // check for update every 100 milliseconds