working prototype storage daemon functionality:

introduction ¶

* Commands are given to storage daemon over a socket

* Implemented as a queue (commands are being queued)

* Multiple worker threads handle the commands.

* Based on policies the storage daemon will know what todo

What is the main purpose of the storage daemon ¶

processing of new or modified files on OSIS (V1) ¶

* copy new or modified file on OSIS to one or more of the storagepools available depending on the policy.

* While copying do optionally compression & encryption if necessary based on policy.

* Calculate the CRC while doing copying.

* Put a link on OSIS server towards the copied file on primary storagepool.

* If a slave OSIS server is present talk to daemon there and make sure a link is create there as well. (V2)

o Create a link to an existing file on storagepool (for slave operation)

delete OSIS file (V1.5) ¶

* Delete a file on the storagepool(s)

* Remove link and delete the metadata in the lucene DB.

Check health of OSIS file (V2) ¶

* Check health of file on storagepools.

* Multiple levels of checks are possible (for now basic & extended)

o In basic mode only size & file exists need to be checked,

o In extended mode, check if the file exists and read the file and verify against CRC (if encryption was used, create also CRC after encryption and/or compression)

+ more levels will be defined and included in later versions

Repair OSIS file (V1.5) ¶

* Make sure appropriate number of file copies exists on the storagepools (according to the policy), if not copy back.

redundant configuration of OSIS ¶

If installed in redundant mode every handled file on primary node will also be handled on slave.

Primary (V1) ¶

* copy file to storagepool(s) while respecting the policies

* optionally encrypt

* optionally compress

* calculate CRC

* create symlink between primary OSIS & primary storagepool

* convert supported formats to text and populate local lucene for full text search

* put all metadata in lucene

* tell slave to process document as well (as parameters all metada, full text (if supported doc), …)

Slave (V2) ¶

* create symlink between slave OSIS & primary storagepool?

* populate full text lucene as well as metadata lucene

requirements ¶

* written in java

project1/coder1: working prototype storage daemon functionality:

* see [url removed, login to view] (should be ready before 24-8-2006 evening)


Näytä lisää: working prototype, worker search, what levels, search introduction, ready talk, no working, node prototype, make prototype, make a prototype, create a prototype, crc health, what is node, what is a prototype, search o, now later, a prototype is, towards health, threads, socket java, s parameters

Tietoa työnantajasta:
( 2 arvostelua ) Lochristi, Belgium

Projektin tunnus: #82083

Myönnetty käyttäjälle:


DeQuo Systems

$1250 USD 10 päivässä
(4 Arvostelua)