HTTP Filesystem Driver
Looking for someone who will write a HTTP filesystem driver for Windows, following the further mentioned requirements.
As the name suggests, the goal is to mount a HTTP server as a disk volume in windows (for example to F:\) for both reading and writing.
Standard HTTP doesn't support writing, of course, so it is necessary to implement that by a special way, which will be described later.
It is possible to use pre-existing code (API) for the filesystem driver itself, from:
(I do not insist on that, it's just a suggestion)
The network operation itself has to be developed by you.
The behavior of network layer of the filesystem:
A filesystem as such supports multitasking, where many processes performs many operations simultaneously, at the same time. Contrary to that, HTTP protocol can handle just one operation (request) in the given connection to the server. Because of that, the network layer has to be able to pre-allocate (pre-connect) sockets to the http server, to handle many requests simultaneously using several connections, so the filesystem operations are handled as fast as possible.
The idea is such as you use an array of socked descriptors (sockfds), where the current status of individual sockets (connections) is analyzed by poll(). That way, you can be always sure that you do not write to a connection which should be read first, instead.
In the situation where all sockets are used (all opened connections are queried), an entirely new connection is to be opened (and its descriptor added to the array).
The operation of the HTTP filesystem:
- reading of data from particular files are done by a standard GET request, the HTTP server supports HTTP_RANGE.
- reading of meta data (file permissions, size, etc.) will be done by using a HEAD request. The webserver will be configured to send the requested infos in headers in each reply.
- reading of directory structure will be handled by a GET request, where the required info is read by parsing the HTML response. (more about it at the bottom)
The webserver will be configured to provide such infos in the given format.
- writing of data to particular files will be done by POST requests, also using HTTP_RANGE (the range says where to save the bytes, which are sent in the POST request body). The webserver will be configured to handle such POST requests and write the data.
- writing of metadata (change file permissions, create new file or new directory, etc.) will be done by a POST request without HTTP_RANGE; the command itself will be sent in headers, body of the request will be empty.
Webserver will be configured to handle the POST requests where no HTTP_RANGE header is sent and will understand what to to by reading the header (for example: "Server-Command: mkdir /test")
Example of HTTP response which has to be parsed to read directory structure:
<a href="img001.iso">img001.iso</a> 2009-12-31 17:32:14 345678943<br>
<a href="img002.iso">img002.iso</a> 2009-12-31 17:32:14 2345345345<br>
<a href="filename">filename</a> datetime_of_last_change size_in_bytes
User will enter URL of root-directory. He chooses which disk letter to use (eg. F:) from the select box of available letters. Then he sets how many connections are pre-allocated as default.
After the user clicks Start button, GUI is minimized to system tray and the virtual disk is created. Clicking the tray icon will popup the GUI again, where user can disconnect, and he can also see some stats and debug infos, for example how many connections are opened and what filesystem operations are currently being executed.
Final notes, as response to PMs:
I'm looking just for the CLIENT part. The server part is already developed. Moreover, all HTTP servers with mod_autoindex will support at least the read operations, so if the server part doesn't support write, the filesystem will be just read-only.
HTTP authentization must be supported by the client too.
The Eterlogic VDSDK is probably a bad example, I was told it only supports onRead and onWrite (raw operations to virtual harddisk). This is not enough for a HTTP filesystem, so I guess you can ignore that note.
more answers to questions:
What I need is a virtual disk drive for Windows, which is controlled by a client desktop application in system tray. Connections are opened only of they are needed, if the current operation ends, the connection is kept open for a while, waiting for another operation, and if nothing happens, it is closed by timeout (and reopened on demand if needed.).
The keep-alive header is used to ensure better performance when communicating with server.
If a file is modified, only the modified part of its data is 'uploaded' (using POST request with HTTP_RANGE). Of course if it is necessary to upload the whole file, then OK.