[Date Prev][Date Next][Thread Prev][Thread Next][Author Index][Date Index][Thread Index]

Re: more repairs to green



Reed Hedges wrote:

> How about implementing all the data storage stuff as a filesystem driver?
> Then instead of having to work with the datastore is a single file,
> you can just work in xanadu terms with the disk :)

Depending on what level you're asking for, it might get really ugly. There'd be the development overhead of a year or so if you're looking for a Linux driver for something on par with e2fs or e3fs (e2fs with journalling). And, that would possibly be optomistic, having never written such a widget myself.

There was, at least for a while, a raw device available in Linux, though I was not very interested in it at the time. It might be possible to use the "loopback" device to mount large files as filesystems, but you're still stuck writing a new filesystem. Here's some useful links:

	http://www.itworld.com/nl/lnx_tip/10122001/
	http://www.llnl.gov/linux/ucrl-id-144213.html
http://www.db2.jp/db2manual/en_US/index.htm?openup=admin/t0004971.htm

I >have< used several databases (Sybase, DB2, Oracle) in raw-disk mode. Early Sybase 10 was not all baked, IMHO, but made rapid progress in release 11. It can be notably more efficient to direct to disk, as shown in Oracle's now-mature line. The DB2 ref above specifically addresses Linux.

All those companies inherited one legacy system after another, though, so their support for low-level drivers kept them occupied. Nowadays, "{CPU,Disk,Memory} is cheap," so people give up performance for portability.

The suggestion of using a SQL database as a back end is notably inefficient, but it would allow a database server to manage the low-level IO itself instead of causing everyone to wait for the wheel to be reinvented. It gets gains in portability, debuggability, simplification, and procedural controls to support backups and manage growth.

We're already headed to a point with the Web where it won't be able to encode and index knowledge efficiently for much longer. I would be easily convinced we're past that point if someone shows me where a multinational corp failed in unifying its operations because different subentities couldn't share knowledge. Heck, even a US corp would be enough. </trawl> The Web made people think differently about information and knowledge. That was a shock. Xanadu is another evolution above that. Maybe it won't be more than uncomfortable at first.

	John A. Dormer 2
	jad at texas dot net