JdeBP> LEAN is relatively uncomplicated and thus simpler than
JdeBP> many mainstream filesystem formats (especially it is a lot
JdeBP> simpler than FAT) to implement a full-featured FSD for.
JdeBP> But because it is badly designed it isn't a format that I'd
JdeBP> recommend for actual *use*.

BDL> With out starting a flame war or the such, why is it that
BDL> you think LEAN is "badly designed"?

I've meant to do a detailed write-up on LEAN for a while, now, and
haven't had the time to get around to it. I still haven't. (-: So the
following is cursory.

Some of the design problems that make it a poor choice for actual use
are that the design hasn't learned from history. Henry Spencer's famous
saying applies to filesystem format design just as much as it does to
Unix. Those who don't learn from history are condemned to reinvent it
poorly. There are reasons that modern filesystem formats don't use
linked lists of clusters to record allocations, and use records of
extents instead. There are reasons that modern filesystem designs don't
put superblocks or similar frequently-used primary data structures at
fixed positions at the beginning of the volume, but place them in the
middle of the volume and allow them to be variably positioned. There
are reasons that modern filesystem formats use magic numbers for more
than just superblocks. There are reasons that modern i-node structures
have *four* timestamps, not three. There are reasons (ironically, a
path of deduction that has just been re-trod in this very news group)
that modern filesystem formats use trees, usually B-trees, for
directories, not unsorted linked lists. There are reasons that modern
filesystem formats don't use 32-bit timestamp fields. There are reasons
that modern filesystem formats do not require free space bitmaps to be
contiguous, but in fact encourage them to be discontiguous. There are
reasons that modern filesystem formats permit free space bitmaps to grow
and to shrink without refomatting the volume. The design of LEAN hasn't
learned from history in any of these areas.

Other poor design aspects of LEAN:

Much of what is in the superblock duplicates what is in the BIOS
Parameter Block[1]. That duplication is wholly unnecessary. The design
should let the BPB do its job. The design documentation is also mute on
the subject of BIOS parameter blocks. Modern filesystem formats should
at least specify what the filesystem type field of BPBs should be for
the format[2], and what BPB fields have defined meanings for the format.

LEAN has no provision at all for extended attributes. This is one
design aspect that doesn't merely make LEAN a poor choice for actual
use. It actually _eliminates it_ as a choice for actual use. For
practical use, extended attribute support is a requirement in a
filesystem format, and has been since at least 1990.

LEAN has no provision for several POSIX permission bits, such as a
"sticky" bit.

LEAN has no provision for several Windows file/directory attributes,
such as the "O" attribute.

LEAN has no provision for POSIX special files.

LEAN has no provision for ACLs.

LEAN disallows filename characters less than 0x20, whilst _not_
disallowing the slash character. This is a truly bizarre design choice
(and is a consequence of the poor choice of how to find free space in a
directory for a new entry by maltreating a linked list as if it were an
array). Directories should not be unsorted linked lists in the first
place, and the on-disc data structure should now disallow any characters
in names at all. (The names in directory entries are, after all, length
counted on disc. There's no reason to even require that NUL be treated
specially.) What characters are disallowed in names should be a
function of the operating system, not of the on-disc data structures.
The on-disc data structures should be neutral with respect to operating
systems.

LEAN specifies two distinct "." and ".." entries in a directory. These
are superfluous, even for recovery purposes. For recovery purposes, a
simple parent link suffices. There is no need to explictly have these
two entries in the on-disc data structures at all. Contrast the unified
"start" entry in HPFS directories.

Some minor points:

The efficiency analysis of traversing the cluster chain in the LEAN
documentation is wrong. It ignores the fact that whilst it is possible
to skip entire "indirect clusters", it is still necessary to _read_ them.

The analysis of undeletion in the LEAN documentation is, to be
charitable, grossly superficial. It glosses over the possibility that
the space used to store the file has been re-used for other purposes.
LEAN itself provides no simple means to detect that an "indirect
cluster" has been overwritten with, say, ordinary file data in the
meantime. It doesn't even provide a means for checking that the i-node
hasn't been overwritten. An undeletion tool thus has no way to reliably
determine whether it will be able to get the actual file back. (Such
considerations are one of the several reasons that modern filesystems
use magic numbers for more than just superblocks, as mentioned above.)

One more point:

The LEAN documentation says "It is said that programmers like the FAT
file system for its simplicity.". That's not said by people who have
actually listened to programmers. What is in fact more widely said, by
those that listen to programmers, is that programmers *despise* the FAT
filesystem format. It is complex, inconsistent, replete with bodges and
optional/variant features (requiring multiple code paths), and hard to
implement full-function filesystem drivers for. (Ironically, from the
point of view of a programmer, HPFS is a lot *easier* to implement a
full-function FSD for than FAT is.) Programmers have been despising FAT
for years. Crank up Google Groups, and you'll find programmers railing
against FAT since the 1980s.

[1]
<URL:http://homepages.tesco.net./~J.deBoynePollard/FGA/bios-parameter-block.html>
[2]
<URL:http://homepages.tesco.net./~J.deBoynePollard/FGA/determining-filesystem-type.html>

"Jonathan de Boyne Pollard" <J.deBoynePollard-(E-Mail Removed)> wrote
in message news:c1.01.31ZwDk$(E-Mail Removed)...
> JdeBP> LEAN is relatively uncomplicated and thus simpler than
> JdeBP> many mainstream filesystem formats (especially it is a lot
> JdeBP> simpler than FAT) to implement a full-featured FSD for.
> JdeBP> But because it is badly designed it isn't a format that I'd
> JdeBP> recommend for actual *use*.
>
> BDL> With out starting a flame war or the such, why is it that
> BDL> you think LEAN is "badly designed"?
>
> I've meant to do a detailed write-up on LEAN for a while, now, and haven't
> had the time to get around to it. I still haven't. (-: So the following
> is cursory.
>

<snip a fairly detailed break down of why the LEAN file system
isn't the NTFS/HPFS/EXTx/RieserFS/your favorite FS here file system>

Jonathan,

Granted, the LEAN FS isn't anywhere near what a modern FS should
be, but for what its design was meant for, it doesn't have too
many faults.

Yes, the inode should have at least a magic value and I have asked
the author if he would add a magic and a crc field to it.

However, for its purpose, it really isn't "badly designed" in
my opinion.

Anyway, I am not trying to start/continue a flame war or something
of the sort, so I will end my comments here.

I think for its purpose, the LEAN FS is a decent FS. I appreciate
your comments, though I think they were well overdone.

Thanks,
Ben

Hi Jonathan,
thanks for your notes, although I can't understand some of your
concerns.
Firstly, LEAN is in alpha. It is a 0.1 specification, and nothing
prevents to change it or throw it away entirely. Moreover, I thought I
clearly stated that LEAN aims to be the simplest file system to
implement, while as far as I can see everything you said
[over]complicates it.
In the specification I explicitely asked for constructive criticism, I
wonder why I never received your e-mail before... but I feel that is
not your purpose, hence I'm leaving this discussion, before it goes
flamed and offtopic.
Thank you anyway,
Salvo

Salvo Isaja wrote:
> Hi Jonathan,
> thanks for your notes, although I can't understand some of your
> concerns.
> Firstly, LEAN is in alpha. It is a 0.1 specification, and nothing
> prevents to change it or throw it away entirely. Moreover, I thought I
> clearly stated that LEAN aims to be the simplest file system to
> implement, while as far as I can see everything you said
> [over]complicates it.
> In the specification I explicitely asked for constructive criticism, I
> wonder why I never received your e-mail before... but I feel that is
> not your purpose, hence I'm leaving this discussion,

Hopefully after you distil the constructive advice from Jonathan's
post.

> before it goes
> flamed and offtopic.
> Thank you anyway,
> Salvo

Benjamin David Lunt wrote:
> [...] Yes, the inode should have at least a magic value and I have
> asked the author if he would add a magic and a crc field to it.


Jonathan de Boyne Pollard wrote :

> There are reasons that modern filesystem formats use magic numbers
> for more than just superblocks.


why would that be needed? to ensure that data which spans more than one
sector was written completly? if yes, what about journalling?





> There are reasons that modern i-node structures have *four*
> timestamps, not three.

which are? i use only two timestamps: created and modified.



> For practical use, extended attribute support is a requirement in a
> filesystem format, and has been since at least 1990.

why? have a look at reiserfs: http://www.namesys.com/stream_ans.html

reiserfs does not support extended attributes. and ntfs doesn't make
very much use of the extended attributes.


regards,
simon

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

toby wrote:
> Salvo Isaja wrote:
[snip]
> > hence I'm leaving this discussion,
> Hopefully after you distil the constructive advice from Jonathan's
> post.

Of course. I'm already thinking about several of those issues.
More are welcome (but please bear in mind LEAN's goals).
Thanks,
Salvo
--

Simon Felix wrote:
> ...
> Jonathan de Boyne Pollard wrote,
> > For practical use, extended attribute support is a requirement in a
> > filesystem format, and has been since at least 1990.
>
> why? have a look at reiserfs: http://www.namesys.com/stream_ans.html
>
> reiserfs does not support extended attributes.

You seem to mean "forks" (MacOS since 1983 or so) or "streams" (NTFS,
http://windowssdk.msdn.microsoft.com.../ms685571.aspx ).
Also see: http://en.wikipedia.org/wiki/Fork_%28filesystem%29

I think Jonathan really did mean extended attributes ("arbitrary
name/value pairs which are associated with files or directories", see,
e.g. http://acl.bestbits.at/about.html or
http://wiki.linuxquestions.org/wiki/Extended_attributes ), as his
following examples suggest. Reiser3fs (and Reiser4) do support such
attributes, as do, "in kernel version 2.6, ext2, ext3, ..., JFS, XFS
and NFS". These can be used to implement ACLs.

Additionally, Reiser4 has generalised support for "forks" or "streams".

> and ntfs doesn't make
> very much use of the extended attributes.

Given the interoperability problems with one-fork filesystems (viz. the
great proliferation of incompatible encodings for MacOS data +
resource, for instance), that's probably a blessing.

>
>
> regards,
> simon
>
> ----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
> http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
> ----= East and West-Coast Server Farms - Total Privacy via Encryption =----

toby wrote:
> Simon Felix wrote:
>> ...
>> Jonathan de Boyne Pollard wrote,
>>> For practical use, extended attribute support is a requirement in a
>>> filesystem format, and has been since at least 1990.
>> why? have a look at reiserfs: http://www.namesys.com/stream_ans.html
>>
>> reiserfs does not support extended attributes.
>
> You seem to mean "forks" (MacOS since 1983 or so) or "streams" (NTFS,
> http://windowssdk.msdn.microsoft.com.../ms685571.aspx ).
> Also see: http://en.wikipedia.org/wiki/Fork_%28filesystem%29
>
> I think Jonathan really did mean extended attributes ("arbitrary
> name/value pairs which are associated with files or directories", see,
> e.g. http://acl.bestbits.at/about.html or
> http://wiki.linuxquestions.org/wiki/Extended_attributes ), as his
> following examples suggest. Reiser3fs (and Reiser4) do support such
> attributes, as do, "in kernel version 2.6, ext2, ext3, ..., JFS, XFS
> and NFS". These can be used to implement ACLs.
>
> Additionally, Reiser4 has generalised support for "forks" or "streams".

no it has not, according to their own website:
http://www.namesys.com/v4/v4.html#compose_streams


instead hans reiser made the decision to make directories also look like
a file. because reiserfs is optimized for many small files it implements
extended attributes and streams just like regular files in a directory.

________________________________________________________________________

There are also requests that we add more and more different kinds of
attributes using more and more different APIs. Can we do everything that
can be done with {files, directories, attributes, streams} using just
{files, directories}? I say yes [...]
________________________________________________________________________


and i think they got this right.



>> and ntfs doesn't make
>> very much use of the extended attributes.
>
> Given the interoperability problems with one-fork filesystems (viz. the
> great proliferation of incompatible encodings for MacOS data +
> resource, for instance), that's probably a blessing.

100% ack.


that's why i don't like the idea of streams. the first filesystem i
designed used streams for everything. but after working with it i
recognized that it's just not worth it. a single stream per file is
easier to maintain, just as flexible (if your filesystem supports
directories) and more portable.


regards,
simon


--

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

JdeBP> LEAN is relatively uncomplicated and thus simpler than
JdeBP> many mainstream filesystem formats (especially it is a lot
JdeBP> simpler than FAT) to implement a full-featured FSD for.
JdeBP> But because it is badly designed it isn't a format that I'd
JdeBP> recommend for actual *use*.

BDL> With out starting a flame war or the such, why is it that
BDL> you think LEAN is "badly designed"?

JdeBP> I've meant to do a detailed write-up on LEAN for a while,
JdeBP> now, and haven't had the time to get around to it. I still
JdeBP> haven't. (-: So the following is cursory.
JdeBP>
JdeBP> <snip a fairly detailed break down of why the LEAN file system isn't
JdeBP> the NTFS/HPFS/EXTx/RieserFS/your favorite FS here file system>

No. It was why it isn't a format that I'd recommend for actual *use*.

BDL> Granted, the LEAN FS isn't anywhere near what a modern
BDL> FS should be, but for what its design was meant for, it
BDL> doesn't have too many faults.

According to its documentation, what it was meant for was actual use.
To quote:

"to provide a new free, simple, portable, personal file system
alternative
to the proprietary (and in part patented) Microsoft FAT file system,
primarly intended for media exchange and for use with embedded
devices, [...]"

It's poorly designed and not suitable for actual use. You may think
that its purpose is to be a platform for experiment, or that its purpose
is to be an exercise in filesystem implementation for hobbyist operating
system developers. It may well be suitable for those purposes. But
those are *not* its stated purposes. Its stated purpose is to provide a
free alternative filesystem format for *use* in "media exchange" and
"with embedded devices".

BDL> However, for its purpose, it really isn't "badly designed" in
BDL> my opinion.

Its stated purpose is actual use, for which I wouldn't recommend it
because it is badly designed. Again, you appear to have some idea of its
purpose that is at variance with what is actually stated as its purpose
in its documentation.

BDL> With out starting a flame war or the such, why is it that
BDL> you think LEAN is "badly designed"?

JdeBP> I've meant to do a detailed write-up on LEAN for a while,
JdeBP> now, and haven't had the time to get around to it. I still
JdeBP> haven't. (-: So the following is cursory.
JdeBP> [...]

SI> Moreover, I thought I clearly stated that LEAN aims to be the
SI> simplest file system to implement, while as far as I can see
SI> everything you said [over]complicates it.

Producing a usable filesystem format requires that one support a wide
range of features. If one is aiming for implementation simplicity
alone, one might as well reinvent the FAT format from MS-DOS version 1,
which had no subdirectories, a fixed-size root directory, and a
directory format that was just a straightforward array of entries. That
is not a filesystem format that I'd recommend for actual use, for many
of the same reasons that LEAN is not a filesystem format that I'd
recommend for actual use. But you aren't aiming for implementation
simplicity at all. Your aim, stated in your document, is to produce
something that is free of the intellectual property problems of the FAT
filesystem format, and that is suitable for real world use in "media
exchange" and in "embedded devices".

You need to work out what your actual aim is. If it still is what your
document *says* it to be, namely a usable filesystem format that is free
of the encumbrances of FAT, then you should be designing with *use* in
mind. If it not what your document states it to be, but in fact is
implementation simplicity alone, then you should give up any pretense
that a usable filesystem format will result. You also need to rid
yourself of the misconception, apprently underlying your thinking, that
people pick FAT because it is simple to implement. FAT isn't simple to
implement. (Several other filesystem formats are simpler to implement.
I've already mentioned that HPFS is. They have fewer optional/variant
features and inconsistencies to deal with, and fewer alternative code
path requirements.) The commercial operating system developer picks FAT
because of widespread interoperability. The hobbyist picks FAT for one
of many possible reasons (It's what's described in the primary case
study in a book on operating systems that xe has. Xe thinks that
because it was once simple, with MS-DOS version 1, that it is simple
now. Xe thinks that it is easier to implement because it is widely
documented. Or xe thinks that imitating MS-DOS is the best path to take
when learning operating system design.). Neither encounters
implementation simplicity. The former doesn't even consider it to be
the goal.

SI> In the specification I explicitely asked for constructive criticism,
SI> I wonder why I never received your e-mail before...

There is no need to wonder. I told you why, right at the start.

SI> but I feel that is not your purpose,

My (readily apparent) purpose was to answer Benjamin David Lunt's direct
question, quoted above.