[plug] moving mail in postfix

[Russell Steicke]

On 4/24/09, Daniel Pittman <daniel at rimspace.net> wrote:
> Russell Steicke <russellsteicke at gmail.com> writes:
>> On 4/17/09, Daniel Pittman <daniel at rimspace.net> wrote:
>>>...
>>> What, like that they are a dreadfully poor implementation of mh folders
>>> that aims to address one locking related issue that is no longer hugely
>>> relevant, and do so mostly by wishful thinking?
>>
>> What is the wishful thinking you're referring to here?
>
> The algorithm DJB describes for delivery is faulty in at least two
> cases, revolving around the "lockless" nature of maildir.
>
> One is that renaming messages while a readdir operation is in progress
> can, in practice, cause them to be omitted from the value returned to
> the calling program.
>
> This can cause concurrent operations on the maildir to cause a message
> to briefly vanish, then reappear; since the filename contains message
> flags this is not an uncommon operation.

This is something that I haven't seen mentioned before, and it sounds
like a potential problem.  I can't say that I've seen it in practice.

> The other is the delivery process, in which the implementer is
> instructed to construct a filename, stat it, then sleep two seconds if
> there was a file found.
>
> That races against the next operation, which is to create the file in
> .../tmp/ inside the maildir, where two processes could concurrently
> generate the same filename, receive a negative answer from stat and then
> try to create the same file.

This is overcome by using guaranteed unique information in the file
name: timestamp, host and process id.  He does mention that it is
possible for a host to recycle PIDs within a second, but very
unlikely.

Have you seen this create name collisions in practice?

> The same problem happens in the next step, where you link(2) the file
> from tmp to new, since renaming from new to cur could race with that and
> cause a duplicate delivery.

Ditto with the unique information in the file name.

I think you should bear in mind that djb spec'd Maildir a few years
ago (13 or 14 off the top of my head) and NFS delivery was much more
common then, and locking over NFS was then very unreliable.  Only a
couple of years after that, I was regularly losing mail as it was
delivered into mh mail directories, and was very glad to find Maildir,
which has never lost me any mail.  (IIRC, the mh man pages warned
about the possibility of losing mail with concurrent access.)

You've mentioned that NFS locking is fixed, but in my case, I've been
bitten by it being unreliable too often to trust my mail with it.  To
be clear: NFS is fine; trusting locks over NFS is not.


-- 
Virus found in this message.