Alan Gutierrez

In response to my frank comments at the indev.ca forums and my blog post How I Lost My Morning to Tagging and EMail, one Scott Morrison wrote me to assure me that the data was not lost, and asked that I help troubleshoot the problem.

Obviously someone who suffers the delusion that the customer is always right. I subscribe to the Be Nice Or Leave school of customer service. Especially when the customer has yet to pay for anything.

So, rather than be a complete bastard, I zipped up a copy of a sample message, one from the server, one from my MacBook, and sent it on its way. Maybe that will tell him something.

But, what would I do if it was my code? I’d connect to the IMAP server and trace though the code. I really want MailTags to work. I figured, why not just set up an email account for Scott? No better way to reproduce the problem.

I set up an account and sent him a message.

A few hours later, three messages from Scott. The first two zeroing in on the fact that Dovecot stomps the X-Keywords header, where the tags are stored. They are also stored in a JSON string in the X-MailTags header, so the third message had a new MailTags bundle that reads the tags from that header.

After dropping it into place, I rebuild my mailboxes, and my tags reappearedThey were never lost, stored in two separate headers. Apparently, Courier IMAP is not capricious as Dovecot. It doesn’t munge headers the way Dovecot does. Scott’s communications shows that he knows his way around IMAP. Certainly keeping a MailTags specific header for the tags shows foresight. That time spent tagging was not lost.

Now, I’m tagging away again. MailTags is very much a part of my workflow.

That and Mail ActOn, which is another indev.ca addition to Mail.app.

Actually, I’d spent the day despondent wondering what I was going to do without MailTags, now that I’d become accustomed to it.

Update: It is a known issue with Dovecot, as noted in Problem/workaround with X-Keywords and Apple’s Mail.app. I wonder how this is address in Thunderbird, since it seems to always behave correctly, although there is evidence of issues as in 1.0 beta3 not preserving X-Keyword header?.

Update:: I’m now, as of April 14th, 2007, eagerly awaiting Public Beta 10. I’ll be checking the Mail Tags 2.0 site frequently. Public Beta 9 just expired. Scott Morison expires his betas so people are forced to upgrade, which is not a bad idea at all.

I’ve been fastidiously tagging my email using Mail.app and MailTags. I’d developed a routine, where I’d copy my mail into a directory for tagging, and then add tags. I’d make sure that the days mail was tagged before shuffling it to an archive.

When MailTags is added to Mail.app, search becomes very slow. This was a problem that I reported in the forums under Search is Unacceptably Slow.

Now, when I rebuild a mailbox, the tags are not downloaded. I’ve reported this in IMAP Loses Tags. This is the deal breaker.

Now I’m looking at Tag the Bird again and wondering how long it would take to hack in IMAP support. Tagging in Thunderbird 2.0 is a joke, as noted in the discussion on the Mozilla wiki, one choses their tags from a drop down menu, which means you must limit your vocabulary.

Reference was made to Tagocity, an extension for Outlook. Thunderbird and MailTags should strive to be half of this, for starters.

I don’t think the guys at Thunderbird really get tagging. As a developer, when I’m in heavy software development mode, my email channels are few and high-volume. The proverbial fire hose. A focused, high-pressure jet of email.

I’ll subscribe to mailing lists, and exchange information with a few collaborators. I can create folders to manage a handful of projects, that have a large volume of mail.

Thunderbird is great for listservs, but with a listserv, you don’t need futher categorization. The discipline of moderation keeps the listserv on topic. Tagging is nothing you need to do.

In my work in outreach, email comes from everywhere, and will make reference every aspect of my work. It is a monsoon. Classification of the email is the first step in responding to it. Gathering the droplets of information into tributaries. Here tagging is a potential life changing tool.

Developers don’t get tagging. (Thunderbird developers especially. The Mozilla project is hamstrug by it’s many, many forums, and depenedency on IRC.) Developers are accustomed to well moderated listservs and newsgroups.

GMail doesn’t do tagging right. The drop down list is painfully slow. You must create a list of tags to choose from. You can’t add a tag that you might use only every once in a while. “French Quarter Festivial” is something that I might get email about ten times a year. I don’t want to have to scroll past “French Quarter Festival”, 363 days of the year.

It’s only saving grace is that it tags a conversation, so that you only need to do this once, and messages thereafter are tagged.

MailTags is close to ideal, except that it’s broken. I’d love to help fix it, but the new version is not open source.

The last post I wrote, something about citizen journalists, has been lost in the recent exodous from the old server. Maitri left a comment, and her comment was lost as well. My apologies.

• An interface that compares an array, or partial array of comprables, for Strata.
• Then crack objects or XML documents to obtain the comparables.
• This new interface is external to Strata.
• Internal to Strata there are different interfaces for
  • Comparing objects.
  • Determining the equality of objects.
  • Reading objects from the tier.
  • Writing objects to the tier.
  • Obtaining the object to write.

Now I’d like to not have to keep the object in memory at all, but extract the fields. The object to write is still a key, but it can be resolved into just the field values, kept in an array, to compare the array of values.

Why is this not the default implementation?

• Memento Project Page
• Memento Posts

An ideal DOM, with a small footprint would write the file flat, and read it into an array.

Any DOM could be made easier if Bento could divide into blocks and there was a way to read across blocks, as one contiguous output stream.

This low memory answer would be somewhat like tiny tree. An array of nodes, rather than a tree. The arrays are gathered into a linked list. A node wraps a reference to a list node an index into the list node. At regular offsets in the list node, one has an indication of the depth or the index of the parent, so that it is easy to find your way back up the tree. When an array is broken, an offset is adjust so that these things can still be found.

Writing out the array mends it. It gets written out in the clean format, the broken arrays discarded and reread.

Memento does not index nodes. It indexes fragments based on the contents.

XPath can be used to extract the fields of an index, but XPath is not otherwise in play.

With a document, such as…


<person>
<first-name>Alan</first-name>
<last-name>Gutierrez</last-name>
</person>


(Uh, oh! WordPress cannot render example XML? Man this is broken!)

You can create an index using the XPath results from the query /person/last-name. I plan on writing a nice implementation that will select index participants using those XPath statements.

This doesn’t mean that you query the bin that contains the fragment, that you will receive an element with the name last-name. You’ll receive a list of fragments. If you want to extract nodes from those fragments, such as the element with the name last-name, then you’ll have to navigate the fragments by another means.

In fact, the query will return fragment identifiers, so that you can run the query, without deserializing the fragments. The fragments could be indexed using XOM and XPath, for example, but deserialized using JiXB, into a Java “Person” object.

At first, because I imagined a seamless integration with Saxon, it seemed necessary to index individual nodes.

Question: Is it possible to create an index of individual nodes by running the query used to create the fragment index against the returned fragments? Is it desirable?

Question: What happens when a fragment is duplicated in a index because a list property is indexed by individual element? Let’s say, a list of favorite colors indexes a person by both blue and red.

• Memento Project Page
• Memento Posts

At one point, I’d considered providing named fragments, that would be defined by the creator object. This is not necessary if there is a data dictionary, or the potential for a data dictionary.

Indicies are implemented by the object model. They act on a materialized object.

If an object is serialized using one object model, but indexed using another, then the index will have to deserialize the object in order to index it.