Alan Gutierrez

Trying to tone down the self-congradulations at the Wikipedia articles for AmericaSpeaks. Wikipedia, another idea that is not quite right, when you look at it from the perspective of a person who is effected by the information it disseminates.

Not quite right.

I can’t possibly compete with the millions that AmericaSpeaks has to produce advertising copy. How do you find a middle road view based on the advertising copy that people will cut and paste into the AmericaSpeaks article?

The last post I wrote, something about citizen journalists, has been lost in the recent exodous from the old server. Maitri left a comment, and her comment was lost as well. My apologies.

Authentic c-store Japanese bento by Ramil Sagum.

Malloc

The idea behind Bento it to take a file and divide it up using the malloc algorithm as described by Doug Lea.

Bento is written in Java for JDK 1.4.2 using NIO.

You can read more about Bento on my Wiki and you can paruse the source in my Subversion repsitory and you can use it under LGPL.

Design Goals

Bento is a file format for writing out discrete blocks of data. It is designed for the storage of objects. Bento is not supposed to be a database in itself.

Bento is a robust file format that provides

• storage of blocks of arbitrary size,
• random access to stored blocks,
• atomic writes of multiple blocks,
• thread-safe allocation and deallocation of blocks,
• and thread-safe reads and writes.

Funamentally, Bento strives to be a simple data structure that you can use to persist objects in your applications. You can use Bento as the basis for more complicated data structures or databases.

Usage

Bento can be used to serialize objects using Java serialization, to write out objects as Java primitives, or to store Java primitives.

A single Bento file can be organized into a database of different Java types. For desktop applications, this means that a single file can be the container for many different objects of many different classes.

Multiple Bento files can be organized into a database, spread across different file system, to reduce disk contention.

You can design resuable Bento based data structures and mix them into a single Bento file.

Quick Start

An example program that creates a Bento file with a static header and a single allocated block.

Bento.Creator creator = new Bento.Creator();
creator.addStaticBlock(URI.create("flag://agtrz.com/header"), 512);
Bento bento = creator.create(new File("data.bento"));

Bento.Mutator mutator = bento.mutate();
try {
    Bento.Block block = mutator.allocate(512);

    ByteBuffer bytes = block.toByteBuffer();
    for (int i = 0; i < 512 / 4; i++) {
        bytes.putInt(i);
    }
    block.write();

    Bento.Block header = mutator.load(URI.create("flag://agtrz.com/header"));

    ByteBuffer bytes = header.toByteBuffer();
    block.getAddress().write(bytes);
    header.write();

    mutator.commit();
}
finally {
    mutator.rollback();
}

bento.close();

Bento.Opener opener = new Bento.Opener();
bento = opener.open(new File("bento.data"));

ByteBuffer header = bento.read(URI.create("http://agtrz.com/header"));

ByteBuffer bytes = bento.read(new Bento.Address(header));
for (int i = 0; i < bytes.capacity() / 4; i++) {
    if (i != bytes.getInt()) {
        throw new IllegalStateException();
    }
}

bento.close();

Mutation

A Bento file is written using a Bento.Mutator. All of the changes performed by a mutator are atomic. The changes are not made perminent until the commit method of mutator is called. The changes can be discarded by calling the rollback method of the mutator.

After calling commit or rollback the mutator can be reused. Calling commit or rollback when there are no uncommited changes held by the mutator does nothing. Thus, we use a try/finally block to recover the state of a Bento file when a failure occurs.

public void storeObjects(Bento bento, Bento.Address address) {
    Bento.Mutator mutator = bento.mutate();
    try {
        Bento.Block block = mutator.load(address);
        block.toByteBuffer().putInt(this.magicNumber);
        block.write();

        mutator.commit();
    }
    finally {
        mutator.rollback();
    }
}

Concurrency

Bento synchronizes the list of free blocks, but does not synchronize the blocks themselves. Blocks can be allocated and freed in a thread-safe manner without further consideration by the application developer.

If you read and write blocks without some syncrhonization strategy, you'll turn your Bento file into garbage in short order.

Multiple threads can read the same blocks, but blocks that are being written should not be written or read by other threads. Reads can be shared. Writes are exclusive.

A simple strategy is to use a read/write lock from the concurrency library to gaurd the file during updates.

Bento strives for liveness, however. It places as few locks as possible on the management of list of free blocks.

A more sophisticated strategy would place locks on the blocks themselves. A composite object, such as a tree, could lock a tree root element. Different threads can then operate on different trees in the same file. They would not run afoul of each other.

Atomic Writes

A Bento file is edited using a mutator the object. The mutator object keeps all block writes, allocations, and deallocations are kept in memory until the user commits or rolls back the changes made by the mutator.

Journaling

All reads, writes, allocations and frees are performed on buffers in memory. Only when a mutation is committed or rollback, is the underlying file written.

Journaling is implemented by writing out the list of operations into a journal prior to applying the changes to the file.

The journal can be implemented using blocks within the file, or it can be implemented using external temporary files. The former is good for desktop applications, where the user may move a corrupted Bento file out of reach of the external journals, or unwittingly delete the external journals. The latter is good for a server application, where multiple Bento files and journals can exist in a single directory, watched over by an administrator.

Thank you to the people who have been playing along with the Creative Commons. It has made it much easier to post at Think New Orleans. Because we’re sharing our photos in Flickr photo groups, I can pluck an image to put a post in context, in the context of our lives that we’re all living together.

Spoke with Dave Coustan yesterday, over dinner at Coop’s. Dave asked if ther there have been any notorious instances of astroturfing in the New Orleans blogs. I mentioned the carpet bloggers.

There is a group of people who have visited New Orleans and are writing breathless stories in a blog about secrets they’ve uncovered. I a recent video post, a they fear for their lives to write about them. It’s sounds like a radio drama of the days of old, where you couldn’t show horror, only describe it. It’s like Suspense).

They’ve adorned their post using the wealth of heartrending images available on the Internet, as if these were their own photographs.

There is little you can do about this. People are reading them. If someone contradicts them in the comment section, someone else writes a defense. Some locals have been helping them.

They have made content of tragedy. There is a market for everything, it seems. Another bucket into which to wring your sponge of daily anguish.

A local blogger even met with them, not understanding that they are producing a low-budget serial, that will say what it takes to increase page rank.

The long tail will find an audience that will listen to fictional accounts of Katrina, made up as they go along, and not respond with questions, but with emotions. It is participatory disaster porn.

Over at a blog called Kiloblog, I’m writing about my programming experiences. I’m wondering if it belongs there or here? It seems that I ought to write closer to Think New Orleans, so I can attact the attention of people who program in XSLT and Groovy, because I’m about to say raving nice things about both. It may serve to flesh out my readership, and draw more attention to that other thing I like to write about, New Orleans. How to I keep people in New Orleans from running away screaming when they hear me talk about immutable decorator implementations?

The first Geek Dinner with wine supplied by Stormhoek. This dinner will be held at Alan’s ghetto French Quarter courtyard and apartment, on July 14th, 2006 a Friday, at or about 7:00 pm.

It will take the form of a pot luck. I’ll buy as much food as my “budget” will allow, but you should all consider bringing something of your own.

Now go ahead, and RSVP. You can leave a message in the comments. I’m writing a nice document that says who is attending and who is who at the Geek Dinners page of the New Orleans Wiki.

I am creating weekly email newsletter for Think New Orleans. This is in addition to the subscribe by email option on the Think New Orleans weblog. It is a traditional newsletter. It arrives in email, on a Monday morning, to tell a person what’s going to happen in the coming week.

• Subscribe to the Weekly Think New Orleans Newsletter

This is going to cause some confusion. When people get my weekly newsletter, they are going to want a newsletter just like it for their organization. (I am working on a service that will publish a newsletter from a weblog. A real email newsletter, by selecting a specific tag, or tag feed.)

I avoid the term with neighborhood groups, because they are starting with nothing, or a poorly maintained web site. They eed a simple way to publish. A web site where they make frequent updates that look like email messages sounds simple to them. It is like a “newsletter”.

If I were to say “blog”, they might think of MySpace or personal diaries. They might search the web and come across an article like the following…

The second thing blogs do is – to invoke Marx – seize the means of production. It’s hard to underestimate what a huge deal this is. For as long as journalism has existed, writers of whatever kind have had one route to readers: They needed an editor and a publisher. Even in the most benign scenario, this process subtly distorts journalism. You find yourself almost unconsciously writing to please a handful of people – the editors looking for a certain kind of story, the publishers seeking to push a particular venture, or the advertisers who influence the editors and owners. Blogging simply bypasses this ancient ritual.

It creates more work. I’ve had people say, “I don’t want to blog”, or I’m sorry, “I can’t stop thinking of it as something that my son does”.

Some of the organizations for which I’ve established web sites, are further down the road, however.

Bill Knecht and Paul Schafer both know what a blog is. Bill Knecht already runs a very succesful email newsletter. He follows the ettiqutte of infrequent messages, and regularly scheduled announcements. They understand the concept, without quoting Marx. They were enthused about creating the Eracism Blog from the outset.

Thus, it’s not a hard and fast rule. The model for blogging is “conversation”. “Conversation” is the model for ERACE New Orleans. It is a good fit.

Other organizations are not looking to converse on the web. They converse in person. They want to use the web to publish and coordinate. For them it is not siezing the means of production, it’s posting the next meeting agenda.