dojo.storage API

I'm starting to port AMASS over to Dojo, including increasing the range and sophistication of the storage options available. Here's a rough outline of the design I have so far before I launch into coding; if you have any ideas or ways to make the API better feel free to contact me.

Basicly, on startup, a dojo.storage.manager figures out the best possible permanent storage system based on the platform's capabilities and initializes it. For example, if the user has Flash and we are on a browser, we use the FlashStorageProvider; if we don't have Flash and we are on IE, we use the IEStorageProvider which uses proprietary features of IE to get up to 60K of permanent storage. If we are being run as a Firefox extension, then we use the XPCOMStorageProvider, etc. All of these providers expose a single, consistent API that basicly makes storage look like a hashtable, with get and put methods. This means programmers can have permanent storage for their apps, independent of the platform's underlying capabilities.

Users can also manually grab other kinds of storage providers, such as a FormStorageProvider which can save session information using the autosave capability of web forms. This is useful for caching runtime information, for example. It's also useful in various kinds of exotic hacks. We also provide the same hashtable abstraction to cookies, using a CookieStorageProvider.

Here's my current design outline I'm working from:


dojo.storage - the best available permanent storage
dojo.storage.cookies - hashtable interface to cookies


dojo.storage.browser (also exposes
dojo.storage.cookies) - browser.js
FlashStorageProvider - Uses hidden flash applet
IEStorageProvider - Uses IE features for storage
FormStorageProvider - Uses a hidden form
field for storage
CookieStorageProvider - Uses cookies for storage
WhatWGStorageProvider - Uses a browser-provided
storage system as
defined by the WHAT
Working Group;
placeholder until
their work is done

dojo.storage.wsh - wsh.js
ActiveXStorageProvider - Uses ActiveX for storage

dojo.storage.rhino - rhino.js
JavaStorageProvider - Uses LiveConnect against
Java for storage

dojo.storage.xpcom - xpcom.js
XPCOMStorageProvider - Uses XPCOM components
for storage

dojo.storage:

Constants that are returned from various operations:

dojo.storage.SUCCESS
dojo.storage.FAILED
dojo.storage.PENDING

dojo.storage.SIZE_NOT_AVAILABLE
dojo.storage.SIZE_NO_LIMIT

dojo.storage.manager - singleton
-------------------

Initializes the storage systems and figures out
the best available storage options on this platform

/** Instructs the storageManager to use
the given storageClass for all storage requests.

Example:

dojo.storage.setProvider(
dojo.storage.browser.IEStorageProvider)
*/
setProvider(storageClass)



AbstractStorageProvider
-----------------------

The base class for all storage providers

/** Puts a key and value into this storage system.

@param key A string key to use when retrieving
this value in the future.
@param value A value to store; this can be
any JavaScript type.
@param resultsHandler A callback function
that will receive two arguments.
The first argument is one of three
values: dojo.storage.SUCCESS,
dojo.storage.FAILED, or
dojo.storage.PENDING; these values
determine how the put request went.
In some storage systems users can deny
a storage request, resulting in a
dojo.storage.FAILED, while in
other storage systems a storage
request must wait for user approval,
resulting in a dojo.storage.PENDING
status until the request
is either approved or denied,
resulting in another call back
with dojo.storage.SUCCESS.

The second argument in the call back is an
optional message that details possible error
messages that might have occurred during
the storage process.

Example:
var resultsHandler = function(status, message) {
alert("status="+status+", message="+message);
};
dojo.storage.put("test", "hello world",
resultsHandler);
*/
put(key, value, resultsHandler)

/** Gets the value with the given key. Returns null
if this key is not in the storage system.

@param key A string key to get the value of.
@returns Returns any JavaScript object type;
null if the key is not
present. */
get(key)

/** Determines whether the storage has the given
key.

@returns Whether this key is
present or not. */
hasKey(key)

/** Enumerates all of the available keys in
this storage system.

@returns Array of string keys in this
storage system.
*/
getKeys()

/** Completely clears this storage system of all
of it's values and keys. */
clear()

/** Returns whether this storage provider's
values are persisted when this platform
is shutdown.

@returns True or false whether this
storage is permanent. */
isPermanent()

/** Returns whether this storage provider is
supported on this platform.

@returns True or false if this storage
provider is supported.
*/
isSupported()

/** The maximum storage allowed by this provider.

@returns Returns the maximum storage size
supported by this provider, in
thousands of bytes (i.e., if it
returns 60 then this means that 60K
of storage is supported).

If this provider can not determine
it's maximum size, then
dojo.storage.SIZE_NOT_AVAILABLE is
returned; if there is no theoretical
limit on the amount of storage
this provider can return, then
dojo.storage.SIZE_NO_LIMIT is
returned. */
getMaximumSize()

/** Determines whether this provider has a
settings UI.

@returns True or false if this provider has
the ability to show a
a settings UI to change it's
values, change the amount of storage
available, etc. */
hasSettingsUI()

/** If this provider has a settings UI, it is
shown. */
showSettingsUI()



Now, I've just got to code this thing :)

Comments

WPWoodJr said…
Nice design.

What happens if the user has IE without flash, then gets flash? Will the storage system remember that it was using IE proprietary storage before or will it assume Flash and not see that there is stuff stored in IE? Perhaps this could be handled by a cookie which stores which method was previously used?

Will there be any way to tag a set of storage with an application name to prevent variable name collisions? Perhaps in the initializer you could pass an application name which is used to partition the storage.

- Bill
Brad Neuberg said…
Bill, these are both two great issues you've pointed me to, which I didn't think about. I'll have to update the design to incorporate them.

Best,
Brad
Lucas Gonze said…
A really incredible piece of work, Brad. This breaches one of the major walls separating browser apps from local apps, even if only for small amounts of data.
Brad Neuberg said…
Lucas, thanks for the kind comments! Some day we'll get to actually meet and grab some coffee together. Are you ever in San Francisco? You should drop by the coworking space sometime; check it out at http://codinginparadise.org/coworking.

Best,
Brad
Brad Neuberg said…
Thanks Erik! You're one of my heros too. :)

Once I create the scaffolding I will check it into the Dojo subversion repository and you can start filling out one of the providers. One possibility is you can take the general design in that blog post for a SearchProvider and fill it out for a specific kind of provider. What do you think about looking at the XPCOMSearchProvider, the ActiveXSearchProvider, or the JavaSearchProvider? They are all straightforward; the tricky part are the put, get, and isAvailable methods. For the XPCOM and ActiveX offline storage you can take a look at the TiddlyWiki code, which figured out how to do this.
Brad Neuberg said…
One thing I should mention is that if you write any code that we incorporate into Dojo you will have to sign a code submission contract; it's straightforward, basicly the same thing the Apache projects and others require. It's so that copyright is clearly assigned. I can ask Alex Russell to send you the form if you are interested. Dojo itself is under both the BSD and the AFL licenses, so it's easy to incorporate into other projects.