OpaDo Data Storage

OpaDo (a port of the TodoMVC app to Opa) now persists todo items to the Opa database. The new version is up on dotcloud, http://opado-tristan.sloughter.dotcloud.com/

I’ve added a todo_item type which stores the item’s value and two other attributes we won’t use until the next post when we have user accounts for their own todo_item stores.

type todo_item = { user_id : string
                 ; value : string
                 ; created_at : string

To tell Opa where to store the records we’ll create, we provide a path to the Opa db function and set its type. For our todo items we use a stringmap since currently the id’s are randomly generated strings (I know, I know, but its just an example!). We can then reference a record in the database with the path /todo_item[some_id_string].

db /todo_items : stringmap(todo_item)

Now we can insert todo_item‘s to this db path as so:

/todo_items[id] <- { value=x user_id="" created_at="" }

For now user_id and created_at are empty, but I’ll be updating that when I add user accounts.

Since we are storing each item, we need to populate the list on page load with whats already stored:

add_todos() =
  items = /todo_items
  StringMap.iter((x, y -> add_todo_to_page(x, y.value)), items)

The first line of the function sets the variable items to all the todo_item records in the database. We use StringMap.iter to take each todo_item and add it to the page. The first argument to the anonymous function is the id the item is stored in the database with (the id we will use in the HTML as well) and the second is the actual todo_item, so we take its value field and pass that to the add_todo_to_page function along with the id.

To have the add_todos function when the list element is ready we add an on_ready attribute that will call add_todos:

<ul id=#todo_list onready={_ -> add_todos() } ></ul>

Lastly, we want to be able to delete a todo_item from the database:

remove_item(id: string) =
  do Dom.remove(Dom.select_parent_one(#{id}))
  do Db.remove(@/todo_items[id])

remove_all_done() =
  Dom.iter(x -> remove_item(Dom.get_id(x)), Dom.select_class("done"))

The main piece to notice here is @/todo_items[id] in Db.remove(). The @ is saying that we are passing the path itself to remove() and not the value at that path.

Nice and easy! No database to setup or deploy, just Opa. Next time we’ll add user accounts, so we don’t have to all share the same todo list.

9 thoughts on “OpaDo Data Storage

  1. Jules Jacobs says:

    How do you deal with changes to the database structure? Say you have an user.firstname and user.lastname, and you want to change your database to just have an user.name. How can you do this while preserving existing data in the database by concatenating the firstname and lastname into name? Is there a way to update an application and migrate the database without downtime?

  2. Tristan Sloughter says:

    I don’t know about without downtime but Opa can figure out at least simple migrations:

    I’m using a user.opa module someone has on github but wanting to change the variable names in the user record (and database) to English.

    When I run the app after these changes it gives this message:

    The structure of this database doesn’t match the current program.
    It differs by some light alterations:
    – field ‘nom’ was renamed to ‘fullname’ at /users[]
    – field ‘passwd’ was renamed to ‘password’ at /users[]
    – field ‘prenom’ was renamed to ‘username’ at /users[]
    Automatic update is possible. If you agree with the above database migration
    information, re-run with the option –db-force-upgrade.

  3. HenJi says:

    If you have complex changes in database structures in a OPA program, you have two choices :

    1 – Keep both structure in your database and create a function which populates the new empty field from the other fields. Launch your program once with –db-force-upgrade and launch this function once (adding fields is safe but the new fields are empty). After this, clean removed fields and the function and re-run your program with –db-force-upgrade once more (removing fields is also safe). After this, you can keep using your program normally. A bit long but convenient for changes in small applications (with one instance running). You don’t have to relaunch your program immediately and you can spread ou migration on two program updates.

    2 – Create another OPA program with both database definitions but with root nodes named differently [1]. Your program should take data from the original node and put it in the new node with your changes. That way you can have a standalone opa application for your migration. Which is more convenient if your program is used by other users. Once your program compiled, you can use command line options (–db-local) to use your true program’s database as the input root node an to put the output root node in a safe place. However, I don’t know if it works if your program has no specific root node defined.

    [1] You can define a root node with `database my_root = @meta` and use it with `db /my_root/toto : stringmap(int)`. You can have multiple root nodes in one program.

  4. lamotta says:

    Is the OpaDo on dotcloud using MongoDB (like your latest github version) ?
    If it is would you ming sharing your dotcloud.yml ?

    • Tristan Sloughter says:

      Not yet. Let me try getting around to making that change today and I’ll share the dotcloud.yml file.

    • Tristan Sloughter says:

      Sorry it took so long to get back on this. But dotcloud has not kept up with Opa releases at all.

      So I’ve just now moved Opado to Rackspace cloud and using MongoLabs for the MongoDB instance.

    • Helpful comparison !I was tnyrig to get more information on Protocol Buffers, and Wikipedia linked me to Thrift. I found Protocol Buffers pretty well documented, while on the other hand Thrift looks like an alpha project from the documentation point of view It is a pity because Thrift seems to have quite a few interesting features. Considering that Thrift has been published like one year ago, I wonder if there is really a community backing it up and any enthusiasm around it ..It is quite an important point, because if you start using any of these libraries to communicate between different servers/services, you will probably have to stick to it for a few years In addition, it would be interesting to see if these libraries could be used to store hierarchical information in a database and see what would be the performances/limitations compared with storing XML or JSON. They should work better for a basic load/save strategy but what if you just want a subset of the data, do you still need to read the whole message ? The same applied if you want to redirect a message depending on one particular field without reading the whole message

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s