imperialWicket

am i the only croquet-playing computer nerd?

« nReduce: Virtual team meetings Getting started with Zotonic »

Colosimo: Chicago Boss, PostgreSQL, and bcrypt

2012-06-10

If you are interested in Chicago Boss and PostgreSQL (then hopefully you'll like this!), and you have not reviewed it already, go check out An Evening With Chicago Boss, I'm going to borrow a good deal from that example.

The items I want to cover that enhance the example in An Evening with Chicago Boss are:

  • PostgreSQL DB shard for content
  • Static content: CSS, images, javascript
  • More in-depth Django templates with extensions and blocks
  • Bcrypt for password storage
  • Front-end user registration

The source for Colosimo is available on github; I did not include boss.config, logs, or ebin. If you clone the repo, add a boss.config with appropriate credentials (my boss.config is below for reference) and make sure you run "./rebar compile", or start it in dev mode.

Install Chicago Boss and create our project

I documented the effort for installing Chicago Boss on Amazon Linux, and there are good instructions on the Chicago Boss wiki. I'm going to skip the actual installation procedures and go right to project creation. Feel free to comment here, or hit the Chicago Boss mailing list if you encounter issues with installation.

Related to installation, I want to note that I'm going to show the colosimo/boss.config file with my Chicago Boss, cb_admin, and colosimo projects all in my home directory. If you install these in unique locations, some path updates will obviously be necessary.

After installing Chicago Boss (not necessary if you clone the github repo):

cd ~/ChicagoBoss/
make app PROJECT=colosimo

boss.config: DB shards, models, and cb_admin

Here is a boss.config file that supports cb_admin, uses Mnesia for most storage, and uses PostgreSQL for our colosimo_user model. We're also using Mochiweb as our server.

[{boss, [
    {path, "/home/imperialwicket/ChicagoBoss"},
    {vm_cookie, "abc123"},
    {applications, [cb_admin,colosimo]},
    {db_host, "localhost"},
    {db_port, 1978},
    {db_adapter, mock},
    {log_dir, "log"},
    {server, mochiweb},
    {port, 8001},
    {session_adapter, mock},
    {session_key, "_boss_session"},
    {session_exp_time, 525600},
    {db_shards, [
        [{db_host, "localhost"},
        {db_adapter, pgsql},
        {db_port, 5432},
        {db_username, "colosimo"},
        {db_password, "colosimo"},
        {db_database, "colosimo"},
        {db_shard_id, shard_pg},
        {db_shard_models, [colosimo_user]}]
    ]}
]},
{cb_admin, [
        {path, "../cb_admin"},
        {base_url, "/admin"}
]},
{colosimo, [
    {path, "../colosimo"},
    {base_url, "/"}
]}
].

Note that if you receive this error upon running init.sh or init-dev.sh:

./init.sh: 55: ERROR:: not found

You probably have a syntax error in the boss.config file that resides in the same directory as the init script you executed.

PostgreSQL database

I am using a vanilla PostgreSQL installation, and you'll notice I'm using the terribly secure user/pass of 'colosimo' and 'colosimo' for my database named 'colosimo'.

Chicago Boss comes with a nice README_DATABASE that details the generation of your tables in PostgreSQL (and other databases), and my aim is not PostgreSQL in this article. Nonetheless, here are the statements to get you up and running with this example (assuming PostgreSQL is installed, a database is initialized, and the database server is running).

postgres=#CREATE USER colosimo WITH PASSWORD 'colosimo';
postgres=# CREATE DATABASE colosimo WITH OWNER colosimo;
postgres=# GRANT ALL ON DATABASE colosimo TO colosimo;
postgres=# \c colosimo colosimo
postgres=# CREATE TABLE colosimo_users (id SERIAL UNIQUE PRIMARY KEY, email VARCHAR(200), username VARCHAR(24), password VARCHAR(64));
NOTICE:  CREATE TABLE will create implicit sequence "colosimo_users_id_seq" for serial column "colosimo_users.id"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "colosimo_users_pkey" for table "colosimo_users"
CREATE TABLE

Static content (html5-boilerplate)

I am using html5 boilerplate as a starter for my template. All you need to do is extract the boilerplate files to the ~/colosimo/priv/static/ directory, and then update the relative paths to map to "/static/" instead of just "/". To fill out the templating effort, I removed a bunch of items that were unnecessary for this project, and added a couple template blocks to the index file. I also made some basic changes to the css for things like inline display of the nav li.

In order to use the index.html file as the base for my templates I moved it from the /priv/static/ directory to the colosimo view directory (and renamed it to 'base.html'):

mv ~/colosimo/priv/static/index.html ~/colosimo/src/view/base.html

Template extensions and blocks

Templates are Django-style, and as mentioned I am using the 'base.html' as a base template for colosimo. You can review any of the views to see the 'extends "base.html"' bit. You can review all of the views on github, or check the ~/colosimo/src/views directory if you've already cloned the repo.

Mostly, you just need to keep in mind that the template extends our base.html file and provides input for the title and content blocks.

Routing and Controllers

Routing in Chicago Boss is likely familiar, and I'll just note here that I used the following routes to support my index page and the about page (404 and 500 are in there for good measure):

/priv/colosimo.routes:
% Front page
{"/", [{controller, "main"}, {action, "index"}]}.

% About
{"/about", [{controller, "main"}, {action, "about"}]}.

% 404 File Not Found handler
{404, [{controller, "main"}, {action, "nope"}]}.

% 500 Internal Error handler (only invoked in production)
{500, [{controller, "main"}, {action, "oops"}]}.

We will cover the user controller elsewhere, but I wanted to explicitly detail my 'main' controller here:

~/colosimo/src/controller:
-module(colosimo_main_controller, [Req]).
-compile(export_all).

before_(_) ->
    user_lib:require_login(Req).

index('GET', [], ColosimoUser) ->
  {ok, [{colosimo_user, ColosimoUser}]}.

nope('GET', []) ->
  {ok, []}.

oops('GET', []) ->
  {ok, []}.

about('GET', []) ->
  {ok, []}.

Notice we handle the login check only on the index page.

Install and use bcrypt for passwords

I wanted to install bcrypt at the Erlang level, not just for Chicago boss. On Debian the default location for Erlang libs is the '/usr/local/lib/erlang/lib/' directory. I am new to Erlang, and if there's a better way to accomplish this, please add it in the comments and I'll update the text accordingly. On Debian:

sudo su -
cd /usr/local/lib/erlang/lib/
git clone https://github.com/mrinalwadhwa/erlang-bcrypt.git
cd erlang-bcrypt
make

This puts the necessary *.beam files in your Erlang libs, so you can use bcrypt wherever you want (in Erlang).

As I mentioned, we're going to carry over a lot of the code from An Evening with Chicago Boss, but the appropriate bcrypt modifications to the user controller, the user model and user library are as follows:

~/colosimo/src/controller/colosimo_user_controller.erl

Add registration to the user controller:

-module(colosimo_user_controller, [Req]).
-compile(export_all).

login('GET', []) ->
    {ok, [{redirect, Req:header(referer)}]};

login('POST', []) ->
    Username = Req:post_param("username"),
    case boss_db:find(colosimo_user, [{username, Username}], 1) of
        [ColosimoUser] ->
            case ColosimoUser:check_password(Req:post_param("password")) of
                true ->
                   {redirect, proplists:get_value("redirect",
                       Req:post_params(), "/"), ColosimoUser:login_cookies()};
                false ->
                    {ok, [{error, "Authentication error"}]}
            end;
        [] ->
            {ok, [{error, "Authentication error"}]}
    end.

register('GET', []) ->
    {ok, []};

register('POST', []) ->
    Email = Req:post_param("email"),
    Username = Req:post_param("username"),
    Password = bcrypt:hashpw(Req:post_param("password"),bcrypt:gen_salt()),
    ColosimoUser = colosimo_user:new(id, Email, Username, Password),
    Result = ColosimoUser:save(),
    {ok, [Result]}.
~/colosimo/src/model/colosimo_user.erl

Update the model with additional fields and use bcrypt:

-module(colosimo_user, [Id, Email, Username, Password]).
-compile(export_all).

-define(SETEC_ASTRONOMY, "Too many secrets").

session_identifier() ->
    mochihex:to_hex(erlang:md5(?SETEC_ASTRONOMY ++ Id)).

check_password(PasswordAttempt) ->
    StoredPassword = erlang:binary_to_list(Password),
    StoredPassword =:= bcrypt:hashpw(PasswordAttempt, StoredPassword).

login_cookies() ->
    [ mochiweb_cookies:cookie("user_id", Id, [{path, "/"}]),
        mochiweb_cookies:cookie("session_id", session_identifier(), [{path, "/"}]) ].
~/colosimo/src/lib/user_lib.erl

Update the user lib to use bcrypt:

-module(user_lib).
-compile(export_all).

hash_password(Password)->
    bcrypt:hashpw(Password, bcrypt:gen_salt()).

require_login(Req) ->
    case Req:cookie("user_id") of
        undefined -> {redirect, "/user/login"};
        Id ->
            case boss_db:find(Id) of
                undefined -> {redirect, "/user/login"};
                ColosimoUser ->
                    case ColosimoUser:session_identifier() =:= Req:cookie("session_id") of
                        false -> {redirect, "/user/login"};
                        true -> {ok, ColosimoUser}
                    end
            end
     end.

Registration and Login

We already saw the sections of the user controller that support registration and login behavior. Here are the views that setup the POST requests for our users (sorry, I'm linking to github here, because I'm too lazy to update the blog for embedding html code).

~/colosimo/src/view/user/register.html
~/colosimo/src/view/user/login.html


Colosimo

Hopefully this works for others and provides some useful examples as more people get started with Chicago Boss. Don't hesistate to comment if things don't work, or if there are any errors that should be addressed.

Again, thanks to the Chicago Boss team for a great framework, and particularly to the team that generated, updated, and maintains the wiki page: An Evening with Chicago Boss, it definitely got me started and I used a lot of the sample code in the Colosimo project.

11 Responses to Colosimo: Chicago Boss, PostgreSQL, and bcrypt

Feed for this Entry

11 Comments

  • Hello,
    First, I want to thank you for writing this tutorial. It's yet another piece of documentation that has helped me take one step further toward learning website development.

    I attempted to work through this tutorial, but I seem to have hit a snag. Currently, I can do all the following:
    write to the database from the terminal when running colosimo
    access the pages
    see the admin page
    install and compile bcrypt according to my computer's directory "/usr/lib64/erlang/lib/erlang-bcrypt"
    bcrypt things according to the example on its git page

    However, whenever I click on "Register" on the "/user/register" page, it spits out this error:
    {{badmatch,undefined},
    [{bcrypt,mechanism,0},
    {bcrypt,gen_salt,0},
    {colosimo_user_controller,register,3},
    {boss_web_controller,execute_action,5},
    {boss_web_controller,process_request,5},
    {timer,tc,3},
    {boss_web_controller,handle_request,3},
    {mochiweb_http,headers,5}]}

    I thought that it may not be starting up the bcrypt module, so I tried:
    bcrypt:start()

    and it responded with:
    application: bcrypt
    started_at: colosimo@my_computer

    I tried the register page afterward, and this time it spits out this error:
    {{function_clause,
    [{bcrypt_nif_worker,terminate,
    [{badarg,
    [{bcrypt_nif,hashpw,
    [<<>>,#Ref<0.0.0.184390>,<0.193.0>,"password",
    {ok,"$2a$12$9x3OwL3LN11R4vgxx010ge"}]},
    {bcrypt_nif_worker,handle_call,3},
    {gen_server,handle_msg,5},
    {proc_lib,init_p_do_apply,3}]},
    {state,12,<<>>}]},
    {gen_server,terminate,6},
    {proc_lib,init_p_do_apply,3}]},
    {gen_server,call,
    [bcrypt_nif_worker,
    {hashpw,"password",{ok,"$2a$12$9x3OwL3LN11R4vgxx010ge"}},
    infinity]}}

    I'm kind of lost and I must admit, I'm not the best Erlang debugger yet.
    I've only spent a couple good days teaching it to myself so far. If you
    have any suggestions, they'd be very much appreciated.

    #8861 | Comment by Arthur Burkart on Jul 11, 2012 06:01pm
  • I figured it out. I'm not sure about how correct my solution is, but here's what I got:
    My first problem was the fact that bcrypt and crypto weren't started when I typed: ./init-dev.sh
    I fixed this by adding the following function to colosimo_user_controller.erl:
    cryptostart() ->
    crypto:start(),
    bcrypt:start().

    I call cryptostart() from within the user controller's login('POST', []) and register('POST', []) functions.
    The reason I don't think this is a good solution is because it throws an error every time it's called when bcrypt and crypto are already running. I haven't figured out how to get around this yet.

    The code in the colosimo_user model's check_password(PasswordAttempt) had to be changed to:
    {ok, Password} =:= bcrypt:hashpw(PasswordAttempt, Password).
    Perhaps bcrypt has been updated?

    The code inside the register('POST', []) function now looks like:
    register('POST', []) ->
    cryptostart(),
    Email = Req:post_param("email"),
    Username = Req:post_param("username"),
    %% I needed to do a bit of pattern-matching for gen_salt() and
    %% hashpw() in order to make the password store correctly.
    {ok, Salt} = bcrypt:gen_salt(),
    {ok, Hash} = bcrypt:hashpw(Req:post_param("password"),Salt),
    ColosimoUser = colosimo_user:new(id, Email, Username, Hash),
    Result = ColosimoUser:save(),
    {ok, [Result]}.

    It is now fully functional, but like I said, I don't think my cryptostart() function is a good idea. I don't know where to place the bcrypt:start() function so the application functions appropriately.

    Regards,

    Chachi B

    #8863 | Comment by Arthur Burkart on Jul 11, 2012 09:42pm
  • @Chachi B - Glad this was helpful, even if you found some bugs!

    In terms of erlang-bcrypt, I'd guess it's an issue with erlang loading libs from 32 or 64 bit lib locations (just a guess...). Maybe try symlinking to the library in your 32 bit dir?

    If you are going to manually load bcrypt in your controller, it seems like you should move the library into your application, and not store it in the server level erlang libs. Just my 2 cents.

    I think the functionality issue you identified was a bug in the model that I introduced when moving a cast from within the model out to the module declaration. I'll review the code and try to get that updated as soon as I can. Thanks for reviewing and posting your notes!

  • @Chachi B - Actually, the ChicagoBoss/deps directory would be appropriate for the bcrypt lib, and I believe CB auto loads everything in there when you start the server.

  • @nicholas Thanks, I was starting to wonder how I'm going to push my application to my server without installing all the dependencies directly onto the server. Knowing how to make my application independent definitely eliminates an obstacle.

    #8899 | Comment by Arthur Burkart on Jul 13, 2012 02:22pm
  • @nicholas
    Well, I moved the erlang-bcrypt folder from /erlang/lib to ChicagoBoss/deps and it is recognized, but it didn't solve the bcrypt:start() problem. I still have to run that in order for it to work.

    I'm under the assumption that there is a file somewhere that I have to explicitly say bcrypt:start(). It's by no means urgent for me to find it right away, but when I stumble across it, I'll be sure to tell you.

    best,

    Chachi B

    #8900 | Comment by Arthur Burkart on Jul 13, 2012 04:04pm
  • Anyone here looking for resolution for bcrypt (or other dependency starting) should definitely check the google groups discussion.

  • Hi Nicholas Whittier,
    I have 3 questions:

    1. Is there conflict between {db_adapter, mock} and {db_adapter, pgsql} (in db_shards) ?
    2. If I add "blabla" table. It must be specified {db_shard_models, [colosimo_user, blabla]} or we don't need, "blabla" will be in mock ?
    3. In db_shards could we use more than one database, for example postgres and riak ?

    #17445 | Comment by bronzeboyvn on Jun 12, 2013 01:42am
  • @bronzeboyvn -

    1. The mock adapter will get used as a default adapter. This means that if you try to create a new model and fail to add it explicitly to the pgsql db_shard_models list, it will default to place those entities in mnesia (using the mock adapter).
    2. Exactly.
    3. Absolutely - which is awesome. I had a project that was using postgres and riak exactly, as a matter of fact. pg for user creds and really relational bits, and riak for items served via API that I wanted to be a little less formally structured.

    Unfortunately, I am not at liberty to share much about that project, but hopefully I'll get to play with CB more soon. Check out the mailing list (if you haven't already), many of the database adapters are relatively young, and there are occasionally bugs. The riak adapter had a handful of weird issues when I was using it - many due to Basho's updates from version to version. I know there have been a number of updates to the adapter since I used it.

    Based on lurking in the list and watching the internet, the MySQL, MongoDB, and PostgreSQL adapters get most of the attention. Mnesia is certainly in there too, with most other dbs getting updated primarily when someone finds an issue.

  • Thanks, Nicolas!

    Before I've tested (1) and (2) issues from your colisimo, I asked these question at #chicagoboss. One guy gave wrong answer, he told that we can choose only one database, even not sure if there are conflict between mock and postgres. He's not right person for me to ask.
    I'm happy with your answers. I've learned much from it. Could you show me db_shards part that uses postgres and riak ?

    I'm no sure

    {db_shards, [ [{postgres_info}, {riak_info}] ],

    or

    {db_shards, [ [{postgres_info}], [{riak_info}] ],

    #17472 | Comment by bronzeboyvn on Jun 12, 2013 06:53pm
  • DB Sharding is reasonably new to CB (I want to say around 0.8.0-ish?), so it could be my confusion or a result of the general pace of development in some parts of CB. I haven't spent much time on CB's IRC - sorry if I am leading you astray.

    I believe a config like this would work:

    {db_shards, [ 
          [ 
              {db_host, "localhost"}, 
              {db_adapter, mysql}, 
              {db_port, 3306}, 
              {db_username, "dbuser"}, 
              {db_password, "dbpass"}, 
              {db_database, "database"},
              {db_shard_id, shard_id_atom_mysql}, 
              {db_shard_models, [model_atom_1, model_atom_2,
                                 model_atom_3, etc]}
          ],
           [ 
              {db_host, "localhost"}, 
              {db_adapter, pgsql}, 
              {db_port, 5432}, 
              {db_username, "dbuser"}, 
              {db_password, "dbpass"}, 
              {db_database, "database"},
              {db_shard_id, shard_id_atom_pgsql}, 
              {db_shard_models, [model_atom_4, model_atom_5,
                                 model_atom_6, etc]}
          ]
      ]},
    

    It's unlikely that you'd want to use PostgreSQL + MySQL, but it gives you an idea of syntax. Remember that some of the database drivers require unique params ({riak_search, [{enabled, true}]} - for example).


About You

Email address is not published

Add to the Discussion

Search