Extending Catalyst Controllers

Our API is versioned. Any change made to the API requires a new version at some level or another.

/api/v1/customers
/api/v1.1/customers
/api/v1.1.1/customers

Additionally, some of the URLs may want to be aliased

/api/v1.0.0/customers

When I got to the code we had Catalyst controllers based on Catalyst::Controller::REST, which looked somewhat like this:

package OurApp::Controller::API::v1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };

sub index
    : Path('/api/v1/customer') 
    : Args(1)
    : ActionClass('REST')
{
    # ... fetch and stash customer
}

sub index_GET
    : Action
{
}

1;

In order to extend this API, well, I faffed around a bit. I needed to add a new v1.1 controller that had all the methods available to this v1 controller, plus a new one. It needed to be done quickly, and nothing really stood out as obvious to me at the time.

So I used detach.

package OurApp::Controller::API::v1_1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };

sub index
    : Path('/api/v1.1/customer') 
    : Args(1)
    : ActionClass('REST')
{ }

sub index_GET
    : Action
{
    my ($self, $c) = @_;
    $c->detach('/api/v1/customer/index');
}

1;

This had the effect of creating new paths under /api/v1.1/ that simply detached to their counterparts.

The problem with this particular controller is that in v1.0 it only had GET defined. That meant it only had index defined, and so the customer object itself was fetched in the index method, ready for index_GET. I needed a second method that also used the customer object: this meant I had to refactor the index methods to use a chained loader, which the new method could also use.

sub get_customer
    : Chained('/')
    : PathPart('api/v1.1/customer') 
    : CaptureArgs(1)
{
    # ... fetch and stash the customer
}

sub index
    : PathPart('')
    : Chained('get_customer')
    : Args(0)
    : ActionClass('REST')
{ }

sub index_GET
    : Action
{
    my ($self, $c) = @_;
    $c->detach('/api/v1.1/customer/index');
}

sub address
    : PathPart('address')
    : Chained('get_customer')
    : Args(0)
    : ActionClass('REST')
{}

sub address_GET
    : Action
{
    # ... get address from stashed customer
}

The argument that used to terminate the URL is now in the middle of the URL for the address: /api/v1.1/customer/$ID/address. So it's gone from : Args(1) on the index action to : CaptureArgs(1) on the get_customer action.

The problem now is that I can't use detach in v1.1.1, because we'd be detaching mid-chain.

I had¹ to use goto.

package OurApp::Controller::API::v1_1_1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };

sub get_customer
    : Chained('/')
    : PathPart('api/v1.1.1/customer') 
    : CaptureArgs(1)
{
    goto &OurApp::Controller::API::v1_1::Customer::get_customer;
}

#...
1;

This was fine, except I also introduced a validation method that was not an action; it was simply a method on the controller that validated customers for POST and PUT.

sub index_POST
    : Action
{
    my ($self, $c) = @_;
    my $data = $c->req->data;

    $self->_validate($c, $data);
}

sub _validate {
    # ...
}

In version 1.1.1, the only change was to augment validation; phone numbers were now constrained, where previously they were not.

It seemed like a ridiculous quantity of effort to clone the entire directory of controllers, change all the numbers to 1.1.1, and hack in a goto, just because I couldn't use Moose's after method modifier on _validate.

Why couldn't I? Because I couldn't use OurApp::Controller::API::v1_1::Customer as the base class for OurApp::Controller::API::v1_1_1::Customer.

Why? Because the paths were hard-coded in the Paths and PathParts!

This was the moment of clarity. That is not the correct way of specifying the paths.

To Every Controller, A Path

There is actually already a controller at every level of our API.

OurApp::Controller::API
OurApp::Controller::API::v1
OurApp::Controller::API::v1_1
OurApp::Controller::API::v1_1_1
OurApp::Controller::API::v1_1_1::Customer

This means we can add path information at every level. It's important to remember the controller namespace has nothing to do with Chained actions - The : Chained(path) and : PathPart(path) attributes can contain basically anything, allowing any path to be constructed from any controller.

In practice, this is a bad idea, because the first thing you want to know when you look at a path is how it's defined; and you don't want to have to pick apart the debug output when you could simply make assumptions based on a consistent association between controllers and paths.

But there is a way of associating the controller with the chained path, and that's by use of the path config setting and the : PathPrefix and : ChainedParent attributes. Both of these react to the current controller, meaning that if you subclass the controller, the result changes.

First I made the v1 controller have just the v1 path.

package OurApp::Controller::API::v1;
use Moose;
BEGIN { extends 'Catalyst::Controller'; };

__PACKAGE__->config
(
    path => 'v1',
);

sub api
    : ChainedParent
    : PathPrefix
    : CaptureArgs(0)
{}

1;

Then I gave the API controller the api path.

package OurApp::Controller::API;
use Moose;
BEGIN { extends 'Catalyst::Controller'; };

__PACKAGE__->config
(
    path => '/api',
);

sub api
    : Chained
    : PathPrefix
    : CaptureArgs(0)
{}

1;

This Tomato Is Not A Fruit

You may be wondering, why isn't ::v1 an extension of ::API itself? It's 100% to do with the number of path parts we need. The ::API controller defines a path => '/api' , while the ::API::v1 controller defines path => 'v1' . If the latter extended the former, it would inherit the methods rather than chaining them, i.e. v1 would override rather than extend /api.

So we have one controller per layer, but things in the same layer can inherit.

package OurApp::Controller::API::v1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };

__PACKAGE__->config
(
    path => 'customer',
);

sub index
    : Chained('../api')
    : PathPrefix
    : Args(1)
    : ActionClass('REST')
{}

sub index_GET {}

1;

package OurApp::Controller::API::v1_1;

use Moose;
BEGIN { extends 'OurApp::Controller::API::v1'; };

__PACKAGE__->config
(
    path => 'v1.1',
);

1;

The reason we can inherit is that everything we've done is relative.

• ChainedParent

This causes ::API::v1::api to be chained from ::API::v1::api, but when inherited, causes ::API::v1_1::api to be chained from ::API::v1_1::api.

• Chained('../api')

This causes ::API::v1::Customer::index to be chained from ::API::v1::api, but when we inherit it, the new ::API::v1_1::Customer::index will be chained from ::API::v1_1::api.

• PathPrefix

This causes these methods to have the PathPart of their controller's path_prefix. The most important example of this is in ::API::v1. Here, we see the api method configured with it:

sub api
    : ChainedParent
    : PathPrefix
    : CaptureArgs(0)
{}

This last is the central part of the whole deal. This means that the configuration path => 'v1' causes this chain to have the PathPart v1. When we inherit from this class, we simply redefine path, as we did in the v1.1 controller above:

__PACKAGE__->config( path => 'v1.1' );

The code above wasn't abbreviated. That was the entirety of the controller.

We can also create the relevant Customer controller in the same way:

package OurApp::Controller::API::v1_1::Customer;
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1::Customer'; };
1;

This is even shorter because we don't have to even change the path! All we need to do is establish that there is a controller called ::API::v1_1::Customer and the standard path stuff will take care of the rest.

Equally, you can alias the same version with the same trick:

package OurApp::Controller::API::v1_0;
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1'; };
__PACKAGE__->config( path => 'v1.0' );
1;

And of course the whole point of this is that now you can extend your API.

package OurApp::Controller::API::v1_1::Customer
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1::Customer'; };

sub index_PUT { }

sub _validate {}

1;

This is where I came in. Now I can extend v1.1 into v1.1.1 and use Moose's around or after to change the way _validate works only for v1.1.1, and thus I have extended my API in code as well as in principle.

CatalystX::AppBuilder

We're actually using CatalystX::AppBuilder. This makes subclassing the entire API tree even easier, because you can inject v1 controllers as v1.1 controllers.

after 'setup_components' => sub {
    my $class = shift;

    $class->add_paths(__PACKAGE__);

    CatalystX::InjectComponent->inject(
        into      => $class,
        component => 'OurApp::Controller::API',
        as        => 'Controller::API'
    );
    CatalystX::InjectComponent->inject(
        into      => $class,
        component => 'OurApp::Controller::API::v1',
        as        => 'Controller::API::v1'
    );
    CatalystX::InjectComponent->inject(
        into      => $class,
        component => 'OurApp::Controller::API::v1_1',
        as        => 'Controller::API::v1_1'
    );

    for my $version (qw/v1 v1_1/) {
        CatalystX::InjectComponent->inject(
            into      => $class,
            component => 'OurApp::Controller::API::' . $version . '::Customers',
            as        => 'Controller::API::' . $version . '::Customers'
        );

        for my $controller (qw/Addresses Products/) {
            CatalystX::InjectComponent->inject(
                into      => $class,
                component => 'OurApp::Controller::API::v1::' .  $controller, # sic!
                as        => 'Controller::API::' . $version . '::' .  $controller
            );
        }
    }
};

Now we've injected all controllers that weren't changed simply by using the v1 controller as both the v1 and the v1.1 controllers; and the Customer controller, which was subclassed, has had the v1.1 version added explicitly.

The only thing we can't get away with injecting with different names are subclassed controllers themselves. Obviously that includes the v1.1 Customer controller because that's the one with new functionality, but don't forget it is also necessary to have a v1_1 controller in the first place in order to override the path config of its parent.

We would also have to create subclasses if we wanted to alias v1 into v1.0 and v1.0.0. That is the limitation of this, and it's a few lines of boilerplate to do so; but it's considerably better than an entire suite of copy-pasted controllers using goto.

I expect there's a good way to perform this particular form of injection without CatalystX::AppBuilder, but I don't know it. Comments welcome.

¹ Chose.

CPAN installation order

At work we use Catalyst. Catalyst apps can be (should be?) built up from multiple modules, in the sense of distribution. This allows them to be modular, which is kind of why they're called modules.

That means each project is a directory full of directories, most of which represent Perl modules, and most of which depend on each other. In order to deploy we throw this list at cpanm (http://cpanmin.us) and let cpanm install them all.

This works by accident, because they're all installed already, and so module X depending on module Y is normally OK because Y will be updated during the process.

For a fresh installation, cpanm will fail to install many of them because their prerequisites are in the installation list:

$ cpanm X Y
--> Working on X
...
-> FAIL Installing the dependencies failed: 'Y' is not installed
--> Working on Y
...
-> OK
Successfully installed Y

Now Y is installed, but not X.

I wrote a script to reorder them. https://gist.github.com/Altreus/26c33421c36cc1eee68c

$ installation-order X Y
Y X

$ cpanm $(installation-order X Y)
--> Working on Y
...
-> OK
Successfully installed Y
--> Working on X
...
-> OK
Successfully installed X

This will use the same information that cpanm used in the first place to complain that Y was not installed; which is to say, if a dependency is missing, the original cpanm invocation would not have failed anyway.

Update

It is worth noting that cpanm can install from directories; and it will always try this if the module name starts with ./.

Therefore, X and Y above can be the result of a glob, so long as you include the ./ in the glob:

$ echo ./*
./Module1 ./Module2
$ installation-order ./*
./Module2 ./Module1

This also works with absolute paths.

Catalyst Models

A Catalyst model is simply a package in the MyApp::Model namespace.

$c->model('DBIC')

simply returns

"MyApp::Model::DBIC"

I recently spent some time at work trying to work out quite how Catalyst models work with relation to, well, everything else.

Our app structure is based on CatalystX::AppBuilder, and I needed to add a model to one of the components, in order to provide a caching layer in the right place.

The mistake I'd been making was that the Schema subclass is not the same thing as the model. Rather, the model is an interface into the Schema class. Essentially, I had one class too few.

You can determine that by creating a new Catalyst application and then running the helper script that creates a model from an existing schema. You get a class like this:

package MyApp::Model::FilmDB;

use strict;
use base 'Catalyst::Model::DBIC::Schema';

__PACKAGE__->config(
    schema_class => 'MyApp::Schema::FilmDB',

    connect_info => {
        dsn => 'dbi:mysql:filmdb',
        user => 'dbusername',
        password => 'dbpass',
    }
);

A Model class is created and it points to the Schema class, being your actual DBIC schema.

Once I'd realised the above rule it was easy enough to create MyApp::Extension::Model::DBIC to go alongside MyApp::Extension::Schema.

Further confusion arose with the configuration. There appeared to be no existing configuration that matched any of the extant classes in the application or its components. However, it was clear which was the DBIC model configuration because of the DSN.

I wanted to follow suit with the new module, which meant that some how I had to map the real name to the config name.

<Model::DBIC>
</Model::DBIC>

This makes sense; if I do $c->model('DBIC') I'll get "MyApp::Model::DBIC", and that'll be configured with the Model::DBIC part of the config.

What I'd missed here was that we were mixing CatalystX::AppBuilder with CatalystX::InjectComponent:

package MyApp::Extension;
use CatalystX::InjectComponent;

after 'setup_components' => sub {
    my $class = shift;

    ...

    CatalystX::InjectComponent->inject(
        into      => $class,
        component => __PACKAGE__ . '::Model::DBIC',
        as        => 'Model::DBIC',
    );
}

This was the missing part - the stuff inside the CatalystX::AppBuilder component was itself built up out of other components, aliasing their namespace-specific models so that $c->model would return the appropriate class.

Now, Model::DBIC refers to MyApp::Extension::Model::DBIC, which is an interface into MyApp::Extension::Schema.

User groups in Odoo 8

Odoo has a user group concept that, if you Google for errors, crops up all the time. Odd that when you first run Odoo, you can't assign users to groups.

The answer is you have to give the Administrator user the "Technical Features" feature in Usability. Navigate to Settings > Users, click Administrator, click Edit, check the relevant box, click Save, and finally refresh.

If you Google for it, there's hardly any information on the subject. However, Odoo is quite happy to occasionally tell you what groups you need to be a part of in order to access something.

User groups are access control, so it's common that you'd want to create levels of access and assign the user to them. I first discovered an issue with this when trying the Project Management module - trying which was the entire point of me running Odoo 8 in the first place. (I can't reproduce the problem now that it's a new year. Maybe Odoo's NYR is to be less whiny.)

You can run a Docker container with Odoo 8 in it from the tinyerp/odoo-docker github repo; either the Debian or the Ubuntu version should work fine.¹

¹ I recommend the Debian version, since Ubuntu is just Debian with extra, irrelevant stuff bundled in, making it not entirely useful to have an Ubuntu version in the first place. Licensing is probably involved.