tag:blogger.com,1999:blog-48576735669674293852024-03-13T21:02:10.808+00:00Pod::CatsAltreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.comBlogger52125tag:blogger.com,1999:blog-4857673566967429385.post-53361223626758162242017-02-10T13:16:00.001+00:002017-02-10T13:16:03.162+00:00Bootstrapping Perl<p>This blog post shows a simple, <em>hands-off</em>, automated way to get yourself a Perl environment in user land. If you already know enough about all of this to do it the hard way, and you prefer that, then this post is not aimed at you.<br />
<p>Here's what we are going to achieve:<br />
<ul> <li>Set up a Perl 5.24 installation<li>Set up your environment so you can install modules<li>Set up your project so you can install its dependencies</ul><p>These are the things people seem to struggle with a lot, and the instructions are piecemeal all over the internet. Here they are all standing in a row.
<h2 style="">Perlbrew your Perl 5.24</h2><p>As this blog post becomes older, that number will get bigger, so make sure to alter it if you copy this from the future.
<p>Do this as root:
<p><b>Debian</b>
<pre>apt-get install perlbrew</pre><p><b>FreeBSD</b>
<pre>fetch -o- https://install.perlbrew.pl | sh</pre><p><b>Whatever else</b>
<pre>curl -L https://install.perlbrew.pl | bash</pre><p><b>Windows</b>
<p>Haha, yeah, right.
<p>Once you've installed perlbrew, log out of root and init it as your user. Then install a perl. <b>This will take a while.</b>
<pre>perlbrew init
perlbrew install perl-5.24.0</pre><p>There, you now have a Perl 5.24.0 installation in your home folder. <code>which perl</code> will still say <code>/usr/bin/perl</code> so you can change that:
<pre>perlbrew switch perl-5.24.0</pre><p>It will have already told you that you need to alter your <code>.bashrc</code> or suchlike, with something like this:
<pre>source $HOME/perl5/perlbrew/etc/bashrc</pre><p>You should do that.
<p>Perlbrew does other stuff - see <a href="https://perlbrew.pl">https://perlbrew.pl</a> for details.
<h2 style=""><code>cpanm</code></h2><p>You want to be able to install modules against your new perl.
<p><b>You will have to reinstall modules under every perl you have</b> if you want to use the same modules under different versions. This is because of reasons.<sup><a class="footnote-from" href="#footnote-8b6256b13aadbfed2978db25351e865d0109d5c2-1" name="fn-8b6256b13aadbfed2978db25351e865d0109d5c2-1">1</a></sup>
<pre>perlbrew install-cpanm</pre><p>Now you can use <a href="https://metacpan.org/module/App::cpanminus"><code>cpanm</code></a> to install modules. If you install a new Perl with perlbrew, you will have to
<pre>perlbrew switch $your_new_perl
perlbrew install-cpanm</pre><p>All over again. If you're dealing with multiple Perl versions for a reason, you've probably already read the docs enough that you know which commands to use.
<h2 style=""><code>cpanfile</code></h2><p>A <code>cpanfile</code> is a file in your project that lists the dependencies it requires. The purpose of this file is for when you are developing a project, and thus you haven't actually installed it. It looks like this.
<pre>requires "Moose";
requires "DBIx::Class" => "0.082840";
requires "perl" => "5.24";
test_requires "Test::More";</pre><p>You use it like this
<pre>cpanm --installdeps .</pre><p>The <code>.</code> refers to the current directory, of course, so you run this from a place that has a <code>cpanfile</code> in it.
<p>The full syntax is <a href="https://metacpan.org/pod/cpanfile">on CPAN</a>.
<h3 style="">Purpose of <code>cpanfile</code></h3><p>A "project" here refers to basically anything you might put on CPAN - a distribution. It might be a module, or just some scripts, or a whole suite of both of those things.
<p>The point is it's a unit, and it has dependencies, and you can't run the code without satisfying those dependencies.
<p>If you install this distribution with <code>cpanm</code> then it will automatically install the dependencies because the author set up the makefile correctly so that <code>cpanm</code> knew what the dependencies were. <code>cpanm</code> also puts the modules in <code>$PERL5LIB</code> and the scripts in <code>$PATH</code> so that you can use them.
<p>If you have the source code, either you <em>are</em> the author, or at least you're a contributor; you don't want to run the makefile just to install the dependencies, because this will install the development version of the module too. Nor do you want to require your contributors to install the <a href="http://dzil.org/">whole of dzil</a> just to contribute to your module. So, you provide a <code>cpanfile</code> that lists the dependencies they require to run or develop your module or scripts.
<p class="footnote"><sup><a class="footnote-to" href="#fn-8b6256b13aadbfed2978db25351e865d0109d5c2-1" name="footnote-8b6256b13aadbfed2978db25351e865d0109d5c2-1">1</a> The primary reason is that every Perl version has a slightly different set of development headers, so any modules written in C will be incompatible. It's too much effort to separate them and disk space is cheap; so we just keep separate libraries and avoid the problem.</sup>
Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com2tag:blogger.com,1999:blog-4857673566967429385.post-38932172736904197052015-07-29T10:53:00.001+01:002015-07-30T12:41:33.131+01:00Extending Catalyst Controllers<p>Our API is versioned. Any change made to the API requires a new version at some level or another.</p><pre>/api/v1/customers
/api/v1.1/customers
/api/v1.1.1/customers</pre><p>Additionally, some of the URLs may want to be aliased</p><pre>/api/v1.0.0/customers</pre><p>When I got to the code we had Catalyst controllers based on <a href="https://metacpan.org/module/Catalyst::Controller::REST">Catalyst::Controller::REST</a>, which looked somewhat like this:</p><pre>package OurApp::Controller::API::v1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };
sub index
: Path('/api/v1/customer')
: Args(1)
: ActionClass('REST')
{
# ... fetch and stash customer
}
sub index_GET
: Action
{
}
1;</pre><p>In order to extend this API, well, I faffed around a bit. I needed to add a new v1.1 controller that had all the methods available to this v1 controller, plus a new one. It needed to be done quickly, and nothing really stood out as obvious to me at the time.</p><p>So I used <code>detach</code>.</p><pre>package OurApp::Controller::API::v1_1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };
sub index
: Path('/api/v1.1/customer')
: Args(1)
: ActionClass('REST')
{ }
sub index_GET
: Action
{
my ($self, $c) = @_;
$c->detach('/api/v1/customer/index');
}
1;</pre><p>This had the effect of creating new paths under <code>/api/v1.1/</code> that simply detached to their counterparts. </p><p>The problem with this particular controller is that in v1.0 it only had GET defined. That meant it only had <code>index</code> defined, and so the customer object itself was fetched in the <code>index</code> method, ready for <code>index_GET</code>. I needed a second method that also used the customer object: this meant I had to refactor the <code>index</code> methods to use a chained loader, which the new method could also use.</p><pre>sub get_customer
: Chained('/')
: PathPart('api/v1.1/customer')
: CaptureArgs(1)
{
# ... fetch and stash the customer
}
sub index
: PathPart('')
: Chained('get_customer')
: Args(0)
: ActionClass('REST')
{ }
sub index_GET
: Action
{
my ($self, $c) = @_;
$c->detach('/api/v1.1/customer/index');
}
sub address
: PathPart('address')
: Chained('get_customer')
: Args(0)
: ActionClass('REST')
{}
sub address_GET
: Action
{
# ... get address from stashed customer
}</pre><p>The argument that used to terminate the URL is now in the middle of the URL for the address: <code>/api/v1.1/customer/$ID/address</code>. So it's gone from <code>: Args(1)</code> on the <code>index</code> action to <code>: CaptureArgs(1)</code> on the <code>get_customer</code> action.</p><p>The problem now is that I can't use <code>detach</code> in v1.1.1, because we'd be detaching mid-chain.</p><p>I had<a href="#footnote-42f88e4c573e8adcf4397583528cc5b391f6ad37-1" name="fn-42f88e4c573e8adcf4397583528cc5b391f6ad37-1"><sup>1</sup></a> to use <code>goto</code>.</p><pre>package OurApp::Controller::API::v1_1_1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };
sub get_customer
: Chained('/')
: PathPart('api/v1.1.1/customer')
: CaptureArgs(1)
{
goto &OurApp::Controller::API::v1_1::Customer::get_customer;
}
#...
1;</pre><p>This was fine, except I also introduced a validation method that was <em>not</em> an action; it was simply a method on the controller that validated customers for POST and PUT.</p><pre>sub index_POST
: Action
{
my ($self, $c) = @_;
my $data = $c->req->data;
$self->_validate($c, $data);
}
sub _validate {
# ...
}</pre><p>In version 1.1.1, the only change was to augment validation; phone numbers were now constrained, where previously they were not.</p><p>It seemed like a ridiculous quantity of effort to clone the entire directory of controllers, change all the numbers to 1.1.1, and hack in a <code>goto</code>, just because I couldn't use Moose's <code>after</code> method modifier on <code>_validate</code>.</p><p>Why couldn't I? Because I couldn't use <code>OurApp::Controller::API::v1_1::Customer</code> as the base class for <code>OurApp::Controller::API::v1_1_1::Customer</code>.</p><p>Why? Because the paths were hard-coded in the Paths and PathParts!</p><p>This was the moment of clarity. That is not the correct way of specifying the paths.</p><h2>To Every Controller, A Path</h2><p>There is actually already a controller at every level of our API.</p><pre>OurApp::Controller::API
OurApp::Controller::API::v1
OurApp::Controller::API::v1_1
OurApp::Controller::API::v1_1_1
OurApp::Controller::API::v1_1_1::Customer</pre><p>This means we can add path information at every level. It's important to remember the controller namespace has nothing to do with <code>Chained</code> actions - The <code>: Chained(path)</code> and <code>: PathPart(path)</code> attributes can contain basically anything, allowing any path to be constructed from any controller.</p><p>In practice, this is a bad idea, because the first thing you want to know when you look at a path is how it's defined; and you don't want to have to pick apart the debug output when you could simply make assumptions based on a consistent association between controllers and paths.</p><p>But there is a way of associating the controller with the chained path, and that's by use of the <code>path</code> config setting and the <code>: PathPrefix</code> and <code>: ChainedParent</code> attributes. Both of these react to the <em>current controller</em>, meaning that if you subclass the controller, the result changes.</p><p>First I made the v1 controller have just the <code>v1</code> path.</p><pre>package OurApp::Controller::API::v1;
use Moose;
BEGIN { extends 'Catalyst::Controller'; };
__PACKAGE__->config
(
path => 'v1',
);
sub api
: ChainedParent
: PathPrefix
: CaptureArgs(0)
{}
1;</pre><p>Then I gave the API controller the <code>api</code> path.</p><pre>package OurApp::Controller::API;
use Moose;
BEGIN { extends 'Catalyst::Controller'; };
__PACKAGE__->config
(
path => '/api',
);
sub api
: Chained
: PathPrefix
: CaptureArgs(0)
{}
1;</pre><h2>This Tomato Is Not A Fruit</h2><p>You may be wondering, why isn't <code>::v1</code> an extension of <code>::API</code> itself? It's 100% to do with the number of path parts we need. The <code>::API</code> controller defines a <code>path => '/api' </code>, while the <code>::API::v1</code> controller defines <code>path => 'v1' </code>. If the latter extended the former, it would <em>inherit</em> the methods rather than <code>chaining</code> them, i.e. <code>v1</code> would <em>override</em> rather than <em>extend </em><code>/api</code>.</p><p>So we have one controller per layer, but things in the same layer can inherit.</p><pre>package OurApp::Controller::API::v1::Customer;
use Moose;
BEGIN { extends 'Catalyst::Controller::REST'; };
__PACKAGE__->config
(
path => 'customer',
);
sub index
: Chained('../api')
: PathPrefix
: Args(1)
: ActionClass('REST')
{}
sub index_GET {}
1;</pre><pre>package OurApp::Controller::API::v1_1;
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1'; };
__PACKAGE__->config
(
path => 'v1.1',
);
1;</pre><p>The reason we can inherit is that everything we've done is <em>relative</em>.</p><ul><li><code>ChainedParent</code></li>
<p>This causes <code>::API::v1::api</code> to be chained from <code>::API::v1::api</code>, but when inherited, causes <code>::API::v1_1::api</code> to be chained from <code>::API::v1_1::api</code>.</p><li><code>Chained('../api')</code></li>
<p>This causes <code>::API::v1::Customer::index</code> to be chained from <code>::API::v1::api</code>, but when we inherit it, the new <code>::API::v1_1::Customer::index</code> will be chained from <code>::API::v1_1::api</code>.</p><li><code>PathPrefix</code></li>
<p>This causes these methods to have the PathPart of their controller's <code>path_prefix</code>. The most important example of this is in <code>::API::v1</code>. Here, we see the <code>api</code> method configured with it:</p><pre>sub api
: ChainedParent
: PathPrefix
: CaptureArgs(0)
{}</pre></ul><p>This last is the central part of the whole deal. This means that the configuration <code>path => 'v1' </code> causes <em>this</em> chain to have the PathPart <code>v1</code>. When we inherit from this class, we simply redefine <code>path</code>, as we did in the v1.1 controller above:</p><pre>__PACKAGE__->config( path => 'v1.1' );</pre><p>The code above wasn't abbreviated. That was the entirety of the controller.</p><p>We can also create the relevant Customer controller in the same way:</p><pre>package OurApp::Controller::API::v1_1::Customer;
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1::Customer'; };
1;</pre><p>This is even shorter because we don't have to even change the <code>path</code>! All we need to do is establish that there <em>is</em> a controller called <code>::API::v1_1::Customer</code> and the standard path stuff will take care of the rest.</p><p>Equally, you can alias the <em>same</em> version with the same trick:</p><pre>package OurApp::Controller::API::v1_0;
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1'; };
__PACKAGE__->config( path => 'v1.0' );
1;</pre><p>And of course the whole point of this is that now you can extend your API.</p><pre>package OurApp::Controller::API::v1_1::Customer
use Moose;
BEGIN { extends 'OurApp::Controller::API::v1::Customer'; };
sub index_PUT { }
sub _validate {}
1;</pre><p>This is where I came in. Now I can extend v1.1 into v1.1.1 and use Moose's <code>around</code> or <code>after</code> to change the way <code>_validate</code> works <em>only for v1.1.1</em>, and thus I have extended my API in code as well as in principle.</p><h2>CatalystX::AppBuilder</h2><p>We're actually using <a href="https://metacpan.org/module/CatalystX::AppBuilder">CatalystX::AppBuilder</a>. This makes subclassing the entire API tree even easier, because you can inject v1 controllers as v1.1 controllers.</p><pre>after 'setup_components' => sub {
my $class = shift;
$class->add_paths(__PACKAGE__);
CatalystX::InjectComponent->inject(
into => $class,
component => 'OurApp::Controller::API',
as => 'Controller::API'
);
CatalystX::InjectComponent->inject(
into => $class,
component => 'OurApp::Controller::API::v1',
as => 'Controller::API::v1'
);
CatalystX::InjectComponent->inject(
into => $class,
component => 'OurApp::Controller::API::v1_1',
as => 'Controller::API::v1_1'
);
for my $version (qw/v1 v1_1/) {
CatalystX::InjectComponent->inject(
into => $class,
component => 'OurApp::Controller::API::' . $version . '::Customers',
as => 'Controller::API::' . $version . '::Customers'
);
for my $controller (qw/Addresses Products/) {
CatalystX::InjectComponent->inject(
into => $class,
component => 'OurApp::Controller::API::v1::' . $controller, # sic!
as => 'Controller::API::' . $version . '::' . $controller
);
}
}
};</pre><p>Now we've injected all controllers that weren't changed simply by using the v1 controller as both the v1 and the v1.1 controllers; and the Customer controller, which was subclassed, has had the v1.1 version added explicitly.</p><p>The only thing we can't get away with injecting with different names are subclassed controllers themselves. Obviously that includes the v1.1 Customer controller because that's the one with new functionality, but don't forget it is also necessary to have a <code>v1_1</code> controller in the first place in order to override the <code>path</code> config of its parent.</p><p>We would also have to create subclasses if we wanted to alias <code>v1</code> into <code>v1.0</code> and <code>v1.0.0</code>. That is the limitation of this, and it's a few lines of boilerplate to do so; but it's considerably better than an entire suite of copy-pasted controllers using <code>goto</code>.</p><p>I expect there's a good way to perform this particular form of injection <em>without</em> CatalystX::AppBuilder, but I don't know it. Comments welcome.</p><p class="footnote"><a href="#fn-42f88e4c573e8adcf4397583528cc5b391f6ad37-1" name="footnote-42f88e4c573e8adcf4397583528cc5b391f6ad37-1"><sup>1</sup></a> Chose.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-82044732204874482232015-05-27T12:32:00.000+01:002015-09-22T13:22:47.634+01:00CPAN installation order<p>At work we use Catalyst. Catalyst apps can be (should be?) built up from multiple modules, in the sense of distribution. This allows them to be modular, which is kind of why they're called modules.</p><p>That means each project is a directory full of directories, most of which represent Perl modules, and most of which depend on each other. In order to deploy we throw this list at <code>cpanm</code> (<a href="http://cpanmin.us">http://cpanmin.us</a>) and let <code>cpanm</code> install them all. </p><p>This works by accident, because they're all installed already, and so module X depending on module Y is normally OK because Y will be updated during the process.</p><p>For a fresh installation, <code>cpanm</code> will fail to install many of them because their prerequisites are in the installation list:</p><pre>$ cpanm X Y
--> Working on X
...
-> FAIL Installing the dependencies failed: 'Y' is not installed
--> Working on Y
...
-> OK
Successfully installed Y</pre><p>Now Y is installed, but not X.</p><p>I wrote a script to reorder them. <a href="https://gist.github.com/Altreus/26c33421c36cc1eee68c">https://gist.github.com/Altreus/26c33421c36cc1eee68c</a></p><pre>$ installation-order X Y
Y X
$ cpanm $(installation-order X Y)
--> Working on Y
...
-> OK
Successfully installed Y
--> Working on X
...
-> OK
Successfully installed X</pre><p>This will use the same information that <code>cpanm</code> used in the first place to complain that Y was not installed; which is to say, if a dependency is missing, the original <code>cpanm</code> invocation would not have failed anyway.</p><h3>Update</h3><p>It is worth noting that cpanm can install from directories; and it will always try this if the module name starts with <code>./</code>.</p><p>Therefore, X and Y above can be the result of a glob, so long as you include the <code>./</code> in the glob:</p><pre>$ echo ./*
./Module1 ./Module2
$ installation-order ./*
./Module2 ./Module1</pre><p>This also works with absolute paths.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-4117268465581446052015-04-14T17:49:00.000+01:002015-04-14T17:49:00.772+01:00Catalyst Models<p>A Catalyst model is simply a package in the MyApp::Model namespace.</p><pre>$c->model('DBIC')</pre><p>simply returns</p><pre>"MyApp::Model::DBIC"</pre><p>I recently spent some time at work trying to work out quite how Catalyst models work with relation to, well, everything else.</p><p>Our app structure is based on CatalystX::AppBuilder, and I needed to add a model to one of the components, in order to provide a caching layer in the right place.</p><p>The mistake I'd been making was that the Schema subclass is <em>not</em> the same thing as the model. Rather, the model is an interface <em>into</em> the Schema class. Essentially, I had one class too few.</p><p>You can determine that by creating a new Catalyst application and then running the helper script that creates a model from an existing schema. You get a class like this:</p><pre>package MyApp::Model::FilmDB;
use strict;
use base 'Catalyst::Model::DBIC::Schema';
__PACKAGE__->config(
schema_class => 'MyApp::Schema::FilmDB',
connect_info => {
dsn => 'dbi:mysql:filmdb',
user => 'dbusername',
password => 'dbpass',
}
);</pre><p>A <code>Model</code> class is created and it points to the <code>Schema</code> class, being your actual DBIC schema.</p><p>Once I'd realised the above rule it was easy enough to create <code>MyApp::Extension::Model::DBIC</code> to go alongside <code>MyApp::Extension::Schema</code>.</p><p>Further confusion arose with the configuration. There appeared to be no existing configuration that matched any of the extant classes in the application or its components. However, it was clear which was the DBIC model configuration because of the DSN.</p><p>I wanted to follow suit with the new module, which meant that some how I had to map the real name to the config name.</p><pre><Model::DBIC>
</Model::DBIC></pre><p>This makes sense; if I do <code>$c->model('DBIC')</code> I'll get <code>"MyApp::Model::DBIC"</code>, and that'll be configured with the <code>Model::DBIC</code> part of the config.</p><p>What I'd missed here was that we were mixing CatalystX::AppBuilder with CatalystX::InjectComponent:</p><pre>package MyApp::Extension;
use CatalystX::InjectComponent;
after 'setup_components' => sub {
my $class = shift;
...
CatalystX::InjectComponent->inject(
into => $class,
component => __PACKAGE__ . '::Model::DBIC',
as => 'Model::DBIC',
);
}</pre><p>This was the missing part - the stuff <em>inside</em> the CatalystX::AppBuilder component was itself built up out of other components, aliasing their namespace-specific models so that <code>$c->model</code> would return the appropriate class.</p><p>Now, <code>Model::DBIC</code> refers to <code>MyApp::Extension::Model::DBIC</code>, which is an interface into <code>MyApp::Extension::Schema</code>.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-2777340527836220432015-01-05T16:43:00.000+00:002015-01-05T16:43:12.229+00:00User groups in Odoo 8<p>Odoo has a user group concept that, if you Google for errors, crops up all the time. Odd that when you first run Odoo, you can't assign users to groups.</p><p>The answer is you have to give the Administrator user the "Technical Features" feature in Usability. Navigate to Settings > Users, click Administrator, click Edit, check the relevant box, click Save, and finally refresh.</p><p>If you <a href="https://www.google.co.uk/search?q=odoo+8+groups">Google for it</a>, there's hardly any information on the subject. However, Odoo is quite happy to occasionally tell you what groups you need to be a part of in order to access something.</p><p>User groups are access control, so it's common that you'd want to create levels of access and assign the user to them. I first discovered an issue with this when trying the Project Management module - trying which was the entire point of me running Odoo 8 in the first place. (I can't reproduce the problem now that it's a new year. Maybe Odoo's NYR is to be less whiny.)</p><p>You can run a <a href="https://www.docker.com/">Docker</a> container with Odoo 8 in it from the <a href="https://github.com/tinyerp/odoo-docker">tinyerp/odoo-docker</a> github repo; either the Debian or the Ubuntu version should work fine.<a href="#footnote-b22d9735a4d3c5b0e15d0cd43df57e42865ba45e-1" name="fn-b22d9735a4d3c5b0e15d0cd43df57e42865ba45e-1"><sup>1</sup></a></p><p class="footnote"><a href="#fn-b22d9735a4d3c5b0e15d0cd43df57e42865ba45e-1" name="footnote-b22d9735a4d3c5b0e15d0cd43df57e42865ba45e-1"><sup>1</sup></a> I recommend the Debian version, since Ubuntu is just Debian with extra, irrelevant stuff bundled in, making it not entirely useful to have an Ubuntu version in the first place. Licensing is probably involved.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-8432470464332373422014-12-22T23:24:00.000+00:002014-12-22T23:24:37.798+00:00Day 22 - The nth Day Of Christmas<p>How many presents were given, in total, on the 12th day of Christmas, by the so-called "true love"? How many for the <em>n</em>th day?</p><p>For each day we know that each other day was done again, so we have a shape like this:</p><pre>1
12
123
1234
...</pre><p>Each column is as tall as the number of rows, and the number of rows is 12.</p><p>This means the 1 column is 12 tall, the 2 column 11, and so on.</p><p>This is 12 * 1 + 11 * 2 + 10 * 3 ...</p><p>That's boring. That's not what computers <em>or</em> maths are for. Let's generalise.</p><p>We can see that each section of the summed sequence follows a pattern of <em>x * y</em>, where <em>x + y = 13</em>. </p><p>It is common, when analysing sequences, to forget that the order matters, and the row number can be used as a variable. If we call that variable <em>i</em> then each section is <em>(13 - i) * i</em>, and the total is the sum over <em>1, 12</em>.</p><pre> 12
Σ (13 - i) * i
i=1</pre><p>13 is suspiciously close to 12. What happens if we do this?</p><pre> 12
Σ (12 + 1 - i) * i
i=1</pre><p>And then replace the <em>12</em> with our <em>n</em> to answer "What about the <em>n</em>th day?"</p><pre> n
Σ (n + 1 - i) * i
i=1</pre><p>Does it work? Let's Perl it up. Each value of <code>(n + 1 - i) * i</code> can be produced by a <code>map</code> over the range <code>1..$n</code>, using <code>$_</code> in place of <em>i</em>, since that's exactly what it is in this case.</p><pre>sum0 map { $_ * ($n + 1 - $_) } 1 .. $n</pre><p><code>sum0</code> comes from <a href="http://p3rl.org/List::Util">List::Util</a>, and does a standard <code>sum</code>, except the list is assumed to start with zero in case the list is empty - this just avoids pesky warnings.</p><p>Try it. Using <code>$ARGV[0]</code> for <code>$n</code> we can give our <em>n</em> on the command line:</p><pre>perl -MList::Util=sum0 -E'say sum0 map { $_ * ($ARGV[0] + 1 - $_) } 1 .. $ARGV[0]' 12</pre><p>Vary the 12 to solve for different values of <em>n</em>.</p><p>The answer, incidentally, is 364.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-23924754591266041612014-12-22T13:39:00.000+00:002014-12-22T13:39:20.145+00:00Day 18: The URI<p>I've talked a lot about this resource-first way of dealing with the web, and really the internet in general, but it isn't a tool that fits all things. For instance, today I was looking at the point-of-sale module in <a href="http://odoo.com">Odoo</a>, which is essentially an HTML representation of the index resource of the products in the system, but is actually more complicated than that, because it includes that resource, a numeric input box, the bill of items so far, a search box, and a few other twiddly bits to improve the cashier's use of the system. Plus, it is designed with tablets in mind.</p><p>This is quite different from the list of products you get when you look for the list of products in Odoo itself.</p><p>However, we must construct a URI that refers to this view of the data if we're to be able to access that view of the data in the first place. That means that we somehow have to shoehorn this not-a-resource idea into the everything-is-a-resource idea.</p><p>Today I'm going to deconstruct the URI and explain how each part can be used, in order to avoid too much in the way of special behaviour. Ideally we'd like every resource to be represented by a single URI, but that's clearly not going to work.</p><p>Allow me to state up front that I consider Odoo's URI scheme to be utterly shocking. But it appears to be a legacy from back in the old days when more people made web things than really understood what URIs were for.</p><h2>The URI</h2><p>The URI is made up of several parts. Here is what I consider to be the simplest URL that contains all common parts<a href="#footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-1" name="fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-1"><sup>1</sup></a>:</p><pre>http://www.example.com:8080/resource/id?query=data#part-of-document
|_____|___|_______|___|____|________|__|__________|_______________|
1 2 3 4 5 6 7 8 9</pre><ul><li>1. Schema</li>
<li>2. Subdomain</li>
<li>3. SLD<a href="#footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-2" name="fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-2"><sup>2</sup></a></li>
<li>4. TLD</li>
<li>5. Port</li>
<li>6. Resource (type) name</li>
<li>7. Resource (instance) identifier</li>
<li>8. Query string</li>
<li>9. URI fragment</li>
</ul><p>Together, <em>2</em>, <em>3</em> and <em>4</em> comprise the <em>hostname</em>; <em>6</em> and <em>7</em> are the <em>path</em>.</p><h2>Breaking down the URI</h2><h3>Schema</h3><p>The schema is the first place where you restrict yourself. Often referred to as <em>protocol</em>, the schema usually determines how the URI should be used. In this example <em>http</em> is the assumed protocol by which web requests are made. The <em>http</em> schema tells the client to use the HTTP protocol to make the request.</p><p>This is very useful because it means we can immediately assume a large quantity of knowledge about the system that we wouldn't have without the schema. Particularly useful is that we know what sort of programs can be used to actually access this URL<a href="#footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-3" name="fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-3"><sup>3</sup></a>. This is, if you think about it, what the word protocol means: it is those things that are assumed to be the case, given a certain situation. When we all follow protocol, we don't need to explain why we're doing what we're doing.</p><p>Mostly we come across URLs specifying the HTTP schema; in fact, it's assumed, in many cases, that a URI with no schema is an HTTP URL, because if you click on it, it opens up in your browser. However, some places have started using their own schemata, such as the <code>spotify:</code> schema, which opens URLs in the Spotify client, or the <code>steam:</code> schema, which opens things with Steam.</p><p>It's worth noting that the entire hostname can also be omitted from a URI, but this usually means you get <em>three</em> slashes, not two. This is commonly seen with the <code>file</code> protocol, such as <code>file:///home/user/documents/example.html</code>; where the third <code>/</code> is actually part of the path. For this reason it can be observed that the <code>steam:</code> schema does not quite follow the normal URI standards, since the part immediately following the schema is an action - arguably a resource - and not a hostname.</p><p>By inventing our own schemata like this we can create entire applications with a new way of communicating, but we're focusing on the web here, which means we're going to use HTTP(S), like it or lump it.</p><h3>Subdomain</h3><p>The term "subdomain" is a bit of a colloquialism. Each section of the hostname is a subdomain for the part to the right. The host name is a hierarchy with, in this case, <code>com</code> at the top. We usually call this part the "subdomain" because it's the first subdivision that is really relevent to a human.</p><p>When we have a subdivided subdomain we sort of stop talking about them and start mumbling and saying "that bit" and pointing.</p><p>The subdomain is a tool we can use to do many things. Traditionally the web is in the <code>www</code> subdomain, but the <code>http</code> protocol is usually sufficient to assume web, these days. However, that's starting to change, as we start to send non-web things over HTTP. These non-web things are, e.g., the API, or the <a href="http://en.wikipedia.org/wiki/Content_delivery_network">CDN</a>.</p><p>Really consider using an <code>api</code> subdomain for your API. You'll find that if you have an <code>api</code> and a <code>www</code>, then your website can have, in the majority, the exact same URI structure as the API. This is more often the case than it appears to be, because people don't tend to think of their web pages as representing a resource in HTML format.</p><h3>Domain</h3><p>The SDL is the part of the domain that really, to a human, represents where the site is. This is usually your company or organisation name, or some other thing whose entire purpose is to say what this whole web site is about.</p><p>You can install a system under multiple domains and thus they would all have the exact same URI scheme, except that, because they're in different places, the records that you get would be different.</p><p>Because <code>yoursite.com/user/1</code> is not the same person as <code>mysite.com/user/1</code>, except by coincidence.</p><p>I've lumped the TLD in here too, because the TLD is, to most people, part of your domain name - which is why we call the subdomain the subdomain regardless of where it appears on the actual hostname.</p><h3>Port</h3><p>When designing URI schemes it's helpful to drink a lot of port, for inspiration.</p><p>Commonly there are alternative services associated with your website, meaning they're on the same domain, and you can't use the subdomain because these other services need <code>api</code> and <code>www</code> subdomains of their own.</p><p>One trick is to mount these services under a part of the path, and consider them a big resource with sub-resources; but easier is to install them on a different port.</p><p>For example, your <a href="http://www.elasticsearch.org/">Elasticsearch</a> instance - which communicates entirely via HTTP - can be running on the same hostname as your website, but a different port. Elasticsearch's default port is 9200, going up to 9300 as you add instances on the same machine.</p><h3>Resource name</h3><p>The first part of the path of the URL I'm calling the resource name. That's because this is where the actual resource you're requesting starts. Everything before the path is defining whose resource you are asking for, but once the path starts you're starting to get a handle on the actual information.</p><p>The resource name, when requested, can have multiple behaviours, depending on the purpose of the resource, but common is simply to be an index of all the items of that type. Since that can be cumbersome, it is perfectly legitimate to both paginate this list and summarise the entries. That sort of stuff is well out of scope of this article, though.</p><p>Other uses of the first part of the path are organisational, and may be handled better as a subdomain. For example, having an <code>api</code> part of the path here is not as useful as it would be to have an API subdomain, because if the paths to the resources can be consistent then we don't have to ask questions about what they should be.</p><pre>https://www.example.com/resource
https://api.example.com/resource</pre><p>Other times, you may want to use a different port. For example, if the web stuff is on port 80 then the administration part could be on port 8080. This also allows you to control access to the different parts of the site at the kernel level, using routing rather than soft authentication.</p><pre>https://www.example.com/admin
https://www.example.com:8080</pre><p>Doing this also means that it's harder to guess the correct path to the admin area, since you can use an obscure port. Denying access based on IP rules means you'd never report to unauthorised users when they guessed right in the first place.</p><p>But really, there's no exact reason why you would or would not add parts of the path to the URL in order to divide it up into separate logical zones. This can certainly help with human comprehension of the purpose of your URL. Sometimes you may even want to provide dummy paths - paths that refer to the same resource as other paths, but assist with conceptual compartmentalisation by having different subpaths.</p><pre>https://www.example.com/shop/product/1
https://www.example.com/blog/post/1</pre><p>In these examples, the first part of the path could be omitted, provided that <code>post</code> is always the blog post and <code>product</code> is always a shop product. Consider also that you could still use subdomains for these.</p><pre>https://api.shop.example.com/product/1</pre><p>The important part would be to ensure that your uses are consistent. Always have each part of the URL refer to the same logical division of your resource structure.</p><h3>Item ID</h3><p>Once you've decided at which point of the path to put the resource type, you should probably put the next part as an optional ID field.</p><p><b>The combination of a resource name and an item ID should be entirely sufficient to retrieve all the information about that specific instance of that type.</b></p><p>This is a reasonably central principle to the resource-first model of your system - all your things have a <em>type</em> and an <em>ID</em> and that's all you need to provide to retrieve it, or at least a representation of it. Everything else is your organisational whimsy and the system really shouldn't have to know.</p><p>More formally than dismissing it as whimsy, I should point out that even the type names and shapes can change, and that's difficult enough to deal with. Every level of organisation you add on top of this is another changeable shape of the system that at some point you're going to have to adapt. The fewer of those you have, the better.</p><p>The actual format of your identifier is up to you, but there's really nothing else you can put after the resource name that is relevant at this point.</p><h3>Query string</h3><p>If I catch you using a query string to tell a dynamic resource to load a specific other resource I will murder you in your sleep.</p><pre>https://example.com/index.php?type=resource&id=1234</pre><p>Seriously, this sort of crap is all over the internet. Yes, it's usually PHP.</p><p>You are using a URI - at least put the resource identifier in the resource identifier.</p><p>It is important to note that the query string is not the same thing as the "GET parameters". A query string does not have to be in the format <code>key=value&key=value</code> - the web server passes the query string straight to the app, and it is the application that decodes it in its own way. It is common to use the <code>key=value&key=value</code> structure but not required.</p><p>The query string's most obvious purpose is to pass a query to a resource that expects one, or that at least accepts one. Often the index resources will allow for some sort of search or filter functionality, and if that's not the case then special resources designed to search and filter - and possibly concatenate - other resources will accept search parameters.</p><p>Further specialisation of resources would not even use the KVP format of "GET parameters", and simply take the query string as instruction. These types of resource are drifting away from the "object" type of resource and moving towards "function" resources, which are a separate discussion.</p><p>The thing about the query string is that it is usually only relevant to GET requests, which is why it is sometimes called the GET string. But GET is an HTTP verb and the query string is part of the URL; and URLs don't have to be <code>http://</code>, so the query string can really be used against any scheme.</p><p>It is often said the query string should not be used to send data to the server, but I'm really not sure that's the case. The server should not store data as the result of a read request (HTTP's GET), but it is welcome to store data as the result of a write request (HTTP's POST or PUT). In which case it is entirely up to the server the mechanism by which the data are provided to it.</p><p>These are why you should call it the query string, not the GET string.</p><h3>Fragment</h3><p>The part of the URL after the <code>#</code> is called the <em>fragment</em>. This is not actually part of the resource identifier, but is provided for the <em>client's</em> benefit.</p><p>If you click on any of the footnote marks in this document<a href="#footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-4" name="fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-4"><sup>4</sup></a>, most browsers I give a toss about will jump to the footnote, and back again when you click on the number of that footnote.</p><p>No new page request is made. The browser is not being instructed to access a <em>different</em> resource. In the example earlier, the fragment is <code>#part-of-document</code>. The fragment is usually used to refer to a part of the document. In HTML and XML, this is either by the <code>id</code> or <code>name</code> attributes of the elements.</p><p>In this document, the <code>a</code> tags that jump around the page have <code>name</code> attributes that the browser uses to scroll to them when the URL fragment changes, i.e. in these blog-post resources, the parts-of-the-document that I refer to with URL fragments are the footnotes and the places the footnotes refer to.</p><p>Using the document fragment to refer to specific resources is a crime committed by many "JavaScript apps" today. The reason this is a crime is that it is not identifying the <em>resource</em>; it's identifying the <em>resource proxy</em>, which means the correct client must be used to actually access the resource itself. It's like having a proprietary browser that only understands a completely different URI format.</p><p>It's a crime because browsers are more than capable of intercepting URI requests inside an application and getting the application to update as necessary, and servers are more than capable of returning a javascript-app-with-resource-in-it as the HTML representation of the resource.</p><p>There is no reason besides lack of imagination to trample all over that URI system just to avoid reloading the page every so often.</p><h2>TODO</h2><p>Not mentioned is the idea of a "related resource". This can be a third part of the URI path whereby you request an index of a separate resource based on the current one:</p><pre>https://www.example.com/blog/post/1/comments</pre><p>This is, conceptually, the same as</p><pre>https://www.example.com/blog/comments?post=1</pre><p>but you may wish to return the results differently, e.g. with more expanded objects rather than just URLs to the results.</p><p>In upcoming posts we'll probably have a look at those "functional" resources I mentioned in passing. This post has been entirely about "object" resources, i.e. those resources that simply represent some representation of a real-world object, or a fake-world object, but ultimately something that can be represented as a JSON object with fields and values. I will also try to discuss the resource-first view of website building using the aforementioned point-of-sale in Odoo as an example.</p><p>We also haven't discussed how it is that you would relate resources to one another in knowable ways. This ties in with the hyperlink concept and is the thinking behind <a href="http://altreus.blogspot.co.uk/2014/12/day-16-webmachine.html">Web::HyperMachine</a> - HTML pages are already linked together with <code><a href="related-link"></code>, but there are myriad other ways even those use hyperlinks to refer to other resources, and even more ways in HTTP itself.</p><hr /><p class="footnote"><a href="#fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-1" name="footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-1"><sup>1</sup></a> I've omitted from this the <code>user:pass@</code> part that can be used before the hostname, because it's not very common.<br />
</p><p class="footnote"><a href="#fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-2" name="footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-2"><sup>2</sup></a> The "second-level domain" is colloquially the "company" part of the name, i.e. the first part that actually identifies at a human-readable level what it is the URI refers to. In some cases, such as <em>.co.uk</em>, the TLD is actually the SLD (co) and TLD (uk), and it is the third-level domain that is the company part. Colloquially, we can refer <em>.co.uk</em> as a TLD, so that this remains the SLD.<br />
</p><p class="footnote"><a href="#fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-3" name="footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-3"><sup>3</sup></a> A URL is basically a URI that you can actually use. That is, there exist URIs that refer to resources but that cannot actually be used to access that resource; for example the ISBN URI schema cannot be used to get an actual book.<br />
</p><p class="footnote"><a href="#fn-b8c799aa4a385a44b5cdc57b0c96dae07585b309-4" name="footnote-b8c799aa4a385a44b5cdc57b0c96dae07585b309-4"><sup>4</sup></a> Like this one.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com1tag:blogger.com,1999:blog-4857673566967429385.post-62243523704780081372014-12-17T14:45:00.001+00:002014-12-17T14:45:37.832+00:00Day 17: A complex and detailed investigation into the various merits and faults of the assorted combinations of codepage, character set and byte encoding of human-readable text.<p>There are 127 characters in ASCII and tens of thousands of characters in the real world. It is probably an interesting debate, trying to come up with the most efficient way of encoding non-ASCII characters without screwing everything up.</p><p>Don't waste your time. Use UTF-8 and Unicode.</p><p>"But what about UTF-16?" <b>No.</b></p><p>"But what about--" <b>NO.</b></p><p>ASCII is included in UTF-8 Unicode. So is everything else. Everyone understands it, everything's assuming it, and all the other encodings and charsets are more obscure and therefore harder to deal with.</p><p>Everyone (except PHP) has UTF-8 Unicode built in to whatever programming language they're using.</p><p>Unless you're writing for devices with memory measured in bytes and a network connection measured in baud then you have time and space to use the bloating of UTF-8 Unicode. So suck it up, be inefficient, and accept the VHS of UTF-8 over the Betamax of whatever you're looking all cow-eyed at today.</p><p>And, in case you were wondering, ASCII is <em>never</em> the right answer.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-46980180935669744612014-12-16T17:18:00.003+00:002014-12-16T17:18:53.912+00:00Day 16: Web::Machine<p><a href="http://p3rl.org/Web::Machine">Web::Machine</a> is pretty cool because it reorganises the way you think about your website's structure, focusing on the perspective you should really be starting with in the first place.</p><p>Web::Machine encourages you to construct several objects, each of which handles a URI by representing the resource to which that URI points.</p><p>Remember that URI is a Uniform Resource Identifier. <a href="http://altreus.blogspot.co.uk/2014/10/whats-wrong-with-javascript-in-template.html">We've had this discussion.</a> The parts of the internet that use URIs are based on the assumption that they are sharing information about resources, and hence the focus is on the resource.</p><p>Web::Machine starts with the resource. You construct an object and mount it as Plack middleware to handle the URI to that resource. These objects are actually the machines. You construct a Web::Machine with a subclass of Web::Machine::Resource, and if that's all you want to do, you call <code>->to_app</code> on it and plack it up.</p><p>Each Web::Machine so constructed is a Plack::Component. That means you can bring in a Plack::Builder and mount machines in it.</p><pre>my $builder = Plack::Builder->new;
$builder->mount(
'/resource' => Web::Machine->new(
resource => 'MyApp::Resource'
)
);</pre><p>Alternatively, you might prefer to use something like Path::Router, providing subs that build Web::Machines based on arguments.</p><pre>my $router = Path::Router->new;
$router->add_route('/resource/:id' => sub {
my ($req, $id) = @_;
Web::Machine->new(
resource => 'MyApp::Resource',
resource_args => [
id => $id,
],
)
->call($req->env);
});</pre><p>Two things are notable about this particular invocation. First, it is necessary to run <code>call</code> on the resulting machine manually. The second is that, now that we have actual args coming in, we're seeing how Web::Machine takes an <em>array ref</em> for these, not a <em>hashref</em>; i.e. it's an argument list and not required to be hash-shaped.</p><p>MyApp::Resource is what handles the actual magic: Web::Machine expects certain subroutines to be overridden from the base class <a href="http://p3rl.org/Web::Machine::Resource">Web::Machine::Resource</a> that define what this resource can do.</p><p>The sensible ones to provide are <code>content_types_provided</code> and the <code>to_*</code> filters that define how to represent this resource as the various content types it supports.</p><p>The documentation lists all of the functions that can be overridden to provide behaviour specific to this class.</p><h2>RFPR: Web::HyperMachine</h2><p>I've started taking this a step further. Resources are only part of what makes the interwebs work. The other part is the fact the resources are related to each other: hypermedia.</p><p>Up on the githubs is a start to the module <a href="https://github.com/Altreus/Web-HyperMachine">Web::HyperMachine</a>, which tries to wrap Web::Machine in an understanding of how the resources relate to one another. By adding a couple of DSL-like functions to the Resource class it is possible to automatically construct the URI schema for the system, using the declared names of resources and relationships within the resource classes themselves.</p><p>The user simply mounts those resources and the machine does the rest:</p><pre>#!/usr/bin/perl
use strict;
use warnings;
use Web::HyperMachine;
my $app = Web::HyperMachine->new;
$app->with('MyApp::Resource');
$app->to_app;</pre><p>And the resource would be e.g.:</p><pre>package MyApp::Resource;
use strict;
use warnings;
use parent 'Web::HyperMachine::Resource';
__PACKAGE__->uri('resource');
our @data = qw( hello hi hey howdy );
sub content_types_provided { [{ 'text/html' => 'to_html' }] }
sub fetch {
my ($self, $id) = @_;
return $data[$id];
}
sub to_html {
my $self = shift;
my $resource = $self->{resource};
q{<h1>} . $resource . q{ world</h1>}
}
1;</pre><p>If you plackup that script, you'll find that <code>/resource/0</code><a href="#footnote-4daefa2567b949f2b7b1a378b8f16c8176f3f205-1" name="fn-4daefa2567b949f2b7b1a378b8f16c8176f3f205-1"><sup>1</sup></a> will return an HTML page with "Hello world" in it; and other values will correspondingly index into the array.</p><p>Feedback on this concept is encouraged; it's not been worked on for some time, like most things I do, because I got bored of it, because I didn't have an actual use for it. </p><p class="footnote"><a href="#fn-4daefa2567b949f2b7b1a378b8f16c8176f3f205-1" name="footnote-4daefa2567b949f2b7b1a378b8f16c8176f3f205-1"><sup>1</sup></a> If 0 doesn't appear to work, you may have an outdated version of Path::Router. The issue tracker says it is fixed on CPAN now.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-21069615719406855902014-12-15T23:47:00.002+00:002014-12-15T23:47:57.394+00:00Day 15: Crime and Punishment<p>In today's post I'm going to try to convince you to think of the interfaces you make in terms of punishment, in order to find the path of least punishment.</p><p>Here's a perspective for you to consider: when someone uses your system, <em>they</em> are doing <em>you</em> a favour. Don't try to yes-but-what-if your way out of this; I'm not asserting that it is the case. I am saying that is how you should <em>consider</em> it to be. Assume that the user, given the option, will pick an alternative system. Design the interface from the point of view that it is the very fact people use the system that is the currency that measures its success. If people don't like using it, if you make it hard to do, they simply will stop doing so.</p><p>This is an important perspective if you are a business, because your system needs to get the user from state 1, wherein they have their money, to state 2, wherein you have their money. If you make that difficult to do, then they won't do it. You are not doing them a favour; don't treat them like you are.</p><h2>Punishment</h2><p>Punishment probably makes you think of unwanted tasks doled out to people for correction or restitution of some misdemeanour or other. This is a bit of a goal-oriented definition, because it implies a perpetrator in the first place; i.e. it expects that some misdeed has been undertaken for which recompense needs to be made.</p><p>People are, of course, falsely accused and given punitive action nevertheless. The focal point of the above definition is that of an unwanted task; some chore that must be gone through, which one is inconvenienced, perhaps embarrassed or humiliated, to do. The concept is one of a strong antipathy or disinclination to do the thing; hence it is considered punitive to require that the person do it.</p><h2>Crime and Punishment</h2><p>When you design an interaction between a human and a computer you are establishing a sequence of events that will allow the user to eventually find themselves in a situation whereby the thing they set out to do has been done. Within this highly abstracted scenario there are three players:</p><ul><li>You (the entity with which the task is being performed)</li>
<li>The user (the entity trying to perform the task)</li>
<li>The task (the sequence of events by which the thing moves from not-done to done)</li>
</ul><p>This set of three players has implied with it several types of tasks:</p><ul><li>Expected but trivial; these things do not inconvenience</li>
<li>Expected but undesirable; the user has prepared for this</li>
<li>Unexpected but trivial; these things are minor inconveniences</li>
<li>Unexpected and undesirable; necessary evils</li>
<li>Unexpected and undesirable <em>and avoidable</em>; <b>punishment</b></li>
</ul><p>When you design an interface and you've added something to that interface, seriously consider whether that thing can be considered punishing the user for something they didn't do wrong.</p><p><em>Especially</em> consider whethere it is punishment for something out of their control. In many cases it is necessary to inform the user that there was a problem; this may seem like punishment, because it is quite undesirable to have to go through all that again.</p><p>Well, it is. Reduce the impact of problems by not discarding all the information the user has entered. If the problem is on your side, don't force the user to pick up the pieces, because they won't. If the problem is on their side, only require the re-entry of that information - not the entire thing.</p><p>And if there isn't a problem, why are you making one?</p><h2>Amazon</h2><p>Amazon punished me recently. They have this 1-Click registered-trademark button that allows you to find something you want and have it on its way to you just by pressing a button. That's a great feature - they are absolutely doing me a favour by having it. And they do me a second favour by letting me amend the order for up to 30 minutes after it's created.</p><p>Then they punish me for wanting to do that.</p><p>If you try to change the delivery address of such an order you are required to "confirm" your payment details. Why? They told me (on <a href="https://twitter.com/AmazonHelp/status/543550733946454016">Twitter</a>) that it was a security precaution to prevent others from accessing my personal information.</p><p>What utter, rotten bullshit. This is rubbish design, pure and simple. If I <em>didn't</em> change my delivery address, I would not have to confirm anything! This is unexpected, undesirable, and completely avoidable. It is punishment for wanting to have it delivered somewhere else. That is not a punishable offence.</p><h2>SimplyBe</h2><p>I get very upset sometimes. SimplyBe are absolutely not the sort of company that want me to give them any money. Every single step in between me selecting a product and me paying for the product was a pain in the arse.</p><p>Here are the necessary evils of buying something online:</p><ul><li>Entering your payment details</li>
<li>Telling them where to send the product</li>
</ul><p><em>That is it</em>. Everything else beyond that is you not doing me a favour. Sometimes we accept certain things, like do you want to sign up for the newsletter? (No.) But there are really only two things a place needs to know about you in order to get your money from your pocket and into theirs. If they punish you for trying to do that, go somewhere else.</p><p>For the curious, my tirade can also be seen on Twitter, written live as I came across the problems with the checkout. Finding it is left as an exercise to the reader. Every single tweet in that set is about something I consider a punishment, and I consider myself as having been punished for <em>wanting to give them money</em>.</p><h2>Metro 2033</h2><p>I first started thinking about interfaces in terms of punishment while playing this game, Metro 2033, of which many readers may have heard. It was touted as one of the best games of whatever year I missed it in when it first came out. It's set in the subway of Moscow - the Metro - where humanity has retreated from whatever disaster has yet to be revealed.</p><p>The game goes, by stages, from stealth to survival to legging it to brawling to just wandering around in a township buying stuff. And it punishes you.</p><p>Progress in the game is saved by a checkpoint mechanic, although it doesn't tell you where the checkpoints are. All you know is that, if you die, you're going to be set back some arbitrary distance; although once you've failed once, you know where you're going to go back to.</p><p>The game is therefore, at the abstract level, a series of challenges that must be overcome in order to progress; failure in a particular challenge sets you back to, at best, the start of that challenge or, at worst, the start of the level. You don't know where until you fail a challenge, but when you've failed a challenge you have some idea of the new worst-case scenario.</p><p>The problem is that some challenges are more, well, challenging than others, but failing them causes you to have to repeat the less obnoxious ones in order to retry the difficult one. In a save-when-you-want game you would simply save before you reached the difficult challenge, in order to avoid repeating the easy ones more than once.</p><p>This reduces the easy challenges to chores, trivial tasks that you gradually become adept at and simply have to slog through to try the part you keep failing at, until eventually you find the secret to the difficult part. This quickly stops being entertaining. </p><p>Games should not be chores. Chores are punishment.</p><p>Incidentally, the game (so it calls itself) has another punishment mechanism: traps. Consider the welcome form of punishment, whereby you are set back for failing a challenge - this is the expected function of a game, since a game is supposed to be entertaining by presenting a challenge, and a challenge you can't fail is not a challenge at all. The trap I'm talking about is not a trap for the character in the game, but a trap for the player. In the game, traps are visible and have a disarming mechanism; but traps for the player are unexpected, random events. Unexpected, undesirable, but avoidable <em>by the designer</em>. </p><p>Twice, so far, the game has required me to be discreet, quiet, stealthy - this means <em>light off</em> - and then punished me by leaving traps in the dark. Things I cannot have avoided by using skill - points in the game where the only two approaches to the challenge would have caused me to fail. Damned if you do, and damned if you don't. The only way to beat the challenge is to have failed it <em>at that point</em> once already. How do I know there won't be another trap ahead? This challenge has become a chore.</p><h2>Codec<sup><a href="#footnote-c70d6cdec094c68923162fc6ec3b087ccf4bcc96-1" name="fn-c70d6cdec094c68923162fc6ec3b087ccf4bcc96-1">1</a></sup></h2><p>Maintain flow. Most of the things I've listed as examples of punishment are flow-breaking. Most of the time, the user doesn't want to have to know how to perform the task; they need to be prompted to enter information, and as little information as possible. Every step along the way is a step further away from them achieving their goal, and the value of your system is entirely measured in how many people use it to achieve their goals.</p><p>Common punishments include:</p><ul><li>Forcing the user to manually type information they use a computer to automate in the first place (autofill forms, or refusing to let me paste my generated passwords into the confirmation box).</li>
<li>Repetition of trivial tasks that shouldn't have to be done at all. </li>
<li>Requirement of information you don't strictly need. </li>
<li>Considering valid data to be invalid because your validation is broken (or vice versa).</li>
<li>Similarly, rejecting sensible input because you're scared of it (like most of my randomly-generated passwords).</li>
<li>Pretending to let you do something, and then moving the goalposts and not actually doing it.</li>
<li>Not providing sufficient information to help the user rectify the problem.</li>
<li>Fragmenting input forms across multiple pages.</li>
<li>Cramming a single page with too much input.</li>
<li>Discarding information because your fragile system shat itself.</li>
<li>Choosing difficult fonts and colours to read.</li>
<li>Making the user hunt for the next thing they have to do.</li>
<li>Related, leaving the user at the end of a process with no confirmation or failure message, so they don't know that they're done, or feeling that they have to do it all again.</li>
</ul><p>I'm sure if I use the internet for another day I'll be able to double this list but you get the idea. For every action the user has to take, is it something they've prepared for, and do they actually have to do it?</p><p class="footnote"><sup><a href="#fn-c70d6cdec094c68923162fc6ec3b087ccf4bcc96-1" name="footnote-c70d6cdec094c68923162fc6ec3b087ccf4bcc96-1">1</a></sup> [sic]<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-34176199804058724602014-12-15T14:43:00.003+00:002014-12-15T14:48:34.429+00:00Day 12ish: PERL<p><a href="http://perl-begin.org/learn/Perl-perl-but-not-PERL/">PERL is wrong</a>. It was invented at some point to mean Practical Extraction and Report(ing) Language but Perl was never called that originally.</p><p>Although I do quite like the interpretation <em>Poor Excuse for a Real Language</em>, which unfortunately doesn't initialise to <b>PHP</b>.</p><p>There's also a <a href="http://www.scriptarchive.com/"
rel="nofollow">swathe</a> of <a href="http://www.tizag.com/perlT/"
rel="nofollow">awful</a>, <a href="http://p3rl.org/CGI" rel="nofollow">ancient</a> code written in Perl.</p><p>This legacy dogs Perl's steps, despite the recent rise of Perl like an X-Wing rising out of Dagobah swamps.</p><p>Thus I propose a naming convention: Anything that can be considered to be dragging Modern Perl down be referred to as PERL code. It's clear how PERL is indeed a pathetic excuse for a real language. Perl resembles PERL as much as Episode IV resembles Episode I.</p><p>PERL is dead. Long live Perl.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com1tag:blogger.com,1999:blog-4857673566967429385.post-1117753926045634902014-12-11T16:28:00.001+00:002014-12-11T16:28:14.121+00:00Day 11: List context and parentheses<p>It's common to start off believing that <code>()</code> <a href="http://altreus.blogspot.co.uk/2011/08/lists-and-things-made-of-lists.html">make a list</a>, or create <a href="http://altreus.blogspot.co.uk/2011/11/in-context.html">list context</a>. That's because you normally see lists first explained as constructing arrays:</p><pre>my @array = (1,2,3);</pre><p>and therefore it looks like the parentheses are part of list context.</p><p>They aren't. Context in this statement is determined by the assignment operator. All the parentheses are doing is grouping up those elements, making sure that all the <code>,</code> operators are evaluated before the <code>=</code> is.</p><p>There is <em>exactly one place</em> in the whole of Perl where this common misconception is actually true.</p><h2>LHS of <code>=</code></h2><p>On the <em>left</em> of an assignment, parentheses create list context. This is how the <a href="http://search.cpan.org/dist/perlsecret/lib/perlsecret.pod#Goatse">Saturn operator</a> works.</p><pre>$x = () = /regex/g;
# |______________|</pre><p>The marked section is an empty list on the left-hand side of an assignment operator: the global match operation is therefore in list context.</p><h2>LHS of <code>x</code></h2><p>This is a strange one. The parentheses do construct a list, but the stuff inside the parentheses does not gain list context.</p><pre>my @array = (CONSTANT) x $n;</pre><p>In this case, <code>CONSTANT</code> - presumably <code>sub CONSTANT {...}</code> - is in list context; <code>x</code> gains list context from the <code>=</code>, and <code>CONSTANT</code> inherits it.</p><pre>my $str = (CONSTANT) x $n;</pre><p>Here we have <code>x</code> in scalar context because of <code>$str</code>, and <code>CONSTANT</code> in scalar context because of that. This is not really a whole lot of use, however.</p><h2>Various Contexts</h2><p>This sub reports whether it's called in scalar, list or void context<a href="#footnote-4cb29fdd5eef6a713018a81f09e3e893315f56df-1" name="fn-4cb29fdd5eef6a713018a81f09e3e893315f56df-1"><sup>1</sup></a>:</p><pre>sub sayctx { say qw(scalar list void)[wantarray // 2] }</pre><p>Now we can test a few constructs for context:</p><pre># void
sayctx;
# scalar
scalar sayctx;
# scalar
my $x = sayctx;
# list
my @x = sayctx;
# list
() = (sayctx) x 1;
# scalar
my $x = (sayctx) x 1;
# list
last for sayctx;
# scalar
while (sayctx) { last }
# scalar
1 if sayctx;
# scalar, void
sayctx if sayctx;
# scalar, scalar
sayctx > sayctx;</pre><p class="footnote"><a href="#fn-4cb29fdd5eef6a713018a81f09e3e893315f56df-1" name="footnote-4cb29fdd5eef6a713018a81f09e3e893315f56df-1"><sup>1</sup></a> Understanding it is left as an exercise to the reader.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-28293632334717346292014-12-10T17:35:00.002+00:002014-12-10T17:37:42.873+00:00Day 10: Fixes to DBIx::Class::InflateColumn::Boolean<p>I'm finding my new position at OpusVL ever more valuable. We like to put extra time into getting to the bottom of an issue because we rely so heavily on open-source software. Problems we discover in the modules we use are worth investigating for their own sake, simply because the amount of time already put into the modules by other people is years; years we didn't have to spend ourselves.</p><p>Today I discovered that, if I ran my <a href="http://p3rl.org/Catalyst">Catalyst</a> application under <code>perl -d</code>, it didn't actually run at all.</p><p>After much involvement from various IRC channels I came to the conclusion that the problem was in <a href="http://p3rl.org/Contextual::Return">Contextual::Return</a>; or rather, the problem was in the 5.14 debugger, since it seems OK in 5.20.</p><p>Anyway, Contextual::Return was employed by <a href="http://p3rl.org/DBIx::Class::InflateColumn::Boolean">DBIx::Class::InflateColumn::Boolean</a>, which I was using because SQLite <a href="https://www.sqlite.org/lang_altertable.html">doesn't have ALTER COLUMN</a>. We test <a href="http://p3rl.org/CatalystX::AppBuilder">components of Catalyst applications</a> as small <a href="http://p3rl.org/PSGI">PSGI</a> applications with SQLite databases backing them, which has its own problems, but in this case the issue was the column in question being <code>closed boolean NOT NULL DEFAULT false</code>, and SQLite not translating "false" as anything other than the string "false", and then shoving it in a boolean column anyway.</p><p>So DBIC faithfully gave me "false" back when I accessed the row, and "false" is true, so everything broke.</p><p>So I inflated the column.</p><p>This all resulted in a <a href="https://rt.cpan.org/Public/Bug/Display.html?id=100827">patch to DBIC:IC:Boolean</a>, authored by <a href="https://metacpan.org/author/HAARG">haarg</a>, removing the dependency on Contextual::Return entirely.</p><p>This may be a case of avoiding rather than fixing the problem, but since the problem appears to exist in the 5.14 debugger, the only way to fix that is to update to 5.20 - or whenever it was that it was fixed.</p><p>It also prompted me to rebuild the SQLite database to remove that default. Turns out DBIC doesn't fill in default values when creating rows.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-33742322286600294752014-12-09T12:13:00.002+00:002014-12-09T12:13:46.273+00:00Day 9: Scalar filehandles, or IO, IO, it's not to disk we go<p>Did you know you can open a variable as a file handle?</p><p>This is a great trick that avoids temporary files. You can write to the filehandle, and the stuff written thereto are available in the other variable. I'm going to call the other variable the "buffer"; this is a common term for a-place-where-data-get-stuffed.</p><p>Here's an example whereby I created an XLS spreadsheet entirely in memory and uploaded it using <a href="https://metacpan.org/pod/WWW::Mechanize">WWW::Mechanize</a>. The template for the spreadsheet came from <code>__DATA__</code>, the <a href="http://perldoc.perl.org/perldata.html#Special-Literals">special filehandle</a> that reads stuff from the end of the script. </p><p>This allowed me to embed a simple CSV in my script, amend it slightly, and then upload it as an XLS, meaning I never had to have a binary XLS file committed to git, nor even written temporarily to disk.</p><p>In the example below, a vehicle, identified by its VRM (registration plate) is uploaded in an XLS spreadsheet with information about its sale. The <code>$mech</code> in the example is ready on the form where this file is uploaded.</p><p>The main problem this solves is that the VRM to put into the spreadsheet is generated by the script itself, meaning that we can't just have an XLS file waiting around to be uploaded. As noted, it is also preferable not to have to edit an XLS file for any reason, essentially because this can't be done on the command line - LibreOffice is required, or some Perl hijinks.</p><pre>open my $spreadsheet_fh, ">", \my $spreadsheet_buf; # [1]
my ($header, $line) = map { chomp; [split /,/] } <DATA>; # [2]
my $xls = Spreadsheet::WriteExcel->new($spreadsheet_fh); # [3]
my $sheet = $xls->add_worksheet();
# processing
$line->[0] = $vrm;
$sheet->write_col('A1', [ $header, $line ]); # [4]
$xls->close;
$mech->submit_form(
with_fields => {
file => [ [ undef, 'whatever',
Content => $spreadsheet_buf ], # [5]
1 ]
},
button => 'submit',
);
# [5]
__DATA__
VRM,Price,Fees,Collection,Valeting,Prep costs
,2333,10,0,10,0</pre><p>The key to this example is in [1], which looks like a normal <a href="http://perldoc.perl.org/functions/open.html"> <code>open</code></a> call except for the last expression:</p><pre>\my $spreadsheet_buf;</pre><p>This is a valid shortcut to declaring the <code>$spreadsheet_buf</code> and then taking a reference to that:</p><pre>my $spreadsheet_buf;
open my $spreadsheet_fh, ">", \$spreadsheet_buf;</pre><p>The clever part is that now, <code>$spreadsheet_fh</code> is a normal filehandle that can be used just like any other; just as if we'd used a filename instead of a scalar reference. At [3] you can see a normal <a href="https://metacpan.org/pod/Spreadsheet::WriteExcel">Spreadsheet::WriteExcel</a> constructor, taking a filehandle as the argument, as <a href="https://metacpan.org/pod/Spreadsheet::WriteExcel#new">documented</a>.</p><p>At [2] you can see <code>DATA</code> in use, which reads from <code>__DATA__</code> at [5]. This also acts like a normal filehandle; <code><DATA></code> reads linewise, and we have to <code>chomp</code> to remove the newlines.</p><p>We <code>map</code> over these lines, chomping them and using <code>split /,/</code> to turn them into lists of strings; and this list is inside the arrayref constructor <code>[...]</code>, meaning we get an arrayref for each line.</p><p>At [4] we have processed sufficiently to have installed the VRM in the gap at the front of the second line, i.e. the zeroth element of <code>$line</code>, so <code>write_col</code> is employed to write both arrayrefs as rows (yes I know) into the spreadsheet.</p><p>When we call <code>$xls->close</code>, this writes the spreadsheet to the filehandle. But no file is created; instead, the data go to <code>$spreadsheet_buf</code>. If we were to print <code>$spreadsheet_buf</code> to a file now, we would get an XLS we can open.</p><p>Instead, at [5], we use the trick <a href="https://metacpan.org/pod/WWW::Mechanize#mech-submit_form">documented in <code>submit_form</code></a> (ether++ for reading everyone's mind) to use the file data we already have as the value of the form field.</p><p>This trick is remarkably useful. You can reopen <code>STDOUT</code> to write to your buffer:</p><pre>{
local *STDOUT;
open STDOUT, ">", \my $buffer;
do_stuff_that_prints();
do_stuff_with($buffer);
}</pre><p>but that's better written</p><pre>my ($buffer) = capture { do_stuff_that_prints() };</pre><p>from <a href="https://metacpan.org/pod/Capture::Tiny">Capture::Tiny</a>.</p><h2>See also</h2><p>If you use <a href="https://metacpan.org/pod/IO::Handle">IO::Handle</a> then your <code>$spreadsheet_fh</code> will be an object like any other - but these days, you get that simply by using lexical filehandles anyway.</p><p><a href="https://metacpan.org/pod/IO::Scalar">IO::Scalar</a> seems like a decent OO-type module to deal with this but also look nice.</p><p><a href="https://metacpan.org/pod/IO::String">IO::String</a> also works with strings-as-IO.</p><p>I've not tried either of these latter two, but YMMV etc.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-72657831261741654622014-12-08T18:16:00.000+00:002014-12-08T18:16:06.475+00:00Day 8: Mindset<p>It doesn't matter what language you start in. The language doesn't help. The problem is you; you're the new developer, the inexperienced green sapling; you're the one with no instinct, no sense of <a href="http://en.wikipedia.org/wiki/Code_smell">smell</a>, and no idea where to begin. You probably don't even have a problem you want solving.</p><p>Whenever we solve a problem we draw on our knowledge and experience to solve it. Knowledge and experience differ like theory and practice do. Knowledge is the theory. You can know something because you were told it, and it stuck. Arguably, the best way to know something is to understand it; then you know why it is the case, and what you really know is more general, more applicable, and hence more useful. Experience is practice; you've done this before. Experience is the sort of knowledge you need in order to produce a good solution to a problem, because experience tells you what the next problem is, and how to avoid it now.</p><p>Experience alters your thought process.</p><p>Today's example comes from irc.freenode.com#perl, where we see a green programmer trying to solve a problem:</p><h2>Report the powers of two that sum to produce a given integer</h2><p>That is, break down an integer into the powers of two from which it is composed.</p><p>Scroll no further if you wish to solve it yourself. In Perl.</p><p>No language can provide you, up front, with the knowledge you need to answer this question. Most languages have for loops and while loops, and something that can raise 2 to a power. But that's all you <em>know</em>. You have a few bits of theory, but no experience to draw upon. So your thought process goes something like this:</p><ul><li>I can take a number <em>n</em> and find the <em>n</em>th power of two <code>2 ** $n</code></li>
<li>I can store a value and compare it to my target <em>num</em> <code>$total > $num</code></li>
<li>I can loop an indefinite number of times with <code>while</code></li>
<li>The biggest power of two less than <em>num</em> is definitely part of it</li>
</ul><p>You reach the conclusion, using knowledge, that you can subtract ever-decreasing numbers from your target, in a loop. Any number that leaves you with a positive number simply means you can repeat the process with the new number, having remembered that particular power of two.</p><pre>use 5.010;
use strict;
use warnings;
my $num = shift;
my $power = 0;
$power++ until 2 ** $power > $num;
$power--;
while ( $power ) {
if ($num - (2**$power) >= 0) {
say "$power (" . (2**$power) . ")";
$num -= 2 ** $power;
}
$power--;
}</pre><pre>4 (16)
2 (4)</pre><p>Reasonable. Now here's my thought process:</p><ul><li>They want all powers of two that come together to sum a number</li>
<li>That's how binary works</li>
<li>We can ask the binary representation of <em>num</em> for all the on bits</li>
<li>The positions of those on-bits are the answer.</li>
</ul><p>So we write that.</p><pre>say for grep { $_ } map { 2 ** $i++ * $_ } reverse split //, sprintf "%b", shift</pre><p>This is a one-liner. Try it in <code>perl -E'...' 20</code>, in place of the <code>...</code>.</p><pre>4
16</pre><p>OK we'll break it down, but you'll see that each section maps roughly to each of the items in that list.</p><h3>"They want all powers of two"</h3><p>The answer is going to be a list. <code>say for LIST</code>, and we have to construct LIST. The powers of two have a test for validity, so there's probably a grep. <code>say for grep { CONDITION } LIST</code>.</p><p>We should really build an array for LIST, and use it at the end.</p><pre>use 5.010;
my @bits;
...
say for @bits;</pre><h3>"That's how binary works"</h3><p>Getting the binary representation of a number is easy; <code>sprintf "%b", EXPR</code>. In the one-liner we used <code>shift</code> to take the first command-line argument. We can put <code>$num</code> here and save the result of <code>sprintf</code> instead of using it directly.</p><pre>my $num = shift;
my $binary = sprintf "%b", $num;</pre><h3>"We can ask the binary representation for all the on bits"</h3><p>How? This is a two-parter. First you have to turn the string into bits. Then you have to find the on-bits.</p><p>Turning the string into bits is easy - you split it on the gap between characters:</p><pre>my @bits = split //, $binary;</pre><p>Not obvious is the finding the on-bits. See, we don't want the actual bits themselves; all the on-bits are <code>1</code>, so finding them all would simply tell us how many there are. We actually want to know <em>where</em> they are.</p><p>Trouble is, <code>sprintf</code> gives us <code>10100</code> for 20. The first bit is the high bit, but that has the smallest offset, i.e. it's the 0th digit in that string. And the other 1 is the 2th digit. Knowledge tells us that our 20 working example should report 4 and 16; but <code>2 ** 0</code> is neither of those, even though <code>2 ** 2</code> is.</p><p>The answer to this is actually in the original solution: we have to work backwards, biggest number <em>last</em>. That's why we reverse it.</p><pre>my @bits = reverse split //, $binary;</pre><h3>"The positions of those on-bits are the answer"</h3><p>In the final solution I report the powers of two, not the numbers we raise two to, and the positions are the numbers to raise two to, not the power of two to that. Clear?</p><p>The positions of the on-bits are found using a bit of a naughty <code>map</code>, which uses a counter outside its scope. <code>map</code> should really <a href="http://altreus.blogspot.co.uk/2011/10/understanding-and-using-map-and-grep.html">not have side-effects</a>. We can work around this in a proper script, however.</p><p>By iterating through the bits and incrementing a counter as we go, we can determine the value that this bit represents.</p><pre>2 ** $i++</pre><code><p>$i++</p></code><p>of course returns the value of <code>$i</code><em>before</em> incrementing it, meaning it starts off undefined. We can't have that.</p><pre>my $i = 0;</pre><p>Now we can produce a list of all those values:</p><pre>map { 2 ** $i++ } @bits;</pre><p>Plug this into <code>say</code> for debugging purposes:</p><pre>say for map { 2 ** $i++ } @bits;
1
2
4
8
16</pre><p>We've lost information - what happened to the fact some of the bits were turned off? Although I had this in knowledge, it was experience that reminded me that I can multiply:</p><pre>map { 2 ** $i++ * $_ } @bits;</pre><p>That's better - we also should <a href="http://altreus.blogspot.co.uk/2011/10/understanding-and-using-map-and-grep.html">always use <code>$_</code> in a <code>map</code></a> because <code>map</code> is supposed to transform <code>$_</code>.</p><pre>0
0
4
0
16</pre><p>Now we have something we can grep: <code>$_</code> itself!</p><pre>my @powers = map { 2 ** $i++ * $_ } @bits;
say for grep { $_ } @powers;</pre><p>This collects all powers, but only reports those with a nonzero value.</p><p>We can fix the <code>$i</code> situation by using <a href="http://perldoc.perl.org/functions/keys.html"> <code>keys</code></a> on <code>@bits</code>. <code>keys</code> on an array returns the list of indices, even though they're <a href="http://altreus.blogspot.com/2014/12/different-shapes-of-data.html">not really keys</a>.</p><pre>map { 2 ** $_ * $bits[$_] } keys @bits</pre><p>This uses <code>$_</code> in place of <code>$i</code> (0 to 4), but now that <code>$_</code> is the index, we have to get the actual bit value by looking it up in <code>@bits</code>.</p><h2>Answers on a postcard, please</h2><p>Here's the final script, then</p><pre>use 5.010;
use strict;
use warnings;
my $num = shift;
my $binary = sprintf "%b", $num;
my @bits = reverse split //, $binary;
my @powers = map { 2 ** $_ * $bits[$_] } keys @bits;
say for grep { $_ } @powers;</pre>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-69664281004861399602014-12-05T10:07:00.000+00:002014-12-05T10:07:01.433+00:00Day 4: RFPR for daemonize.pl<p>I've embarked on a new term, RPFR. An RFPR is a Request For Pull Requests: like an RFC, except for when you've already started writing code and you want people to add features or fix it, instead of bikeshedding about the spec for it.</p><p>This first one is for my daemonize script at <a href="https://github.com/Altreus/daemonize.pl">https://github.com/Altreus/daemonize.pl</a>. This script is a wrapper around <a href="https://metacpan.org/pod/Daemon::Control">Daemon::Control</a> (<a href="https://github.com/symkat/Daemon-Control">https://github.com/symkat/Daemon-Control</a>), which I wrote essentially so I could type</p><pre>daemonize starman --something --etc webapp.psgi
^M^M^M^M^M^M^M^M^M</pre><p>... and end up with an LSB script in init, because all the default answers to the questions were right.</p><p>Unfortunately the very first time I tried to use this somewhere else I discovered that it wasn't so straightforward, so now I'd like to collect either patches or issues on the repository for features or changes that would make this script that much more useful.</p><p>Essentially the goal is to automate as much of writing the Daemon::Control script as possible, and also to have an option to write it out as an init script instead of a Perl script.</p><p>Welp, just a brief one for day 4. They can't all be deep essays on the holistic nature of abstract data.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-87932092843719241142014-12-03T21:45:00.001+00:002014-12-05T10:07:43.021+00:00Day 3: Different shapes of data<p>One of the main points of suffrance for PHP is the conflation of what the rest of the world consider to be separate data structures: the array and the hash/dictionary/map/object/etc. Everyone agrees on the name of the array; less so on the name of the hash. We'll stick with hash (but later I'll say object, just to troll you).</p><p>This conflation is vehemently defended by PHP programmers, but I sense a certain cart-before-the-horse expectation if you try to get a PHP programmer to realise the problem with it. Which is to say, a PHP programmer has only seen PHP do it, and has seen how PHP works around the limitations of doing it, and therefore doesn't have the experience of languages with separate types to be able to understand intuitively that they are fundamentally different.</p><p>I'm not going to directly attack the fact it clearly has limitations, because this is acknowledged and understood; and everything has limitations. If we didn't have limitations, we wouldn't really have things at all, would we?</p><p>It is not the limitations of the aforementioned conflation that make it a problem; it is a deeper-seated, fundamental difference; logical in nature. Almost mathematically different, like numbers and vectors are.</p><p>I'm going to try to formalise the difference. Properly explain it, and make it plain.</p><p>We can start to understand the difference by scrutinising those very workarounds that PHP does use - to cope with the limitations - and the inconsistencies that we expect from any PHP anything at all ever.</p><p>Consider the <code><a href="http://php.net/manual/en/function.array-merge.php">array_merge</a></code> function:</p><blockquote><p>If the input arrays have the same string keys, then the later value for that key will overwrite the previous one. If, however, the arrays contain numeric keys, the later value will not overwrite the original value, but will be appended.</p></blockquote><p>And</p><blockquote><p>Values in the input array with numeric keys will be renumbered with incrementing keys starting from zero in the result array.<sup><a href="#footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-1" name="fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-1">1</a></sup></p></blockquote><h2>Doublethink</h2><p>It is being recognised that the structure is performing two functions; the first, with string keys, has unique properties. The same value cannot be repeated in the structure, because the identifying property of that piece of information is its string name: if the array were to have two keys of the same name, it would be impossible to distinguish between them on access. We can give this concept formal terminology: <b>it doesn't make sense</b>.</p><p>We say it does not make sense to have two keys with the same name. Looking at this under a semantic microscope we come to the realisation that we've accidentally used two different words for the same thing: "key" and "name". The key does not <em>have</em> a name; the key <em>is</em> a name. We can't restructure that sentence to avoid using both words, because whenever we try the thing we end up with doesn't make sense. We're forced to conclude that the reason we can't make the sentence make sense is that the concept we're trying to express cannot be formally expressed. Something that cannot be formally expressed can only be described as wrong, or nonsense, or such other dismissive words. The concept does not exist <em>to be expressed</em>.</p><p>The second concession this <code>array_merge</code> makes is that numeric keys are normally sequential. This, at first glance, appears to point to another uniqueness of key; two keys in an ordinal array will never be the same, for the exact same reason: the key is the key, and any access of that key will inevitably refer to the value associated with it.</p><p>Why, then, this acknowledgement that numeric keys are expected to be sequential? That is, why, if merging two arrays with numeric keys, do we concatenate, instead of overwrite?</p><p>This question starts to show the fundamental difference between the data structures. The principle is that of purpose. </p><h2>Shape of a hash</h2><p>String names are often called <em>properties</em>. This is because they:</p><ul><li>Tend to refer to a real-world attribute of a real-world concept, such as a person's name or an item's weight.</li>
<li>Don't make sense independently of the item. A person's name isn't a person's name if the person isn't involved. "Name" is meaningless if you don't know what it's the name <em>of</em>.</li>
<li>Together, as a collection, sufficiently define the object being described.</li>
</ul><p>Last things last, because that's important. All the properties of an object together define sufficient information about the object to perform all necessary tasks <em>with</em> that object, within the system. I'm saying object because that's a word we use both in the real world and in programming. An object in an object-oriented system has <em>properties</em>, or <em>attributes</em>. And observe that it is the <em>set of attributes</em>, not their names, that define the data structure.</p><p>A hash, or associative array, or whatever, is defining a <em>single thing</em>. The keys of this hash are the properties that are required to capture the important information about that item, just as the properties of an object are.</p><p>We will call the set of keys, or properties, that the hash has its <b>shape</b>. We can consider that formal terminology as well<a href="#footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-2" name="fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-2"><sup>2</sup></a>.</p><h2>Shapes of arrays</h2><p>It is not infeasible that an object can have a numerical property. This is often proscribed by programming languages, who won't let you start property names with numerical values when defining classes, but we're talking about hashes here. They can take any string value and use it as a property for this object.</p><p>For example, perhaps this object's keys are all identifiers into other things, and all values are boolean. It's an object representing associations between other things. A node on a graph, perhaps, storing other nodes' identifiers as keys, and boolean values determining whether there's a link to it.</p><p>A stretch, but not totally crap.</p><p>What of the ordinal array then? This is just it: the index you use to access an item in an array is <em>not a property of the array</em>.</p><p>We can actually see this best in a Java scenario: in Java, an array is an object that contains other objects. But the array has properties of its own; a length, a max length, a stored data type. It has functions that can be run on it: push, pop, splice, etc. It does <em>not</em> have a property called <em>0</em>, a property called <em>1</em>, etc. It is a completely different thing.</p><p>In C++ the same structure (an array with flexible size) is called a <code>Vector</code>. This is apt. Arrays are vector structures. The thing that PHP calls a "key" is actually an <em>index</em>; I already used the word, and so does PHP, interchangeably. But it is not a key! A key is a <em>property</em> of the data structure; an index is a <em>position</em> in the data structure, not a property <em>of</em> the data structure.</p><p>The array is a line; a mathematical, one-dimensional structure. At integer points along its length can be found data of arbitrary type. But these are not properties of the array, any more than the values described by a line on a graph are properties of the line. The fact these things are in order - 0, 1, 2, 3 - is a phenomenon that <em>follows on</em> from the fact we're sticking more things onto the end. The ordering of the items in the array is not defined by the indicies; the indices are defined by the ordering. The data in the array defines the shape of the array.</p><p>The hash is a bag; a lookup table. There is no graph that can describe a hash, because there is no natural ordering to the keys in it. Strings don't have natural ordering: "a" is only before "b" because we invented "a" and "b" and put them in that order. We didn't invent 1 or 2 and we didn't make 2 bigger than 1.<sup><a href="#footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-3" name="fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-3">3</a></sup> Is your name before or after your height? That doesn't make sense!</p><p>The fundamental difference is there, then. The keys to an array are defined by the data in it, but the keys to a hash define the data that goes in it.</p><p class="footnote"><sup><a href="#fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-1" name="footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-1">1</a></sup> A salient question at this point is <em>how do you know whether it is a string or not?</em>. Is <code>"0010"</code> a string? If not, is it the number 10 or the number 2 or the number 8? All four things are valid interpretations under commonly-used rules.<br />
</p><p class="footnote"><a href="#fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-2" name="footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-2"><sup>2</sup></a> As with all language, it doesn't matter what noises or letter-strings we use to define a concept. The important thing is that we all understand the same thing when we hear or see it. Let this word stand for the scope of this post; but you'll likely see the term "the shape of the data" referred to quite a lot in general.<br />
</p><p class="footnote"><sup><a href="#fn-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-3" name="footnote-9e05e3cd42b80f3ff28b1218e640483e570c9ca9-3">3</a></sup> We invented the <em>symbols</em> 1 and 2, but we didn't invent the platonic integers that 1 and 2 refer to. There was 1 earth before we evolved on it and used the symbol 1 to represent this number.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-49869957832332052972014-12-03T10:51:00.000+00:002014-12-05T10:07:52.528+00:00Day 2: Opt::Imistic<p>Can't believe I've not made a post about this ancient module. <a href="https://metacpan.org/pod/Opt::Imistic"><code>Opt::Imistic</code></a> is a module I wrote to facilitate the writing of command-line scripts that take options. It was inspired by the node module of the same(ish) name, <a href="https://github.com/substack/node-optimist">Optimist</a> (now deprecated).</p><p>All <code>Opt::Imistic</code> does is to parse <code>@ARGV</code> for things that look like options (using essentially the same rules as <a href="https://metacpan.org/pod/Getopt::Long"><code>Getopt::Long</code></a> does with <code>gnu_compat</code> options, i.e. the sensible way of doing it that doesn't cause too much ambiguity.</p><p>Long and short options are recognised by default, given GNU style. <code>-xyz</code> is three options and <code>--xyz</code> is one. Use whitespace or <code>=</code> to specify values to options. <code>=</code> can be used if the value looks like an option<a href="#footnote-de9984e00db4bd89be1906e7e794b54604577662-1" name="fn-de9984e00db4bd89be1906e7e794b54604577662-1"><sup>1</sup></a>.</p><p>As the docs say, this is a 90% module - <code>Getopt::Long</code> is for the other 90%.</p><h2>Hacky magic</h2><p><code>Opt::Imistic</code> relies on a piece of Perl magic the reader may not be aware of, which is that, for all of Perl's global variables, it appears to be the entire typeglob by that name that is global.</p><p>Simply put, this means that, because <code>@ARGV</code> exists, so does <code>%ARGV</code>. This is exploited by <code>Opt::Imistic</code>, by putting discovered arguments as the keys to the associated values, if any.</p><h2>Overload magic</h2><p>tm604 on IRC suggested that I can be even more magical if the discovered options were actually objects of a class that behaves correctly in different situations.</p><p>Since you can't prevent a person from multiply specifying a single-use option, instead of bailing horribly in this situation it's traditional to simply take the last instance of it. This implies the option needs a value; otherwise, it doesn't matter how many times you specify it. Think <code>--config</code>, for example.</p><p>Indeed, if the option doesn't take a value, it's usually expected that the script is going to count the number of times it's specified. Think <code>-v</code>, often "verbose", or <code>-vvv</code>, "extremely verbose".</p><p>Perl being Perl, the user doesn't have to care whether it was specified once or many times, if all the script cares about is whether it was specified at all. Zero is the false value here.</p><p>With a simple class<a href="#footnote-de9984e00db4bd89be1906e7e794b54604577662-2" name="fn-de9984e00db4bd89be1906e7e794b54604577662-2"><sup>2</sup></a>, entirely designed to carry overload magic, we can gather all this information at once.</p><pre>package Opt::Imistic::Option {
use overload
'""' => sub { $_[0]->[-1] },
'bool' => sub { 1 }
}</pre><p>This covers the common uses of command-line options:</p><ul><li><b>One or more values</b> - The objects are blessed array refs. Simply deref it for your values.</li>
<li><b>One value</b> - Treat it as a string, and it'll stringify. This also works for numbers. The overload ensures the last value is taken; all options are arrayrefs with at least one thing in them, or absent entirely.</li>
<li><b>A countable option</b> - Simply count your arrayref.</li>
<li><b>A boolean option</b> - Just use it in boolean context. You'll get a 1 if it's there.</li>
</ul><p>Again, this is a 90% solution, but check the docs for the extra functionality I added. You can specify options are required, and specify that at least <em>n</em> arguments must be left on <code>@ARGV</code> at the end of parsing.</p><p class="footnote"><a href="#fn-de9984e00db4bd89be1906e7e794b54604577662-1" name="footnote-de9984e00db4bd89be1906e7e794b54604577662-1"><sup>1</sup></a> I'm not sure whether I just came up with this or not. This might not (yet) be true.<br />
</p><p class="footnote"><a href="#fn-de9984e00db4bd89be1906e7e794b54604577662-2" name="footnote-de9984e00db4bd89be1906e7e794b54604577662-2"><sup>2</sup></a> This package uses the <code>package BLOCK</code> syntax, introduced in 5.14. The module doesn't specify 5.14; this is an oversight.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-59934664069588881102014-12-01T22:01:00.001+00:002014-12-05T10:08:00.711+00:00Day 1: Pod::Cats<p>Today is the first day of the advent calendar blog thing, so I thought I'd give it a whirl. Let's see how far I get.</p><p>I thought I'd do an easy one and put it out there how I actually do my blog. Well, I don't like writing HTML, and I don't like WYSIWYG editors, but I wanted something easy like blogger to actually do all the hard work for me.</p><p>I don't really like <a href="http://daringfireball.net/projects/markdown/syntax">Markdown</a>, primarily because it doesn't let me do certain things easily<a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-1" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-1"><sup>1</sup></a>. Footnotes are something I do commonly when I'm writing<sup><a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-2" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-2">2</a></sup>; they allow a certain second dimension to what would otherwise be a one-dimensional stream of words. In fact it's sort of a hyperlink, from before we had hypermedia.</p><p>You'll note, indeed, that my footnotes are hyperlinks. They link to their location on the page; and the footnotes at the bottom of the page link back to their marks. This is the sort of functionality I wanted from a blog markup language.</p><p>I decided that <a href="http://perldoc.perl.org/perlpod.html">POD</a> has a good balance of DWIM<a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-3" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-3"><sup>3</sup></a> and expressiveness, so I took the concepts and generalised them.</p><p>This led to <a href="https://metacpan.org/pod/Pod::Cats">Pod::Cats</a> being written. It really needs to be rewritten, now that it's something I actually use regularly. It's not my best code.</p><p>The name Pod::Cats came from a conversation I had quite some time ago in the #perl-cats channel on Freenode, wherein we thought it would be neat to have a community blog/podcast site called Podcats: the whole discussion started because someone typoed podcast.</p><p>Anyway, the module defines the grammar of Pod::Cats documents, but is intended to be extended to provide functionality. <a href="https://github.com/Altreus/altreus.blogspot.com/blob/master/lib/PodCats/Parser.pm">PodCats::Parser</a> does just that. This module could also do with a refactor.</p><p>The Pod::Cats parser uses a subclass of <a href="https://metacpan.org/pod/String::Tagged::HTML">String::Tagged::HTML</a> (<a href="https://github.com/Altreus/altreus.blogspot.com/blob/master/lib/PodCats/String/Tagged/HTML.pm">here</a>) whose entire purpose is to just render when stringified. In fact the main module may do this now - I should check!</p><p>Bugs exist in String::Tagged::HTML whereby, because there is no inherent ordering to tags in the same place in the string, the order of render is at the mercy of Perl's hashing algorithm. LeoNerd is pawing at a solution to this, so with luck this will solve my footnote issues soon. I've been helping with moral support and distractions.</p><p>Anyway, I save my files with the .pc extension and use a reasonably consistent set of Pod::Cats commands to mark up my blog posts. The idea is to maintain semantic structure while minimising the amount of actual meta-stuff in the file itself: something I felt POD was good at, with a few amendments of my own.</p><p>Once done I simply run my <a href="https://github.com/Altreus/altreus.blogspot.com/blob/master/parse.pl">script</a>, which overwrites or creates the HTML for any .pc file with a later save date than the equivalent HTML, or missing HTML. Then I upload the HTML. This means I can fudge the HTML afterwards without worrying about it being overwritten the next time I run the script.</p><h2>Images</h2><p>Currently I have no way of supporting images. I did try to; I looked into how Google uploads the images to Blogger. But there's no easy way of automating this, and I really couldn't be bothered working it out the hard way, so, currently, images are inserted in post-processing.</p><p>External images are supported with the <code>=img</code> command with the URL, however.</p><h2>Sauce</h2><p>What follows is the entire .pc file for this post up to the end of this paragraph, so you can have a taste of what it looks like<a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-4" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-4"><sup>4</sup></a><a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-6" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-6"> <sup>6</sup></a></p><pre>Today is the first day of the advent calendar blog thing, so I thought I'd give it a whirl. Let's see how far I get.
I thought I'd do an easy one and put it out there how I actually do my blog. Well, I don't like writing HTML, and I don't like WYSIWYG editors, but I wanted something easy like blogger to actually do all the hard work for me.
I don't really like L<http://daringfireball.net/projects/markdown/syntax|Markdown>, primarily because it doesn't let me do certain things easilyF<1>. Footnotes are something I do commonly when I'm writingF<2>; they allow a certain second dimension to what would otherwise be a one-dimensional stream of words. In fact it's sort of a hyperlink, from before we had hypermedia.
You'll note, indeed, that my footnotes are hyperlinks. They link to their location on the page; and the footnotes at the bottom of the page link back to their marks. This is the sort of functionality I wanted from a blog markup language.
I decided that L<http://perldoc.perl.org/perlpod.html|POD> has a good balance of DWIMF<3> and expressiveness, so I took the concepts and generalised them.
This led to L<https://metacpan.org/pod/Pod::Cats|Pod::Cats> being written. It really needs to be rewritten, now that it's something I actually use regularly. It's not my best code.
The name Pod::Cats came from a conversation I had quite some time ago in the #perl-cats channel on Freenode, wherein we thought it would be neat to have a community blog/podcast site called Podcats: the whole discussion started because someone typoed podcast.
Anyway, the module defines the grammar of Pod::Cats documents, but is intended to be extended to provide functionality. L<https://github.com/Altreus/altreus.blogspot.com/blob/master/lib/PodCats/Parser.pm|PodCats::Parser> does just that. This module could also do with a refactor.
The Pod::Cats parser uses a subclass of L<https://metacpan.org/pod/String::Tagged::HTML|String::Tagged::HTML> (L<https://github.com/Altreus/altreus.blogspot.com/blob/master/lib/PodCats/String/Tagged/HTML.pm|here>) whose entire purpose is to just render when stringified. In fact the main module may do this now - I should check!
Bugs exist in String::Tagged::HTML whereby, because there is no inherent ordering to tags in the same place in the string, the order of render is at the mercy of Perl's hashing algorithm. LeoNerd is pawing at a solution to this, so with luck this will solve my footnote issues soon. I've been helping with moral support and distractions.
Anyway, I save my files with the .pc extension and use a reasonably consistent set of Pod::Cats commands to mark up my blog posts. The idea is to maintain semantic structure while minimising the amount of actual meta-stuff in the file itself: something I felt POD was good at, with a few amendments of my own.
Once done I simply run my L<https://github.com/Altreus/altreus.blogspot.com/blob/master/parse.pl|script>, which overwrites or creates the HTML for any .pc file with a later save date than the equivalent HTML, or missing HTML. Then I upload the HTML. This means I can fudge the HTML afterwards without worrying about it being overwritten the next time I run the script.
=h2 Images
Currently I have no way of supporting images. I did try to; I looked into how Google uploads the images to Blogger. But there's no easy way of automating this, and I really couldn't be bothered working it out the hard way, so, currently, images are inserted in post-processing.
External images are supported with the C<=img> command with the URL, however.
=h2 Sauce
What follows is the entire .pc file for this post up to the end of this paragraph, so you can have a taste of what it looks likeF<4> F<6>
=footnote 1 Like this
=footnote 2 Because I have a lot to say and I don't want to interrupt the flow of the sentence
=footnote 3 Do What I Mean
=footnote 4 I've artificially promoted the footnotes to this point, since they need to be the last thing in the file to render properly. This is something I need to fix; footnotes should be stored and rendered at the end irrespective of where they turn upF<5>.
=footnote 5 In fact an auto-numbering system came and went and shall come back again at some point.
=footnote 6 Also available L<https://github.com/Altreus/altreus.blogspot.com/blob/master/pod/2014-12-01-pod-cats.pc|here></pre><p class="footnote"><sup><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-1" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-1">1</a></sup> Like this </p><p class="footnote"><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-2" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-2"><sup>2</sup></a> Because I have a lot to say and I don't want to interrupt the flow of the sentence </p><p class="footnote"><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-3" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-3"><sup>3</sup></a> Do What I Mean<br />
</p><p class="footnote"><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-4" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-4"><sup>4</sup></a> I've artificially promoted the footnotes to this point, since they need to be the last thing in the file to render properly. This is something I need to fix; footnotes should be stored and rendered at the end irrespective of where they turn up<a href="#footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-5" name="fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-5"><sup>5</sup></a>.<br />
</p><p class="footnote"><sup><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-5" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-5">5</a></sup> In fact an auto-numbering system came and went and shall come back again at some point.<br />
</p><p class="footnote"><a href="#fn-5204c9faf16a627a233f1c277d4a00759c7a6a1a-6" name="footnote-5204c9faf16a627a233f1c277d4a00759c7a6a1a-6"><sup>6</sup></a> Also available <a href="https://github.com/Altreus/altreus.blogspot.com/blob/master/pod/2014-12-01-pod-cats.pc">here</a><br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-38016622741946047762014-10-01T22:26:00.000+01:002014-10-01T22:26:21.400+01:00What's wrong with JavaScript in the template?<p>Those of you keeping score will know that I recently started a new job. This one is Perl, not PHP, and so a certain level of standards is expected from the code. What with Perl having all these neato features and excellent web frameworks, I at least consider it on a par with Python and Ruby in its utility.</p><p>Perusing the new-to-me codebase I of course discover some of the hysterical raisins that live there, much of which is easily forgiven because the original coder had the foresight to apologise in a comment for doing it in the first place. But one thing stood out to me as a prime candidate for refactoring: JavaScript in the templates.</p><p>I said as much and was surprised to be posed the question, "What's wrong with JavaScript in the templates?"</p><p>Surprised not to be asked the question, but because I didn't know what the answer was. I've worked enough on the front end of previous jobs to have enough experience in the matter that seeing JS in template code makes me flinch, but never have I been asked to actually introspect this reaction and explain it.</p><p>Questions like that are primo blog post material, and it's been a while since I properly got my teeth into one, so on my journey home I put my mind to formalising quite what it was about it that made me want to rip it out and refactor the life out of it.</p><h2>What it's not</h2><p>Some obvious answers come to mind, with varying validity. </p><ul><li>Is it because it's hard to find? No. Everything's hard to find. <code>ack</code> for it - you'll find it soon enough.</li>
<li>Is it because it violates separation of concerns? No. In fact, you could argue that it improves it, by encapsulating JavaScript only useful to a template inside that very template.</li>
<li>Is it because the only reason most people put JS in a template is so they can use the templating language to build JS? Well yes, but that's just the same question. What's <em>wrong</em> with it?</li>
<li>Is it because it's not reusable? Well, yes and no. Most template JS is not intended to be reusable; it's quite specific to that particular template, and there's little use for it elsewhere. More on this point later.</li>
<li>Is it the same reason we don't put CSS in the template either? Or inline in the HTML? Yes! By Jupiter, yes! We find the answer in the template itself. It's the other, main part of the template that we've not mentioned yet - the HTML.</li>
</ul><h2>What lies beneath</h2><p>To answer the question, we must deconstruct the web page itself and look at the parts. What are we really looking at when we look at a web page? What are we really providing when we build a template? What is the purpose of the HTML, the TT2 or Jade or Mustache code that wraps or creates it?</p><p>Most web pages follow a similar structure: There's the <code><html></code> with its <code><head></code> and <code><body></code>; the body has a <code><div class="header"></code> or, better yet, a <code><header></code>, and some sort of <code><div id="content"></code>. Then at last there's a bunch of stuff that finally gets to the point, i.e. displays whatever it is the page is displaying.</p><p>Most template structures separate all the pre/postamble from the content itself. Even in the CGI days we, naively but with good intent, would have a <code>header.html</code> and a <code>footer.html</code> and we would render the header, then the body, then the footer, to <code>STDOUT</code>. More recently, we have a single file with the pre- and postamble in it, and we import the rendered content into that. We tend to also have a considerable number of satellite template files representing handy widgets and reusable code and all the other things that I've alredy said aren't really the reason why we don't do the title of this article.</p><p>We knew then, as we know now, something we always forget to talk about; something implicit in everything we do here. While we make all these templates rendering data in consistent ways we somehow lose sight of the simplest of notions: we are <em>representing resources</em>.</p><h3>Resource and Framing</h3><p>"Resource" is a fully-functional word, writ deep into the very clay with which we make our internets; vis-a-vis HTTP. HTTP works with a <em>verb</em> and a <em>noun</em>, i.e. it says "Do this to this". "Framing" is a word I've picked to describe what it is we website-makers do to resources to make them look nice for people using browsers that conform to the standards set out to allow us to do so.</p><p>HTTP's nouns are URIs. URI means <a href="http://en.wikipedia.org/wiki/Uniform_resource_identifier">Uniform Resource Identifier</a>. The R in URI (or URL or IRI) means resource. It means thing; it's identifying the nouns of the internet. We respond to a (request to a) URI with a resource, represented in HTML format for the purposes of this discussion. We <em>know</em> this, but we never <em>say</em> this - and so whenever we get discussions, no one ever uses it as a basis for finding answers. But the concept of resource contains the answer to our question.</p><p>When we divide our templates up into separate files there is the tacit goal that the template we use to represent the actual, specific resource contain as little HTML as possible. Why? Well, mostly for consistency. We want to frame all our resources - at least those related to each other - in the same way. That means that if we put as little HTML as we can get away with into our resource templates, we can put as much as we can get away with into our framing templates, and thus have as little variation between the rendered resources as we can. A side effect, and therefore a second benefit, is that if we want to reuse or amend our framing, we can do this in one place - it's DRY.</p><p>We already recognise the difference between frame and resource: it's encoded right there in <code><div id="content"></code>. How many of your templates resemble this structure?</p><pre><body>
<stuff></stuff>
<div id="content">
<% content %>
</div>
<more stuff></more stuff>
</body></pre><p>That right there is the boundary between Alliance and Reaver space. Uh, I mean, the place where the framing goes away and the resource begins. The resource is all the data that change when you ask for a different ID, or a different resource type. The resource is that which, if you took all the HTML away, would <em>still be what you asked for</em>.</p><h2>I've nearly made my point</h2><p>Not all resources are data. Some resources are forms. I'm choosing forms as an example for another resource type because we're all familiar with them doing stuff.</p><p>Forms contain no data, but instead prompt you for data, and allow you to create more resources. Nominally, they represent the <em>structure of the resource type</em>, but don't represent any particular <em>record</em> of that type. The form holds the key to the answer: <em>behaviour</em>.</p><p>Consider:</p><pre><form action="/upload_image" method="post" enctype="multipart/form-data">
<label for="image">Upload image:
<input name="image" type="file">
</label>
<input type="submit">
</form></pre><p>This is a form with a file control, as you well know. It renders as a box with a "Browse" button. This one renders with a label, "Upload image:".</p><p>If you click on the label, the text of the input, or the browse button, you get the same behaviour: a file browser pops up. When you select a file and confirm it, the name of the file appears in the text part of the input, unless some jackass has installed Uploadify or similar, and broken it.</p><p>It also renders a single submit button. The button looks like all the other buttons on your website because you don't put CSS in your templates. The reason for that is being explained as we speak. I mean, as you read. I mean now.</p><p>When you click the submit button, the browser composes an HTTP POST request to the URL <code>/upload_image</code> on the host that served this resource. This request contains the entirety of the selected file, encoded in such a way that the receiving server can understand it. Presumably, the resource at that URL knows what to do with it.</p><p>Now, kindly point out to me the part of the HTML snippet above that implements any of that behaviour. </p><p>It's not there.</p><p>Nouns and adjectives - that's what the HTML is made of. There is not a single verb in the entirety of that form, and yet those few lines perform, implicitly, functionality that you would probably have to look up on Wikipedia to implement yourself.</p><p>Not all resources are forms, either. Here's a video resource, shamelessly stolen from Wikipedia, and represented in HTML format:</p><pre><video src="/movie.webm" poster="/movie.jpg" controls> </video></pre><p>Here's a more familiar one:</p><pre><img src="/images/avatar.png" alt="avatar" title="Get your pointer off my face"></pre><p>Noun-adjective-adjective-adjective. Noun adjective-adjective-adjective. The <code><video></code> noun:</p><ul><li>Fetches the resource at '/movie.jpg' of the host that served this HTML resource, and renders it at the place in the page concordant with the styling associated with it and the rest of the HTML.</li>
<li>Puts some sort of controls on this image, probably a play button, which, when clicked, causes the resource at '/movie.webm' to be fetched.</li>
<li>Renders the fetched video file <em>in situ</em>, replacing the still image, and plays any sound that comes with it.</li>
<li>Renders further controls, such as a scrubber, pause, volume slider.</li>
<li>Affects the right-click menu of the browser to provide appropriate options to a video: save video, get URL, get URL at this time, etc.</li>
</ul><p>Plus anything else I've forgotten. The <code><img></code> noun has similar, albeit many fewer, effects: the image is fetched and rendered without user interaction. Indeed, if the image is an animated gif, it will animate! On its own!</p><p>This borderline-facetious set of examples serves to point out that the browser has already <em>got</em> verbs. The nouns (HTML elements) say which verbs you want to use (and where to put the visuals for the user's interaction), and the adjectives (the attributes of the elements) control the parameters that the verbs need. (Fetch which video? Play automatically?)</p><p>This is called <em>semantics</em>.</p><h2>Semantics!</h2><p>I'm going to define semantics as the use of nouns to imply verbs<sup><a href="#footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-1" name="fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-1">1</a></sup>. Form fields come with behaviour, and you say which behaviour you want through nouns, i.e. the choice of which input you use. Semantics also covers those adjectives that fine-tune the noun's behaviour by describing it further.</p><p>Semantics tell things how to behave based on what the resource contains. An HTML resource often contains framing. Semantics go into the HTML to tell anyone who cares which bit they can ignore. Semantics is the way you phrase things; it's how you describe the resource.</p><p>Consider:</p><pre><div id="content"></pre><p>A web scraper can use this sort of thing to know what to ignore. Ignore is a verb. The HTML doesn't say "ignore this"; that's for the client to decide.</p><p>The browser isn't going to ignore it - but the browser doesn't care about this particular piece of semantics<a href="#footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-2" name="fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-2"><sup>2</sup></a>. If the CSS says to do something to it then the browser will do that to it, but the browser doesn't do that by default.</p><p>The web scraper will skip anything outside this div - provided it knows what the 'content' ID means - and the browser will do nothing based on this ID because it hasn't been told to.</p><p>That right there is the answer. There is a difference between all the things it is possible for a browser to do and all the things the browser can already do. You can stick together awesome websites entirely using HTML5 and CSS3, but often you want behaviour that is not already built-in to the browser. Maybe you want <code>div#content</code> to have special styling or behaviour, but browsers don't come with that built-in. </p><p>And indeed, styling is just a form of behaviour - CSS tells the browser how to behave when it renders certain elements in certain configurations. JavaScript tells the browser how to behave when the <em>user</em> does things.</p><p>This is the point where people start putting JavaScript into templates. A specific form needs special behaviour, so you add a <code><script></code> tag and then output the form.</p><p>Smash! go the semantics. Fie! cry the tortured frontenders.</p><p>None of the behaviour you ever write is useful only once. I told you I'd get back to the reusability point. The JavaScript doesn't go in the template because it's not reusable, sure, but why is that a problem?</p><p>The problem is the JavaScript defines <em>verbs</em>. Semantic HTML is that HTML which uses only nouns, and lets the browser select the correct verbs.</p><p>JavaScript, therefore, is correctly a <em>separate resource</em> that <em>adds verbs</em> to the browser, and defines the nouns to which they apply. That's why everything eventually ends up as a JavaScript plugin; and sometimes as core browser behaviour.</p><p>Essentially, we're saying that JavaScript is a CSS file that defines behaviour, not styling. Where CSS tells the browser how to interpret the semantics of your HTML in terms of colouring, positioning and so on, JavaScript tells the browser how to interpret the semantics in terms of direct functionality - behaviour.</p><p>Indeed, not only should JavaScript never go into the template, it should never go into <code><script></code> tags either. Just like CSS should never go into <code><style></code> tags.</p><h2>The Related Resource</h2><p>Resources have related resources. If you strip out all the framing of your HTML resource (e.g. you render it as JSON instead) you are still going to keep many of the hyperlinks - the contents of any <code><a></code> tag inside the content div, perhaps some of the image sources. That's because the HTML framing is just rendering the content in a human-readable way<a href="#footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-3" name="fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-3"><sup>3</sup></a>. The relations between resources are actually part of the resource itself, or at least metadata to it.</p><p>This is important because it addresses one of the main reasons people put JavaScript in templates: so that they can use the template language on the JavaScript, and thus build resource-specific JS that renders, e.g., a list of related resources when you click some "See related" button.</p><p>If the resources are related they should <em>already be in the page</em>. I seriously cannot stress that enough. Either the related resources are, or are not, relevant to this representation of the resource.</p><p>If the HTML went away and you were returning JSON, would you, or would you not, list those related resources as metadata, one way or another?</p><p>They cannot be part of the framing: the framing is consistent across the whole site! They are unique to this resource; and the style of list that is invisible until a button is pressed is unique to this type of resource.</p><p>But is "style of list" not an adjective about this list? Is list not a noun? Cannot you use the noun-adjective semantics to say, "This is a list of related resources, and it is of type pop-up-on-button"? HTML is amply equipped to represent this semantically: we even have the <code>rel</code> attribute to let you specify which button should activate the list.</p><p>Related resources belong in the page. Either as a hyperlink, or directly in the HTML. If you want to save bandwidth, you don't put the whole list in, but you put in a hyperlink placeholder instead. The important thing is that the HTML is accurately representing the resource. Just like the JSON would. Don't force non-browser consumers of your HTML resource to figure out how to run the JavaScript just to get related data.</p><h3>e.g.</h3><em><p>This|http://harvesthq.github.io/chosen/</p></em><p>is Chosen. You've probably seen it before. You start typing in a form field, and it lists all matching options, filtering as you type.</p><p>Chosen can either use an existing set of options, such as from a select box, or a URL from which to fetch options that match the string. </p><p>Both of these can be in the HTML before the JS even runs. The list of options is a related resource; it is simply represented in different ways. The first way puts all of the related resources in with the main resource; the second way puts a hyperlink to a single other related resource, from which they can be fetched when it's appropriate to do so.</p><p>At no time is it necessary to put this data into the JavaScript. JavaScript can read. Hell, the JavaScript should work on the JSON representation and all you'd have to change would be how it finds the data.</p><h2>The Answer</h2><p>The answer, then, is semantics. Of course it is. But it's what semantics <em>means</em> that turned out to be the difficult thing to define here.</p><p>Semantics is about saying what this resource is; it's metadata about the resource itself. Semantics allows the client to make the decisions about what parts of the resource are relevant and what parts are not.</p><p>It's exactly the same principle by which responsive web design works.</p><p>It's exactly the same reason you don't put inline CSS into your HTML.</p><p>It's exactly the same reason you've never written a video player, or had to decode the JPEG file format manually in JavaScript and blit the resulting bitstring onto a canvas element.</p><p>It's exactly the same reason you don't know how to launch a file browser dialogue box.<sup><a href="#footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-4" name="fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-4">4</a></sup></p><p>It's exactly the same reason <a href="http://webcomponents.org">web components</a> exist.</p><p>It's exactly the same reason JSON resources don't come with a stylesheet or JavaScript.</p><p>It's exactly the same reason we now have <code><nav></code> and <code><section></code> elements.</p><p>It's exactly the same reason we can produce screen-reader-friendly representations of HTML pages when the HTML page is correctly structured.</p><p>It's because you are describing what the resource <em>is</em>, and letting the client decide what it <em>does</em>.</p><p>*drops mic*</p><hr /><br />
<p class="footnote"><sup><a href="#fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-1" name="footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-1">1</a></sup> A separate discussion</p><br />
<p class="footnote"><sup><a href="#fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-2" name="footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-2">2</a></sup> Not all HTML is for the browser. HTML is a perfectly sensible representation format for machine use as well.</p><br />
<p class="footnote"><a href="#fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-3" name="footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-3"><sup>3</sup></a> Perhaps better: the HTML framing is a machine-readable way of getting the browser to render the content in a human-readable way. </p><br />
<p class="footnote"><a href="#fn-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-4" name="footnote-e74a88a9e0bcf33ddc2c36033ae0e7e4a6dbab8a-4"><sup>4</sup></a> In principle. HTML5 advances in file handling mean it is more common for the file dialogue to be called directly from JS.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-36690481794851517122014-04-27T12:54:00.002+01:002014-04-27T12:55:14.107+01:00Changing OpenElec's /tmp size<br />
<p>OpenElec has a limited <code>/tmp</code> partition. Very limited, i.e. 10MiB. Many things fall over because they need more than this on the occasion - especially if it's not the only thing using the tmpfs.</p><p>In order to change this you either have to hack around with automatically-created symlinks in startup scripts, or change it yourself.</p><p>The size of the <code>/tmp</code> partition is stored in <code>/etc/init.d/01_mount-filesystem</code></p><pre>mount -n -t tmpfs -o size=10m tmpfs /var</pre><p>The problem is, that file is readonly. The reason it's readonly is that the entire root filesystem is stored in a <a href="http://squashfs.sourceforge.net/">squashfs</a> partition.</p><p>To amend it, it is simply a case of unsquashing it, fixing it, and resquashing it.</p><h2>Fix it</h2><p>Pull the SD card out of your RPi (I'm assuming that's where you have it) and put it into your card reader. Let your system mount it.</p><p>You should have a SYSTEM drive somewhere on your computer. Lubuntu mounts it at <code>/media/altreus/SYSTEM</code>, so let's go with that.</p><pre>$ mkdir squash
$ cd squash
$ cp /media/altreus/SYSTEM/SYSTEM SYSTEM.bak
$ unsquashfs SYSTEM.bak</pre><p>Now we have a copy of the OpenElec root filesystem in a <code>.bak</code> file so we can undo it when we screw it up later. We also have the files themselves unpacked into <code>squashfs-root</code>. This is the default place <code>unsquashfs</code> puts them.</p><pre>$ vi squashfs-root/etc/init.d/01_mount-filesystem</pre><p>Change the file to have a better size <code>/tmp</code>. I used 500mb because my SD card is 8GB. Ignore the first instance of <code>tmpfs</code> in the file; we want to change the 10mb one.</p><pre>$ sudo mksquashfs ./squashfs-root SYSTEM</pre><p>It's important that you do this with sudo. The file <code>/etc/shadow</code> has permissions <code>000</code>, making it only accessible by root. This is how we got it when we unsquashed it, so this is how we want to keep it. My <code>/etc/shadow</code> is 600, but they presumably wanted theirs to be <code>000</code>. If we want to do the above step without root, we'd have to change the permissions so our user can see it - we can't change the permissions after it's squashed, so the only way to get a <code>000</code> file into the filesystem is to squash it with root.</p><p>Anyway, done.</p><pre>$ cp SYSTEM /media/altreus/SYSTEM</pre><p>Your new squashfs file will be mounted by OpenElec and your <code>tmpfs</code> will now be mounted with the size you gave it.</p><p>I'm not 100% certain this is stable. My Pi has started rebooting occasionally; but I might be giving it more than it can handle. It is an old model, but if I've introduced a bug because 500mb is too much, or something, I'm sure I'll get to the bottom of it and update the post,</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-73436275721363187842014-02-27T13:06:00.000+00:002014-02-27T13:38:29.516+00:00Code review time!<p><a href="http://www.etsy.com/uk/listing/169160722/php-wedding-code-geek-chic-wedding-art">Look!</a> A horrible piece of code in a horrible language in a horrible frame for a sickeningly twee ceremony that should have been made obsolete along with the Inquisition!</p><p>Let's review it.</p><p>Here's the code, with line numbers.</p><pre>01 <?
02 function do_wed() {
03 if ($objections != true) {
04 function do_vow() {
05 $vow = 1;
06 do {
07 if ($richer === 1
08 && $poorer === 1
09 && $sickness === 1
10 && $health === 1) {
11 function have_hold($a,$b) {
12 ini_set('session.gc_maxlifetime','forever');
13 }
14 have_hold('husband','wife');
15 define('friend', true);
16 define('partner', true);
17 define('faithful', true);
18 if ($i = 'do') {
19 $f = 'finger';
20 $r = 'ring;
21 $f = $f + $r;
22 }
23 }
24 $vow = $vow + 1;
25 } while ($vow != 2);
26 }
27 do_vow();
28 $register = array_fill($details);
29 print_r($register)
30 return $kiss;
31 }
32 }
33 do_wed();
34 ?></pre><p>Let's go!</p><h4>line 1</h4><p>We use long tags here. <code><?php</code></p><h4>line 3</h4><p>Undefined variable <code>$objections</code>.</p><p><code>$objections != true</code> better written <code>!$objections</code>. But this is not what you meant; you meant <code>count($objections) == 0</code>, since it will be an array of them</p><h4>line 4</h4><p>Don't define functions inside other functions.</p><h4>lines 6, 25</h4><p>You know how many vows you want. Use a for loop. Better, use an array of vows and populate it with two Vow objects, which represent the conditions each person agrees to. This means you can marry more than 2 people. The <code>do_wed()</code> function should take the people to wed as arguments. Use <code>func_get_args()</code> to loop over all of them, or <code>(...$parties)</code> in the next version of PHP.</p><p>Useless loop anyway. <code>do_vow()</code> should be called twice with the person currently vowing.</p><p>"Twice" is a western concept. This code is not internationalised.</p><h4>lines 7-10</h4><p>Undefined variables. None of these equals 1. It is unlikely that all four of these things would equal 1 at the same time. You want to test the party's agreement to these concepts, not the value of these variables. You need Person objects.</p><h4>line 11</h4><p>A function in a function in a function? This function takes two parameters and uses neither. Get rid of them.</p><h4>line 12</h4><p>This ini parameter takes an integer. <code>'forever'</code> is not an integer.</p><h4>line 13</h4><p>This closing brace does not line up with the function definition on line 13. It does line up with the if on line 7, which implies you've forgotten to close the function, but scrutiny shows that you've misaligned the brace.</p><h4>line 14</h4><p><code>have_hold</code> does not take any parameters any more.</p><p>This is exclusivist. Not all marriages are between a husband and a wife. These should be parameters to <code>do_wed()</code>.</p><p>This function is run twice, both times with the same parameters. It should swap over for the second iteration.</p><h4>line 16</h4><p><code>'partner'</code> is presumably the person we are not currently dealing with.</p><h4>line 17</h4><p><code>'faithful'</code> is not a boolean value and should be configured per app. It needs to be a data structure containing parameters of faithfulness, i.e. boundaries.</p><h4>line 18</h4><p>This is always true. Remove this condition. <code>$i</code> is never used, so remove the assignment too.</p><h4>lines 19, 20</h4><p>Useless variables. Either accept them as parameters or use the literal strings directly.</p><h4>line 21</h4><p>If you'd not used these useless variables you'd realise you're trying to numerically add strings. <code>.</code> is the concatenation operator. What is a 'fingerring'?</p><p><code>$f</code> is discarded. Just omit this entire block.</p><h4>line 22</h4><p>What is this supposed to line up with?</p><h4>line 23</h4><p>This closes the <code>if</code> that looks like it is closed on line 13. But it does not line up with it.</p><h4>line 24</h4><p>Better written <code>$vow++</code>, but we've replaced this with an array of Vow objects containing agreement parameters, so don't do this any more.</p><h4>line 25</h4><p>The only reason this would be a while loop is if you're just going to keep asking until both (all) parties agree. This is not how one should enter into a marriage.</p><h4>line 26</h4><p>This closes <code>do_vow()</code> but does not line up with it.</p><h4>line 27</h4><p>This is what should be run <em>n</em> times, once per party in the agreement.</p><h4>line 28</h4><p><code>array_fill</code> takes three parameters. Register should be an object.</p><h4>line 29</h4><p>Syntax error - missing semicolon.</p><p><code>print_r</code> is not the best thing to use here. Serialise this properly, perhaps with JSON so it can be consumed by an API or HTML so it can be styled and displayed properly.</p><h4>line 30</h4><p>Undefined variable <code>$kiss</code>. Kiss is a verb and should be a function.</p><h4>lines 31, 32</h4><p>These braces should line up with what they close.</p><h4>line 33</h4><p>Don't run a function when it is defined - that's not how you create a library.</p><p>This function could at least be parameterised with the names of the people being married. Isn't Etsy about crafts and hence personalisation?</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-32341480077957301452014-02-06T13:16:00.001+00:002014-02-06T13:18:14.115+00:00Model student<p>Models! Model trains, model students, model aeroplanes, model citizens. Fashion model, data model, business model. Ford Model T. Model number.</p><p>All these different uses of the word model have a commonality, the understanding of which is important to the understanding of what it is we mean when we talk about models in computing. This commonality may be considered the <em>abstract</em> meaning of "model": the meaning that exists behind all the real-world uses of it.</p><p>This concept is that of representation. Physical models are scaled-down representations of the things they model. A fashion model is really the representation of real people who would wear clothes (showing quite how divorced from reality fashion really is). A business model is a wordy representation of how the business will operate. Even the term "Ford Model T" is actually referring to the <em>blueprint</em> of all cars of that type: "Model" is referring to the type, not the car itself.</p><p>In computing, then, a model is a representation, a blueprint, a prototype that encapsulates the important details about the thing it is modelling. A good model will be a minimal but sufficient representation of the system it is modelling.</p><p>An easy example is the rolling of dice.</p><h2>1d6</h2><p>Dice are a familiar system to everyone, I hope. They neatly encapsulate our idea of randomness, at least that one we're taught in primary school, whereby the outcome of the system is not predictable from the input.</p><p>When we roll a d6 we expect to see one of its six faces pointing upwards but we don't know which one until it does so. Indeed on most dice we see the number represented as a pattern of dots; the number of dots being the number it shows.</p><p>This, if you're not used to thinking in these terms, is <em>very</em> specific. There are many extra features of a d6 that have nothing to do with the randomness of the d6. Every feature of the die except its shape (and mass distribution) can be altered and it would still exhibit the same properties of randomness.</p><p>Modelling systems, therefore, requires a keen eye about what are the underlying mechanics that allow the system to work, and what are the superficial parts of it that happen to be the case in this particular instance.</p><p>At its barest, a d6 is a system that, when run, produces a random integer from 1 to 6. The random distribution is even across all numbers: which is to say, the more times it is rolled, the more we expect to see the counts for each result become equal.</p><p>To model a d6, therefore, we simply need a system that can produce the same result.</p><pre>Math.ceil(Math.random() * 6)</pre><p>This piece of Javascript models a 6-sided die. Run it in your browser's console if you don't believe me. Run it lots. Here's what happened when I ran it 50 times<a href="#footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-1" name="fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-1"><sup>1</sup></a>:</p><pre>[2, 2, 6, 3, 5, 4, 3, 3, 2, 4,
1, 5, 3, 4, 6, 1, 6, 6, 4, 5,
3, 1, 6, 5, 2, 4, 6, 6, 6, 5,
3, 6, 1, 2, 3, 2, 3, 3, 1, 5,
2, 5, 3, 2, 4, 3, 5, 6, 6, 5]</pre><p>And sorted:</p><pre>[1, 1, 1, 1, 1, 2, 2, 2, 2, 2,
2, 2, 2, 3, 3, 3, 3, 3, 3, 3,
3, 3, 3, 3, 4, 4, 4, 4, 4, 4,
5, 5, 5, 5, 5, 5, 5, 5, 5, 6,
6, 6, 6, 6, 6, 6, 6, 6, 6, 6]</pre><p>At this level, Javascript's RNG<a href="#footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-2" name="fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-2"><sup>2</sup></a> should be roughly uniform in distribution, and with true randomness we should not expect uniform results at such small quantities. This distribution certainly seems random and within parameters for uniform distribution, so we've simplified the concept of a d6 into a <em>minimal and sufficient</em> algorithm.</p><h2>d<em>n</em></h2><p>Not all modelling is about functionality. Much of data modelling is about just that: data!</p><p>A model like a d6 is fundamentally fairly useless. Indeed the idea of a d6 is just a very tight constraint on a very useful concept - randomness. It serves little purpose to model a d6 <em>specifically</em>, because the number of uses for a d6 is, in the grand scheme of things, small.</p><p>In the real world, we use models in computing for two basic purposes: retrieval and prediction. The first one is used to store representations of things that exist, such as people or products. Those are <em>data models</em>. We store these data models to let people log into a system, or to display a list of the products to customers. The second is used to try to work out what would happen in certain situations, based on the understanding that we have about the system in the first place - such as weather. These are <em>functional models</em>, of which the d6 above is one example.</p><p>In both situations the model is useless without the things being modelled having <em>data</em>. Properties of the objects store information about the objects and supply parameters to the algorithms we've devised.</p><p>We have hit upon the idea of <em>parameterising</em> algorithms. As noted, the d6 algorithm is somewhat useless because <em>all it does</em> is model a d6, which is of limited utility.</p><p>We can increase the utility by modelling the algorithm of <em>any</em> die. This is the second thing to be aware of when learning to abstract away the fundamentals from the real-world example. Earlier, we learned that we can turn a gazillion atoms' worth of die into a few electrons' worth of RNG by simply taking a number between 1 and 6 - this is the fundamental behaviour of a d6.</p><p>Now, we can look at other real-world dice and see how their behaviour relates to the d6:</p><ul><li>A d4 picks a number between 1 and 4</li>
<li>A d6 picks a number between 1 and 6</li>
<li>A d12 picks a number between 1 and 12</li>
<li>A d20 picks a number between 1 and 20</li>
<li>A d100 picks a number between 1 and 100</li>
</ul><p>It doesn't take a complex neural network to see the pattern here. A d<em>n</em> picks a random number between 1 and <em>n</em>.</p><p>If we wanted to model a d4 we could amend our d6 model:</p><pre>Math.ceil(Math.random() * 4)</pre><p>And we're done. Well done! You've invented job security. Now we've got two models for two different scenarios, and we know how to repeat the process for any die we like.</p><p>You should at least by now have the feeling I'm leading you to a point; and if you haven't guessed it yet I'll make the point.</p><p>We haven't modelled the <em>pattern</em>.</p><p>You can model dice until you're blue in the face but a good model captures the <em>fundamental principles</em>. The d6 model captured the fundamental principles of a d6, but we want a model that captures the fundamental principles of <em>all dice</em>. We need to model the <em>abstract</em>; the <em>pattern</em> that we spotted when we listed our dice.</p><h2>Abstraction</h2><p>"Abstract" is another one of those words that no one understands until they're faced with it, and then it confuses them until they understand it, and then they realise why it's been used all along. Most people know abstract as a form of art, and therefore associate it with meaningless shapes and random colours or something.</p><p>The abstract of something is those features about the thing that remain behind when you take the actual thing away. The abstracts are those <em>conceptual</em> things that mean you can describe it without actually having one; but which, if you had never seen one, would mean you may recreate a different thing.</p><p>This is what we did with the d6. We took the <em>abstract</em> concept of a d6, which is to randomly generate a number between 1 and 6, and then we recreated it in an algorithm that looks <em>nothing like</em> a die. It's a string of characters on a screen, now. It doesn't even roll. Or bounce.</p><p>Abstracting across many things is an art form in itself. For a start, the things have to be related, or else there's no real abstraction to make. Secondly, the degree to which things are actually related to one another can vary wildly, so knowing what level of abstraction to make is also a challenge. Thirdly, abstractions themselves may be similar; in which case you can start relating things that look the same in the abstract but are entirely unrelated in real life.</p><p>Now that I've thoroughly lost you, let me bring you back to earth. When we laid out all the dice we know and examined how they work we saw a pattern, which is that a die with <em>n</em> sides is an RNG between 1 and <em>n</em>. A pattern is something we can model; we model it with parameterisation.</p><p>Parameterisation is when you take a series of concrete examples and you remove one of the things from it and replace it with a variable; in this case, we replaced all the numbers with <em>n</em><a href="#footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-3" name="fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-3"><sup>3</sup></a>. The multiple types of die have been reduced to a single type, whose number of faces is now variable.</p><p>The number of faces the die has is now a property of the die. We have a model with data!</p><p>How do we represent it? Well in Javascript terms, parameters are given to functions, and objects have properties. We can divide the model into the two parts, functionality and data, by using a function to represent rolling a die and an object to represent an actual die.</p><pre>function rollDie(die) {
return Math.ceil(Math.random() * die.sides);
}
var d6 = { sides: 6 };
var d12 = { sides: 12 };</pre><p>Here we have one function that will roll a die and return the result. Then we have two dice, each of which is a simple object with the property <code>sides</code>. Inside the <code>rollDie</code> function we use the <code>sides</code> property of something called <code>die</code>, which we can see is mentioned in the parentheses in the function definition. This together means that whatever is given to <code>rollDie</code> is assumed to be a model of a die, and to have a property <code>sides</code> that represents the number of sides it has.</p><pre>rollDie(d6);
rollDie(d12);</pre><p>If we provide a die model as a parameter to the rolling function, the rolling function can inspect the property of the model, extract the data, and use the data in the original algorithm. The algorithm has not, fundamentally, changed. It is simply the case that now it is parameterised; which is to say that instead of duplicating the function for every possible invocation, we can create data models that represent the thing we are dealing with, and provide the data to the function. We have abstracted the pattern (1d<em>n</em> returns a number between 1 and <em>n</em>) by making the variable, <em>n</em>, well—variable!</p><h2>Verbs and nouns</h2><p>The world is made of verbs and nouns. Systems verb nouns. People roll dice. People buy products. Computers authenticate passwords. Ecommerce systems suggest related products. Search engines search documents. URLs refer to resources.</p><p>Our data models therefore comprise verbs and nouns. Our d6 model was a verb<a href="#footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-4" name="fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-4"><sup>4</sup></a>, but the noun was <em>hard-coded</em>. Hard-coding is the failure to parameterise. Instead of accepting a parameter, the noun - d6 - was assumed by the verb, because the verb was the whole of "roll a d6".</p><p>Our later model had a verb, <code>rollDie</code>, which could roll any noun that looked like a die. It had two dice, <code>d6</code> and <code>d12</code>, which represented 6- and 12-sided dice, respectively. But the <code>rollDie</code> verb did not <em>rely</em> on those dice. The verb was <em>abstracted</em> from the nouns because with the new verb, anyone can create a die of any size and roll it:</p><pre>var d27 = { sides: 27 };
rollDie(d27);</pre><p>... so long as they have access to the <em>verb</em> part - the functionality - of our model.</p><p>By parameterisation we can turn a verb into a verb and a noun - "roll a d6" turns into "roll" and "a d6". By doing the opposite, we can turn a separate verb and noun into a single verb. Good modelling comes from learning when it is right to include the noun in the verb, and when the noun is a parameter. In some cases, the noun is fetched from somewhere else - a different verb (to fetch) and a different part of the model, with its own nouns.</p><p>In the real world, computer modelling is much more involved than this. Data are often linked to other data, such that if one changes another must reflect it. A shopping basket, for example: if you add an item to the basket, the total must increase. If you change the quantity of an item, the subtotal for that item must increase, and so must the basket total.</p><p>In that example, we already introduced nouns and verbs that we can model. Basket; item; total; subtotal; quantity. Some of these are things, and some of them are properties. Some are both! Items are real things, but the list of items is a property of the basket. The total is a property of the basket, and the subtotal is a property of the item when in context of a basket and having a quantity!</p><p>Sometimes we replace nouns with verbs: instead of storing the total, we may choose to calculate the total on demand based on the items.</p><p>Sometimes we replace verbs with nouns: when you roll a die, its value remains the same until you roll it again, but you should be able to ask it what value it shows. Our model could not do this. Alas! Our simple and sufficient model is no longer sufficient.</p><p>Sometimes we separate a verb into a verb and a noun: we turn rolling a d6 into rolling, and create a d6 to roll. This allows us to either roll a different die, or do something different to the die.</p><p>Sometimes we combine a verb and noun into a single verb: when we get the total of a basket, we don't separate it into "get" and "total"; if you change the noun here, the verb makes no sense!</p><p>Even a simple example like a die can escalate, and it is easy to get overwhelmed by the interactions—imagine the complexity of a "simple but sufficient" model of an entire shop!—but ultimately we are modelling <em>nouns</em> and <em>verbs</em>; all we have to do is parameterise correctly and find the correct abstractions.</p><h2>Modelling systems</h2><p>Hopefully you will have, by means of a concrete example and a lot of nebulous ideas, some concept of what it is to model things in computer systems. Ultimately, you will need some way of defining functions - a programming language - and some way of storing data - maybe a database.</p><p>Modelling a system therefore involves a good eye for what is a verb and what is a noun. That is to say, if you want to "roll a d6", does this suffice as a verb? Or is "d6" a noun? What if you want to "calculate the total"?</p><p>There is no cheat sheet here. Experience is your best recourse. But perhaps we can jot down some things to consider when modelling a system.</p><ul><li>How big is the system? The d6 system was small, but the shop system was large. Can it be smaller systems?</li>
<li>How big are the nouns? A d6 has 6 faces, but the number 6 is enough to model that. Meanwhile, a basket has many items, but more information is needed; items are separate things, but faces are not.</li>
<li>Can you de-noun your verb? Does the verb make sense on other things? Does it <em>actually</em>? You can roll anything with sides; but can you get something other than a total from a basket? Can you get a total from something other than a basket?</li>
<li>Can you combine a verb and noun? Have you gone too far parameterising? If your shop has only one basket, the basket is not a parameter: the verbs can assume it.</li>
<li>Can your verb fetch a parameter, instead of accepting or assuming it? When you roll a die, perhaps you can establish elsewhere which die you are rolling. Perhaps the items on a basket know they are items; and there is only one basket, so you can get the items when you need them.</li>
</ul><p>That's all for now on models. In future posts we will take a look at how data get around inside these systems, how we store them, and the transient nature of data while the system is actually running.</p><p class="footnote"><a href="#fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-1" name="footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-1"><sup>1</sup></a> <code>var a = [], i = 0; for (i = 0; i < 50; i++) { a.push(Math.ceil(Math.random() * 6)); } a; </code><br />
</p><p class="footnote"><a href="#fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-2" name="footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-2"><sup>2</sup></a> Random number generator<br />
</p><p class="footnote"><a href="#fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-3" name="footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-3"><sup>3</sup></a> Replacing all the <em>d</em>s with <em>m</em> may be a tempting thing to do here, but we shouldn't. That's because <em>d</em> has been constant across all of our examples; it simply serves to refer to the thing we are modelling in the first place. <em>n</em> is the new <em>variable</em>, because the thing it has replaced <em>varies</em>. <em>d</em>, being constant, is the thing our model is taking away entirely! It serves no purpose to know that we are rolling dice, any more; the <em>d</em> is therefore simply our reminder about what we are aiming for.<br />
</p><p class="footnote"><a href="#fn-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-4" name="footnote-1cc64aceeaa508bfdc4f2f69f5e36c8324b85eaa-4"><sup>4</sup></a> Commonly one would not copy-paste an algorithm into a console and run it. Instead, the algorithm would be packaged in a function and the user would be told to run the function. We did this later, when we parameterised, but to simplify and save on explanations, we avoided using a function in the first examples.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-74942397543178053902014-01-23T20:30:00.000+00:002014-01-23T20:30:14.058+00:00Declaring your intent<p>In Perl it is necessary to declare a variable with <code>my</code> (or <code>our</code>) before using it. This behaviour is enabled with the <code>strict</code> pragma; and recently it has become the default.</p><p>Why?</p><p>Today's theme explores the idea that, when writing code, there is meaning in every statement. A good portion of code will comprise statements that actually implement the logic that causes the program to do what it does; but often overlooked are the statements such as these <code>my</code> and <code>our</code> declarations, which explain your intention for the variable before it's ever even used.</p><p>We'll look at some of the simpler reasons behind it, and later on we shall look at the less apparent ones.</p><h2>Requesting</h2><p>In these cases the intention you are declaring is simple: "I want to use this symbol."</p><p>The humble typo is the most obvious reason espoused for requesting new variables: it stops you using something else. But in Perl this actually covers at least three separate types of typo, all of which are solved by declaring things before you use them.</p><h3>Misspelling it later</h3><p>Misspelling the variable later on is the most common failure.</p><pre>my $hard_to_spell_name;
$hard_tp_spell_name = 'cats';</pre><pre>Global symbol "$hard_tp_spell_name" requires explicit package name at script.pl line 3.
Execution of script.pl aborted due to compilation errors.</pre><p>Saying you want to use symbol A and then using symbol B is an error it is trivial to pick up on.</p><h3>Misspelling it now</h3><p>This is less common because you usually spell the variable name right when you create it because you've just spent ages trying to come up with the name in the first place. It's the same declaration, except you meant B and B, rather than A and A.</p><pre>my $hard_tp_spell_name;
$hard_to_spell_name = 'cats';</pre><pre>Global symbol "$hard_to_spell_name" requires explicit package name at script.pl line 3.
Execution of script.pl aborted due to compilation errors.</pre><h3>Forgetting</h3><p>This requires a module, but declaring your intent allows the <code>warnings</code> pragma to tell you when you didn't use a variable you asked for.</p><p>Install <code>warnings::unused</code> from CPAN in <a href="http://cpanmin.us">the usual way</a>.</p><pre>use warnings::unused;
use strict;
use warnings;
my $foo;
my $bar = 'cats';
say $bar;</pre><pre>Unused variable my $foo at script.pl line 5.</pre><h3>Typing</h3><p>By this I mean the type of the variable, not the typing you're doing when you make a typo.</p><p>In this case, you've declared an array and then accidentally used a scalar, or forgotten it's not an arrayref, or something along those lines. This is also the sort of protection you get from languages with a more C-style typing system, where you have to declare a variable by defining its symbol name and its type (<code>int i;</code>). Basically even though you spelled the symbol name right, you're using it wrongly.</p><pre>my @array_of_cats;
push @$array_of_cats, 'cat';</pre><pre>Global symbol "$array_of_cats" requires explicit package name at script.pl line 3.
Execution of script.pl aborted due to compilation errors.</pre><p>"You're using it wrongly" is a perfectly reasonable statement here. That's because <em>you declared what "right" is</em>: "wrongly" is directly determined by your own <code>my</code> statement.</p><h2>Overwriting</h2><h3>Reuse</h3><p>If you are required to declare your variables the first time you use them then you will always do so. This means that the keyword <code>my</code> is not only used to declare that a variable is supposed to be available, but also to declare that the variable is supposed to be <em>new</em>.</p><p>Hence, if you try to introduce a variable that already exists, it tells you off, and thus you avoid clobbering an existing variable.</p><p>This behaviour is actually only a warning, so comes from <code>use warnings;</code> rather than <code>use strict;</code>. However, it is still a result of <em>declaring your intent</em>.</p><pre>use strict;
use warnings;
my $cats = 'cat';
my $cats = 'horse';</pre><pre>"my" variable $cats masks earlier declaration in same scope at script.pl line 4.</pre><h3>Clobbering</h3><p>It is easy to forget that the use of <code>my</code> and <code>our</code> produce <em>lexical variables</em>. These are variables that are only visible within the block in which they are defined (treating a file as a block for this definition).</p><p>With <code>my</code> you simply cannot clobber this variable from anywhere else. It is either a compiler error, or a different variable.</p><pre># This sub is useless and does nothing
sub one {
my @cats;
push @cats, @_;
return @cats;
}
# This sub can't see @cats from the other sub!
sub two {
push @cats, @_; # line 10
return @cats;
}</pre><pre>Global symbol "@cats" requires explicit package name at script.pl line 10.
Execution of script.pl aborted due to compilation errors.</pre><p>Or:</p><pre># This compiles, but is a new, separate array of cats.
# It is fractionally more useful than sub one.
sub two {
my @cats = ('default_cat');
push @cats, @_; # line 11
return @cats;
}</pre><p>A bonus of <code>my</code> is that when the block has executed, the variable is tidied up. That is, it falls out of scope. This also works in loop bodies, allowing you to trash and recreate data in every iteration by putting a <code>my</code> line inside the loop.</p><pre>package Cat {
my @cats;
# Both of these use the same @cats - the one above!
sub one {
push @cats, @_;
return @cats;
}
sub two {
@cats = ('default_cat'); # whups, overwrote the whole set!
push @cats, @_;
return @cats;
}
}
@Cat::cats = ('cat_one', 'cat_two'); </pre><p>Here, <code>@cats</code> is available to be clobbered anywhere in the Cat package<a href="#footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-1" name="fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-1"><sup>1</sup></a>. However, because it is <em>lexical</em>, it is only available within that block<a href="#footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-2" name="fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-2"><sup>2</sup></a>. Line 18 <em>appears</em> to be altering the same variable (<code>@cats</code> within the package <code>Cat</code>), but in fact this is creating a new package variable in Cat<a href="#footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-3" name="fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-3"><sup>3</sup></a>.</p><p>The intent of using <code>my</code> to declare <code>@cats</code> therefore is to have a variable available throughout the package, but not to be available <em>without</em> the package.</p><p>There is a subtler declaration of intent. The position of this <code>my</code> statement declares that this variable is intended to be used throughout the <em>entire</em> package; therefore it should be applicable to the majority of the behaviour in the package. Were this not the intention, the <code>my</code> statement could be put in a block that encapsulates the variable and any places it is supposed to be used.</p><p><code>our</code> is a similar beast, but it adds the ability for outsiders to also alter the variable, so long as they do so explicitly. The following code differs only in the use of <code>our</code>:</p><pre>package Cat {
our @cats;
sub one {
push @cats, @_;
return @cats;
}
sub two {
@cats = ('default_cat');
push @cats, @_;
return @cats;
}
}
@Cat::cats = ('cat_one', 'cat_two'); </pre><p>Now, the variable <code>@cats</code> inside the package's block can also be accessed as <code>@Cat::cats</code> from outside of it. This is the <em>intent</em> you declare when using <code>our</code>.</p><p class="footnote"><a href="#fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-1" name="footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-1"><sup>1</sup></a> Normally, the package would be defined in its own file, but this format is common for single-use packages, especially in tests.<br />
</p><p class="footnote"><a href="#fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-2" name="footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-2"><sup>2</sup></a> When the package is defined in its own file, the file itself is the scope for such variables.<br />
</p><p class="footnote"><a href="#fn-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-3" name="footnote-c3bfbab59487f0de424a6e5c4842e4014b07f6dd-3"><sup>3</sup></a> The reader should be aware that this is the reasoning behind the message <code>Global symbol "$foo" requires explicit package name</code> when strictures tells you off for an undeclared variable. Any variable name can be used, so long as it explicitly declares a package name like in this example. The difference between a <em>lexical</em> variable and a <em>package</em> variable is not in scope of this blog post.<br />
</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0tag:blogger.com,1999:blog-4857673566967429385.post-19427581298049756342013-10-18T16:05:00.000+01:002013-10-18T16:05:00.270+01:00Fixing PHP<p>PHP is not a bad language. </p><p>Come back. Let me rephrase that. </p><p>PHP is a terrible implementation of what under the surface is a perfectly adequate, dynamic scripting language. Unfortunately it is implemented as a poorly-thought-out, logically bereft templating language, peppered with pitfalls and irritating inconsistencies. </p><p>But it can be fixed. It can be fixed with some simple, non-backwardly-compatible, sensible, welcome-to-the-real-world, feasible alterations. Let us begin. </p><h2>1. Get rid of <?php ?> </h2><p>The fact that PHP used to be a templating language is archaeologically apparent in this vestigial remnant from a bygone era. These tags are still all over the place because PHP is trying to be two things at once: both a templating language and a scripting language. </p><p>Once you grow up (or metastasise) and become a real language, you have to put away childish things. </p><p>These break-in-break-out tags were fine when PHP was designed to be parsed by a Perl script and run as a simple if-this, for-each-that dynamic HTML page generator. They remain fine, if you want to use PHP as the templating language it is. But if PHP wants to be taken seriously, the first thing it needs to do is stop hanging on to that I-can-do-templates-me attitude, and hand over to one of the <a href="http://twig.sensiolabs.org">many</a> <a href="http://www.smarty.net">modern</a> <a href="http://template-toolkit.org">alternatives</a> that have come along since the Internet was still finding its feet. </p><p>In fact there's no real reason PHP should not remain a templating language. After all, <a href="http://www.masonhq.com/">Mason</a> (and indeed Template Toolkit) allow you to inject actual Perl into your web templates for those times when you simply can't be arsed to abstract your logic to where it's supposed to go. However, if PHP is going to behave like this, it needs to understand there is a difference between a PHP template and a PHP script. </p><p>Therefore I propose </p><h3>1a. Create <code>.php</code> and <code>.phpt</code> file types </h3><p>Or suchlike. <code>.php</code> files would naturally be PHP scripts and do away with that ridiculous <code><?php</code> header that persists throughout PHP projects like a blight. <code>.phpt</code> or suchlike would be recognised as text files containing PHP segments, and they can use the old break-in-break-out paradigm to inject program logic into the template. </p><p>Of course it is not recommended in Mason or TT2 that you use actual Perl in your actual templates, because then the temptation is just to merge your views with your controller logic, and then you get into a Right Mess. Better would be simply to have a PHP port of TT2 or Mason, or use Twig or Smarty, and allow those to have their own this-bit-is-PHP-and-I'm-sorry directives. </p><h3>1b. Make it a decent templating language too </h3><p>It's a bit of an issue that PHP is stupid, as well. Modern templating languages offer myriad text processing options as part of the language itself. An example is the way Template::Toolkit allows you to filter output text through, e.g., the HTML filter, sanitising the data just before it's output.</p><p>PHP's best answer to this so far is user-written PHP classes that render PHP templates (two entirely different things written in the same language) by sanitising the data assigned to them at some time or other just before the template file itself is actually rendered.</p><p>That's just one example. PHP is not really a templating language any more either, because templating languages have evolved past the very basic output-string behaviour that PHP was originally tasked with. PHPT would need to catch up as well, and separate itself from PHP proper.</p><h2>2. Stop pretending everything is an HTTP request </h2><p>That PHP never left its template roots shows when you try to write command-line interfaces into your business software. You realise that you've been assuming throughout the code that the <code>$_SERVER</code> variable actually contains a URI of some description; that there's a protocol; that you're outputting HTML. </p><p>As soon as the first file that started with <code><?php</code> and didn't contain a <code>?></code> was created, PHP was broken. As soon as you create a file that contains utility functions, or classes, you have a file that you can run <em>without a webserver</em> . As soon as you have <em>that</em> , you have a scripting language. That was the point at which people should have stood back, taken a look, and dived in to PHP 4 or whatever with the attitude that this time we're going to do it right. </p><p>No one did. </p><p>PHP still outputs HTML whenever it feels like it - see <code>var_dump</code> . It still has global, HTTP-centred variables. It doesn't do exit codes properly. The fact that <code>exit</code> and <code>die</code> are the same damn thing just shows that someone somewhere has completely misunderstood the point of these things. Heck I don't even know whether error messages actually go on stderr. </p><p>At about the time PHP was swapping its soft teething toy for its first big-boy spoon, the rest of the world was discovering that if you interface your HTTP server with your scripting language via stdout, you can maintain a separation of interests wherein your entire business logic is a collection of useful modules or classes or whatever, which when used in a web environment can be wrapped in an HTML layer and called a website - the layer being swappable for a CLI one that outputs the same information in a salient format. Or a JSON one, for public APIs, or even private, socket-based APIs that don't touch either HTTP or even TCP! </p><p>Nope. In PHP's land of unicorns and rainbows the whole world is an HTTP request. The world springs into existence when the request begins and disappears when the response is sent, and if anything happens to be left around since the last universe's brief lifespan came and went then that's just something we have to deal with as part of our new one. Trying to leverage command-line support, or non-HTTP support, into this assembly of spit and chewing gum is baby's first steak knife to PHP. </p><h2>3. Use your own exception mechanism </h2><p>Nothing is as irritating while working with PHP as when it throws its toys out of the pram. Now, I'm quite happy to accept that a parsing error is completely unrecoverable, but that is it, and absolutely it. Anything and everything that happens at runtime should be tryable, and anything that ever goes wrong should be catchable. </p><p>This expected feature of the language should not be taken as a comment on the sense in doing so. Trying to call <code>$app->run()</code> and catching it when it fails is going to be a bit less useful than letting it fail and tell you what was wrong. </p><p>But being <em>able</em> to catch it - now that's a tool we need. Since the original error mechanism was put in place a new, superior nonlocal return is available, and one which puts control in the hands of the user (without horrible <code>set_error_handler</code> hacks). Might as well use it. </p><h2>4. Tidy up the root namespace </h2><p>We get it. You like functions. Well, take stock and look around you. Not only have you implemented exceptions and then completely failed to use them, you've also implemented classes, interfaces, namespaces, closures and traits and failed to use <code>those</code> as well! </p><p>Right. For a start, having all those functions is confusing because there's no consistency in them. I'm not going to rewrite the entirety of <a href="http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/">A Fractal Of Bad Design</a> , but I'm going to borrow from it here. Some of the functions have underscores, some don't ( <code>strpos</code> / <code>str_rot13</code> ). Some take arguments one way, some the other ( <code>array_filter($input, $callback)</code> / <code>array_map($callback, $input)</code> ). Every time we use a built-in function we have to look up how it's spelled and what order the arguments are in and there are so. Damn. Many. </p><p>Secondly, certainly PHP has to lookup every called symbol in both the user's own symbol tables as well as the language's. That sort of thing is surely expensive, especially if this language is aimed at beginner programmers who are only ever going to use 10% of the functions 90% of the time. </p><p>Thirdly, every single built-in function or class is just another name that the user can neither use for their own functions nor override to replace. Sure, PHP has modules that you can jump through hoops to install at the C level, but who needs that? </p><p>All of this might be forgivable if this overabundance of global functions covered literally every possible operation a user could conceivably want; but it doesn't! Worse still, a majority of them can trivially be abstracted into one generic function that takes a callable. All the <code>array_*</code> functions, for example: the sort functions are all just user sort with different sort procedures passed in. The filter functions are all the same with different identity functions passed in - and, for a specific example, recently I needed a version of <code>array_search</code> that took a custom identity function! How dare I want the key of a value that has a sub-value that matches my input! PHP says I may not do that and therefore I may not do that.</p><p>Ridiculous. The fact the PHP team haven't abstracted this stuff sensibly does not speak in favour of their ability to write the code behind PHP in the first place, does it? It doesn't take a genius to tidy all this up, and yet no one has - nor has anyone written the tidied version alongside. That attitude of constant implacability hurts the language and the community and the reputation of the people behind it, and damages confidence.</p><p>Hypothetical inefficiency aside it's just poor maintenance. The language has a mechanism by which to automatically find class files when a non-existent class is requested. So, put all the less-common functions in autoloaded classes and put those classes somewhere discoverable. Everyone else is modular these days. Is it stubbornness or incompetence that's leaving PHP behind?</p><p>Also, quit adding useless prefixes or suffixes to your functions. I know you're going to push onto an array because you push onto arrays. So call it <code>push</code> , not <code>array_push</code> . </p><p>Also also, don't fob us off with <code>mb_</code> crap. Fix your Unicode. There's no excuse whatsoever for a language prevalent in the 21st century to be coded by people who can't cope with Unicode, or its various representations. I know, it's hard. Writing a language is hard. If you can't, don't. </p><h2>5. Expressions, for the love of god </h2><p>PHP's compiler is apparently written by chimps. Do we still really believe that there is a difference between a statement and an expression? Do we really still have to have "language constructs" (PHP's term) that are parsed and treated differently from any other expression? </p><p>No. Maybe back in the stone age we did things that way but here in the age of enlightenment we have come to realise that the only real difference between a statement and an expression is that a statement actually has a persistent effect. </p><p>In PHP, for example, the <code>x or y</code> construct has become possible. Except when <code>y</code> is not an expression - which is 90% of the bloody language. <code>return</code> is not an expression. <code>continue</code> is not an expression. <code>die</code> is not an expression, but it is special-cased to work with <code>or</code> , and has been since before we even had the <code>x or y</code> construct in the first place. Because Perl did it. <code>exit</code> is not an expression and does not have the same special-casing in the language that <code>die</code> does, even though it is <em>the exact same thing</em> . </p><p>Another example. Normally, <code>()</code> is used to group things, i.e. to override precedence. I'm quite OK with the way it's required for function calls, conditions etc. In PHP, however, these seem to form a magical, ref-breaking construct that is parsed under its own rules. That is to say, in PHP, <code>$a</code> is not guaranteed to be the same as <code>($a)</code> . That's because PHP is a language whose every feature is a special case in the parser. If <code>$a</code> is a ref, <code>($a)</code> is not any more. </p><p>So what's the point of all these examples? Well hopefully they all bring up the obvious question: why? Why are these things different? For a given X, why does the way you use X have to be allowed by the compiler? </p><p>A language built out of expressions is obvious - expressions are what make the operands to operators. And an operator is itself another, larger expression. Suddenly the parsing should seem trivial; you look at a line of code, decide which operators and expressions it contains and run them in a well-defined order. You can see it in the language that when you use an expression it behaves exactly like you'd expect any other expression to behave. At least, it <em>compiles</em> like that - runtime behaviour may be bizarre. </p><p>It's trivial to draw up a simple table of PHP's main features in terms of expressions; in all of this the reader is invited to consider in what situations these do not work in PHP's current implementation, and what it means about the compiler for that to be the case. In the table, X and Y mean any expression, i.e. literally anything that compiles. </p><table><tr><th>Construct </th> <th>Meaning </th> <th>Examples </th> <th>Notes </th> </tr>
<tr> <td><code>${X}</code> </td> <td>The value referred to by X </td> <td><code>${$foo} # $$foo</code> <br />
<br />
<code>${f()}</code> <br />
<br />
<code>$a = &$b; ${$a}</code> </td> <td>When X returns a string, look up that variable. Otherwise, treat it as a reference. When X is another variable, the <code>{}</code> can be omitted. </td> </tr>
<tr> <td>X [Y] </td> <td>Return the element Y from the array X </td> <td><code>$array['foo']</code> <br />
<br />
<code>f()['foo']</code> <br />
<br />
<code>x()[y()]</code> <br />
<br />
<code>['a', 'b', 'c'][0]</code> </td> <td>This implements the "feature" that is "special" in PHP 5.5 of array literal dereferencing (example 3) </td> </tr>
<tr> <td>X() </td> <td>Run the closure X </td> <td><code>f()()</code> <br />
<br />
<code>$x()</code> <br />
<br />
<code> ['a' => function() {}, ...][$x]($y) </code> </td> <td>Actual functions like <code>f()</code> are separate, since <code>f</code> is not a valid expression. </td> </tr>
<tr> <td>X or Y </td> <td>If X is false, run Y </td> <td><code>$type = $config['type'] or continue;</code> </td> <td></td> </tr>
<tr> <td>X and X </td> <td>If X is true, run Y </td> <td><code>$val = $config['x'] and return $val;</code> </td> <td></td> </tr>
</table><p>The reader should take away from this at least the awareness that all of the examples in this table would <em>already work</em> if PHP used a proper expression-based grammar; but instead we have been sold these things piecemeal over the past few versions as new features important enough to go on the front page of the release notes. </p><h2>6. Complete the complement of magic methods </h2><p><code>__toString</code> is a pretty good method. It uses an established consistent convention that double-underscore means special-to-PHP. It uses dynamic dispatch so that if it exists it's used, and if it doesn't there's no "default" behaviour - it just complains. </p><p>There are also <code>__isset</code> , <code>__set</code> , <code>__get</code> etc. These do what you'd expect: test for setness, default setter, default getter... </p><p>Where's <code>__toInt</code> ? <code>__toFloat</code> ? <code>__toArray</code> ? Why is <code>__toString</code> represented and not the others? Furthermore, if you can use a string as an integer and only complain <em>after</em> this conversion, why don't you use <code>__toString</code> first and then try to turn the result into an integer? </p><p>Consistency is paramount in a structured, logical world such as programming. Expectations being formed and then violated is the worst of things. It's the <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">Principle of Least Astonishment</a> . Use it. </p><h2>7. Stop pretending you have types. Or: Have proper types. </h2><p>What in god's name is this? <code>(int) $val</code> </p><p>"Casting," I hear you cry. "It is casting the type of <code>$val</code> to <code>int</code> !" </p><p>"Rollocks," I reply in a PG way. For casting is the act of converting a type through known mechanisms to another type. But we don't have <code>__toInt</code> to convert all possible <code>$val</code> s to <code>int</code> , and we don't have mechanisms to convert all possible types in place of <code>int</code> in the first place. </p><p>Nope, it is another special case in the PHP compiler, where someone saw another language doing something and implemented the same syntax but completely failed to <em>understand</em> what it was doing, and implement the <em>theory</em> rather than the <em>practice</em> . </p><p>What about this? <code>function foo(array $arg)</code> </p><p>"Type hinting!" comes the call from the thousands-strong crowd. But if I ask them to explain this mechanism they roll out the usual approximately-right answers they read in the documentation but cannot <em>explain</em> the concept. </p><p>PHP is a dynamic language; that's one of its strengths. Dynamic means that PHP exhibits certain runtime features that static languages require at compile time. For the purposes of this section the dynamic features we are interested in are: </p><ul><li>Runtime method lookup. If an object can perform a method, the method will be performed. If not, a runtime exception is thrown. Inheritance introduces methods from other classes into the object's symbol table, assisting DRY, but otherwise there is no reason every method could not simply be dynamically dispatched to a function somewhere using magic. </li>
<li>Automatic type conversion. If an operation requires a string and an integer is provided, or an integer and a string is provided, or a string and an object is provided, PHP will transparently perform the conversion at runtime and only complain if it didn't work. </li>
</ul><p>Now apply your theories about type hinting to this. What can it do but cripple PHP's dynamicity? <b>Duck typing</b> is the principle by which, if you have dynamic method lookup, an object only has to <em>be able</em> to perform a task in order to be considered suitable for the task. That is, until runtime, until you actually <em>try</em> to run the method on the object, there is no way to know that the object cannot do it. If there were you would have sacrificed dynamic method lookup for static compilation already. Type hinting for classes is completely non-semantic if you have the option of duck typing, because there is literally nothing special about your particular class that makes it important that an object is of this type. </p><p>How about non-object type hinting? Well you can't actually do that, because <code>int</code> and <code>string</code> aren't types to hint about - probably because any scalar can be used as a string! And any string can be used as an integer! So why enforce the check? Or, from the other perspective, why aren't they types? I can cast to them; why can't I require them? </p><p>And why can I require classes but not cast to them? </p><p>If we look at the whole type system of PHP as a looser concept than PHP makes it, it makes a lot more sense. </p><p>Classes are not some promissory aspect of a piece of data that ensure the datum can perform tasks, but an organisational structure allowing you to introduce functionality from other classes into new ones by inheritance or merging traits. From this perspective, duck typing makes sense - you don't need a <em>specific</em> class to ensure an object can perform tasks; any class can theoretically do it, especially if it consumes a trait that provides it. Type hinting for classes, from this perspective, is <em>logically inconsistent with traits</em> - which are considerably more useful - because you can't test for <em>what a class can do</em> , which is the only thing that's important. </p><p>Similarly, basic types are not remotely based on reality either: even if you <em>could</em> ask for a string or an integer, assuming we get the rest of the family of magic methods, any object could have <code>__toString</code> or <code>__toInt</code> . And even if we <em>don't</em> get <code>__toInt</code> , a string <em>can be an int</em> . So if you ask for an int, you could give a string, and you won't know the data the string contains are bad until you <em>try</em> to use it as an int. And you should be able to give an object to a parameter that wants an int simply by casting it to a string and then an int - something PHP should be doing for us already. </p><p>Hopefully the reader has spotted the inconsistency between type hinting and a dynamic language: the language cares about what the datum <em>can be</em> , but the type hinting cares about what the datum <em>is</em> . There is absolutely no logical association between what the datum <em>is</em> and what it <em>can do</em> , because Dyamic Point 1 allows for any object - <em>independently of class</em> , thanks to traits and <code>__call</code> - to be able to perform any task; and Dynamic Point 2 allows any type - thanks to <code>__toString</code> and the proposed <code>__toInt</code> and <code>__toArray</code> - to <em>be</em> any other type. </p><p>If you're going to have type hinting, therefore, you have to have statically compiled types: you have to <em>enforce</em> the relationship between type and behaviour; otherwise, your type hints are just extra bytes in a file that are going to appear in a commit log at some point in the future deleted by some frustrated developer trying to implement a trait and use it in a method that doesn't expect it. </p><h2>That's all</h2><p>I'm sure I could find many more examples of things PHP can fix at a basic level and stop being so irritating about simple things. You'll note I didn't complain about the tiresome conflation of array and dictionary, despite it being the biggest misunderstanding in programming history.</p><p>But surely this is a start? We can keep most of the PHP grammar; the syntax doesn't change (much); and so many of the pitfalls and gotchas that a programmer falls into will be resolved in one fell swoop!</p><p>As with many things PHP has reached sufficient mass that nothing important will ever change, because the politics of the mailing lists drag everything down, with half-right people expressing their ill-informed opinions on stuff that really, actually matters.</p><p>And there's the rub; the alternative is to start again. Start a new, similar language, on the right foot. A language that doesn't have those tags; a language that interfaces with the standard streams properly; a language detached from the web server, that doesn't assume a web environment; a standalone, dynamic, modular language, easy to learn, easy to stick together, easy to run on any decent OS and the not-decent one.</p><p>But why? We already have Perl and Ruby and Python. The amount of changes required to PHP means that literally the only reason to improve it at all is that it's associated with the name PHP. Installing it, upgrading it; these things would take an identical amount of effort as simply using an alternative. It wouldn't be sufficiently backwardly compatible that existing PHP code would run, because all the crap you have to do in existing PHP code wouldn't be possible or necessary.</p><p>It can still be done, though. But it won't.</p>Altreushttp://www.blogger.com/profile/03670647455111183841noreply@blogger.com0