How to Rant About Languages

Opinions are great. They make the world go around. Everyone hates things they don't understand, unless they are intrigued, in which case they learn about it instead and then stop hating it because they don't understand it.

However, you can understand something and still hate it.

When constructing a rant, I find there is one key feature it should have that differentiates a valid complaint from an opinionated spiel: if you can replace the subject of the rant with any other subject and not introduce factual inconsistency, your rant is invalid.

Here's a rant about PHP http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/

Here's a rant about Perl http://www.schnada.de/grapt/eriknaggum-perlrant.html ¹

You will see that if you replace PHP with, say Python in the first rant, the rant becomes factually incorrect. Sure, many of the principles remain true - that consistency is good, that lurching obliviously through errors is not a great idea - but the fact that Python does those things is provably false. This makes a good rant.

But if you replace Perl with Python in the second rant you will find that it is true of Python as well. And C. And PHP. And French if you care. This does not make a good rant, because it's boring, and no one can take away anything constructive from it.

Please, if you must rant, make it specific to your hated language. If one of the people who writes the compiler for that language reads it, they should be able to take away from it something that they can fix. If you can't construct a rant about actual, deniable problems with the language, then you don't know enough about the language to have an opinion in the first place.

¹ Actually about PERL, that dot-com-boom all-purpose no-knowledge-required hacker language that attracted all the awful coders like flies around shit and produced some of the most godawful, buggy, unmaintainable tripe only matched by the same principles being core to PHP.

Simple gnome 3 wallpaper switcher

Gnome 3 uses dconf rather than gconf, so gconf-based wallpaper switchers won't work. Luckily, the actual change is only that one line if you do it right.

I created a script that uses a cache of wallpapers. First, here's the script that searches your wallpapers directory for actual images (to filter out stupid Thumbs.db and things like that) and adds them to a file.

#!/bin/bash
find "$HOME/wallpapers" -type f \
  -exec sh -c 'file "$1" | grep -q "image"' '{}' '{}' \; \
  -a -fprint wallpapers-cache-new \
\
&& mv wallpapers-cache-new wallpapers-cache

That outputs a file called wallpapers-cache, and the wallpaper switcher itself uses that:

#!/bin/bash

WALLPAPERS="$HOME/wallpapers-cache"
export DISPLAY=:0

r=`rl -c1 $WALLPAPERS`

gsettings set org.gnome.desktop.background picture-uri "file://$r"

rl comes from the randomize-lines package in Debian-based OSes.

Add the latter to crontab to change your desktop as often as you'd like. Voilà!

Testing for the end of a for(each) loop

One sign of a good templating language is that you can test for the last iteration of a loop without having to count the iterations. PHP is no exception! Assuming you're quite happy with hacky syntax (you have to be if you're using PHP) then you can do this:

foreach ($array as $item) {
  if (each($array) === false) {
    # last iteration
  }
}

You iterate over the original array using each, but ignore the output if it's not boolean false. foreach derps around on the copy of the array, and when the original array finishes being iterated, each returns false! So you basically iterate over the array while you iterate over the array.

In Template::Toolkit you can also test for the last iteration of a loop with loop.last, which is why that is also a good templating language.

In conclusion, PHP is a templating language.

Oh right, leave a comment with your own one for whatever bozo templating language you like :)

In Context

Natural language is contextual. A famous quotation demonstrates this in a way we all find familiar:

Time flies like an arrow; fruit flies like a banana.

Usually we interpret this as "the manner in which time flies (verb - to fly) is akin to an arrow" and "the creatures called fruit flies (noun pl. - fly) like (to find agreeable) a banana". However, we can also interpret the former as "Time (measure chronologically) flies (noun pl. - fly) the way an arrow would", except of course this is nonsense and we don't consider it.

Perl is the same. The three data structures covered in previous posts can be used in all sorts of ways, and Perl will attempt to do the right thing based on context. This is done by having rules about context, and what to do in those contexts.

Time Flies

When we say "time flies" in English we can either mean a) imperative "time", noun "flies" or b) noun "time" verb "flies". In Perl we can differentiate between nouns and verbs because nouns have sigils and verbs don't.

$scalar variables have $ and a singular name because they represent a single thing. @array variables use the @ sigil and have plural names because they refer to more than one thing. %hash variables refer to a set of named things; their names are often singular because either we refer to the set ("The option set %option") or a thing from the set ("The debug option $option{debug}").

Verbs, of course, are subroutines and so have no sigil. That means in Perl we can't misinterpret that:

time @flies;

Notwithstanding the fact that time is already a Perl function for finding the time, we can understand this because flies has to be a noun.

Flies Like A Banana

Context in Perl is not a grammatical thing in the natural-language sense explained above, but it is used to determine what we do with the nouns that we pass around.

A common example is to check for an array to have content:

if (@flies) {
    ...
}

In list context, an array represents the list of scalars it contains, as we saw demonstrated in this post, but in scalar context the array is treated as the number of items in it; i.e. its length.

The test in an if statement is in boolean context. Booleans are truey and falsey values: they are scalar. An empty array in scalar context is zero, which is false; hence an empty array is false.

The while loop also uses boolean, and hence scalar, context. However, the for(each) loop uses list context:

for my $fly (@flies) {
    ...
}

This is sensible because if it used scalar context it would always loop over one thing, which is the number of things in the array. A scalar in list context doesn't really do anything special; it remains that scalar:

for my $fly ($one_fly) {
    ...
}

but it does help with concatenation of several things:

for my $fly ($bluebottle, $mayfly, @various_flies) {
    ...
}

Fruit Flies Like

Surprises crop up when dealing with context once operators and functions get involved. For example:

my $banana = map { $_->fruit_i_like } @flies;

You might think, maybe this will take the first fruit that a fly likes, or the last, or some data structure representing the result of the map. Well the latter is the closest: the data structure returned is in fact an integer representing the number of fruits the flies, in total, liked. The context in this statement is in the = operator: it is determined by the thing on the left and it is enforced on the thing on the right. perldoc says that map in scalar context returns the length of the constructed list.

We can take the first fruit liked by a fly by putting the = operator in list context. In this case, that is done using parentheses:

my ($banana) = map { $_->fruit_i_like } @flies;

Parentheses only create list context on the left of =; as we learned in this post, parentheses on the right only serve to override the precedence of operators.

Of course, assigning to an aggregate type is also list context:

my @fruits = map { $_->fruit_i_like } @flies;
my %likes = map { $_->name => $_->fruit_i_like } @flies;

(In this latter example we assume fruit_i_like returns only one item; otherwise the constructed list will break the expected hash construction.)

Another common confusion is when the operator or function itself enforces a context:

my $random_fly = $flies[rand @flies];

The rand function returns a random number; if it is provided with an integer it will return a random number between 0 and that integer, including 0 and excluding the integer. Normal behaviour for a function is to use an array as a parameter list, but rand overrides this by enforcing scalar context on its arguments.

That means that the array given to rand is evaluated in scalar context, giving its length. The output is used in [] to index the array; array indices are required to be integers so the decimal return value from rand is truncated. Since rand can never return the integer provided to it, the maximum return value from this is the last index of the array, and the minimum is zero.

Enough With The Rearranging Of The Quotation You Cited

The context of any particular part of Perl can be subtle. I will describe a few common situations where context is key to the operation of the script.

if

As explained, if enforces scalar context.

if ($fly) { ...} 
if (@flies){ ... }
if (%like) { ... }
if (grep { $_->likes($banana) } @flies) { ... }
if (my ($fly) = grep { $_->flies } @times) { ... }

All of these are in scalar context. What about the last one though? We know that () on the left of = creates list context; but if() is scalar context.

Context happens in stages, naturally, just as expressions are evaluated sequentially. The = operator is in list context as defined by its LHS. This allows $fly to contain the first result from the grep. The cunning thing is that the = operator itself returns the value on its LHS after assignment: this is evaluated in scalar context and used as the boolean value for the if.

Consider:

if (my @times_that_fly = grep { $_->flies } @times) { ... }

You can expect @times_that_fly to be set and non-empty in the if block, and also for it to contain the result of the grep.

<>

The operation of the <> "diamond" operator is context-sensitive. That means it has different behaviour in list and scalar contexts. In scalar context, it returns one line of input. In list context, it returns all lines of input.

my $line = <>;
my @lines = <>;

As we know, for is a list-context control structure; hence it will read all lines into memory before iterating them:

for my $line (<>)

The while control construct, like if, tests a boolean value, except it does it repeatedly and stops running when it is false. Hence the expression in the parentheses is evaluated in scalar context. So it is idiomatic to see while used instead of for when iterating over input:

while (<>) { ... }
while (my $line = <>) { ... }

But I hear you cry that the line of input "0" is false! But it should be treated fairly! Well for a start it will probably be "0\n", but apart from that, while(<>) is actually magic and comes out as:

while (defined($_ = <>))

and even if you set your own variable:

while (my $line = <>) { ... }
while (defined(my $line = <>)) { ... }

but this is only true inside a while().

$array[rand @array]

We've covered this one but it's still for this list. The rand function enforces scalar context on its operand; hence, instead of the array providing a list of arguments to the function, it is itself the argument, but in scalar context, hence its length is returned. The array subscript [] then truncates the return value of rand to an integer.

=()=

Called the goatse operator by those with a sense of humour that compels them to do so, this is not really an operator at all but a trick of context. You see, not all operators return a count in scalar context.

For example, the match operator m/.../ or just /.../ merely returns true or false in scalar context. With the /g modifier it returns the next match! In what universe, then, can a mere Perl hacker get m/../g to return the number of matches?

The universe with the goatse operator of course.

First we know that () on the LHS of = creates list context. That is all that is required for m/../g to return a full list of all matches. So the match is run in list context and that's what we want. The output of our goatse operator is of course going to be a scalar since we want an integer; this enforces scalar context on the leftmost =, which has the effect of putting the return value of the rightmost = (which is the list created by ()) in scalar context, hence counting it.

my $count =()= /cats/g
my $count = ( () = /cats/g )

@array[LIST], @hash{LIST}

A subtle use of context is the one in array and hash subscripts. The context inside { } or [ ] when accessing elements is determined by the sigil you used on the identifier in the first place. Thus:

$array[SCALAR]
@array[LIST]
$hash{SCALAR}
@hash{LIST}

This can have unexpected effects; a warning is generated if you use the @ form but only provide a single value, but in some cases the context can propagate, for example if you are calling a function or something.

Context and Subroutines

A subroutine is called with the context of the place where you called it. This is often a useful tool, because a subroutine can also interrogate the calling context with the wantarray operator.

When a subroutine is called its return operator inherits the calling context. If it doesn't have a return operator, the usual rules apply about an implicit one.

The main reason this happens is that any basic function call - i.e. one with no special behaviour based on context - should act as though the return value were in the code in place of the function call. Consider:

rand @flies;

sub get_flies {
    # ... generate @flies
    return @flies;
}

rand get_flies();

It is reasonable to assume that, because the function returned an array, the behaviour of the latter rand will be the same as the former, and it is: the return of the subroutine is evaluated in scalar context because rand enforces scalar context.

Usually you want to save the return value of a function into some local variable, though; otherwise you have to run the function twice to use it twice, which is wasteful:

(get_flies())[rand get_flies()]

It is important to be aware of the context in which you are calling your function, especially if the function has different behaviour in different contexts. For example, here we have list context:

localtime(flies());

and here we have scalar context:

rand(flies());

and here we have a syntax error:

time(flies());

because time doesn't take any arguments. By default, all subroutines give list context when called, so:

sub context_sensitive {
    return wantarray ? 'STRING' : qw(TWO STRINGS);
}

sub print_things {
    print $_, "\n" for @_;
}

print_things context_sensitive;

Here the context_sensitive sub is in list context, so it returns qw(TWO STRINGS), which is indeed two strings, as noted by the print_things function, which will print each thing with a new line after it so we can see what's going on. However, this sort of thing can lead to confusion when you don't realise the function is context sensitive:

my $string = context_sensitive;
print_things $string;

The above code exhibits different behaviour from the first, because context_sensitive was called in scalar context and the result of that was given to print_things. This can be surprising, because it means that context-sensitive subroutines are not always drop-in replacements for the variable they were saved to.

The scalar keyword can be used here to enforce scalar context on the subroutine.

print_things scalar context_sensitive;

The scalar keyword can be used in the general case when you want to either override or be explicit about the context of anything.

scalar @array;
scalar $scalar;
scalar sub_call();
scalar grep {...} @array;

There is no real list-context equivalent because in general list context is a superset of scalar context, but as with the goatse operator, there are exceptions. Hence the construct () = can be used to evaluate something in list context when normally it would be in scalar context. The reason this is not common is the return value of this assignment will still be evaluated in scalar context, which in most cases is the same as simply not doing this at all.


Void Context

Sometimes you might get a warning along the lines of "useless use of constant in void context". What does this mean?

The difference between a statement and an expression is action. An expression is anything that returns a value: 5+5, sub_call(), @array ...

A statement is one that performs an action: $foo = 5+5, sub_call(), if(@array).

Not all legal lines of code in Perl are statements:

my $foo = 5+5;   # statement
5+5;  # not a statement

It is this latter line of code that will cause the warning about void context. It should be at least slightly apparent why it is in void context: there is no operator or function call enforcing context on the expression. There is no context at this point! This is called void context; neither scalar nor list context is enforced on the simple line of code 5+5;

Void context is not always incorrect. For example, we know that the = operator returns the value on the left hand side. That means that whenever you perform an assignment, the whole assignment itself is performed in void context. It all works because of the layered nature of expressions and sub-expressions. Consider:

$foo = @bar = baz()

baz() is performed in list context because of the assignment to @bar, but that assignment itself is performed in scalar context because of the assignment to $foo. But the assignment to $foo doesn't cause baz() to be run in scalar context. @bar = baz() happens first, and baz() is in list context. This whole expression returns @bar because that is the behaviour of =. That means the next thing that happens is $foo = @bar, which puts @bar in scalar context and populates $foo. This expression returns $foo, but there is no other expression in this statement. So this assignment happens in void context, which does nothing.

This is also void context:

baz();

You don't get a warning for the simple reason that it is perfectly legitimate to have a function that doesn't return anything. However, you could generate your own warning if your function is useless in void context:

sub baz {
  warn "baz() called in void context" if not defined wantarray;
}

The wantarray operator is undefined in void context.

In Summary and/or Conclusion

The context in which you are working defines the way Perl's nouns are treated. You should always be aware of what context you are in at any one time, and how to recognise context.

An assignment is in the context of the left-hand side.

Tests (while, if, ?:) are scalar context, and for loops are list context.

Array and hash subscripts are in the context of the sigil you use.

Operators may enforce context, and some operators also enforce coercion, in order to make the operation possible. The scalar operator is solely intended to enforce scalar context.

The argument list to functions is in list context. Overriding this is possible with prototypes but not recommended in the majority of cases.

With nothing enforcing context, the expression is in void context. Void context will throw a warning if the expression is a constant.

Functions inherit the calling context for their return statement. The nullary operator wantarray is used to inspect that context.

What do you people do all day?

Change list for last Perl release:

http://search.cpan.org/~flora/perl-5.14.2/pod/perl5140delta.pod

Change list for last PHP release:

http://docs.php.net/manual/en/migration53.new-features.php

What

Understanding and Using map and grep

Edits: There is no strict reason why map should not return a shorter list, so I'm not discouraging it any more.

map and grep are useful tools in Perl. To understand them needs only the ability to follow some logic, but to use them requires an understanding of lists. You should read this post on lists to understand how lists work in Perl. The main thing you should understand from that is how lists exist in memory, and how they are used with relation to arguments to functions.

map and grep are actually operators, but they behave so much like functions that you'll find them in perlfunc. What they do is they take a list of arguments, like functions do, and convert that list into another list. map transforms the input list, and grep filters the input list.

Remember that lists with zero or one item in them are still lists.

Understanding map

To understand map you need to be able to understand the following:

y = f(x)

If you don't, let's briefly explain it. It means that if you do something (f) to a value (x) you will get another value (y). You've probably seen this before in graphs - the straight line formula is y = mx + c; we can see that all of mx + c is a function using x (m and c are constants), meaning that if you feed an x coordinate in, you will get a y coordinate back. We say that y is a function of x.

In the case of map, x will always be a scalar, because of how lists in Perl are lists of scalars. y could be more than one value, this being how your output list can be longer than your input list. f is provided to map.

map is therefore a transformation. Its purpose is to consistently transform your input list into an output list, by providing each item of your input list to your f, and collecting the ys that you get out of it.

map turns this:

map { f } x, x1, x2, x3, x4...

into this:

f(x), f(x1), f(x2), f(x3), f(x4)...

A simple example is mathematical: even numbers. The nth even number is 2n—you learned this in school. If you didn't go to school then you'd better catch up.

That means, for y to be the xth even number, our f is 2x.

f(x) = 2x
f(1) = 2×1 = 2
f(2) = 2×2 = 4
...

That means that, given the numbers 1 to 10 we can find the first 10 even numbers that are greater than 0. If we provide our f to map, and the list 1 .. 10, we should get that.

Defining f(x) for map is a simple case of using the special variable $_ in place of x. For now, we will define our f using {}.

print join ', ', map { $_ * 2 } 1 .. 10
2, 4, 6, 8, 10, 12, 14, 16, 18, 20

This is a good example of what we were learning about lists in the previous post. The join operator joins a list of values using the specified string, in this case a comma and a space. There is no requirement in Perl that this list be either an array or a hard-coded, comma-separated list: it can be anything that, in list context, returns a list¹. That could be a function, or another operator like map or grep.

This example serves to explain in simple terms how some functionality can be routinely applied to a set of values to return a new set of values. There is no requirement that y is different from x after the execution of f(x). For example, you might wish to search a dictionary for a set of words, but the words themselves might be suffering from grammar, and hence have different letter cases. The dictionary itself would be entirely in lowercase, and hence you would want a lowercase edition of your set of words.

my @lc_words = map { lc } @words

In this case, the f() that we're applying to all members of the array is the operator lc, which operates on $_ by default and returns a lowercase version of its operand. If the input word happened to be already lowercase, the output will equal the input. No matter.

List Return Values

If you recall, lists are lists, in Perl. Arrays and hashes are things constructed from lists, and both of these are treated as lists when used where a list is expected. You can see this principle in action in these examples: our first example used the range operator .. to construct the list of integers between 1 and 10, and fed that to map. The second example used the array @words, which we assumed to exist for the example, as the input list.

And then the output list was used in the first example as the argument list to join, and in the second example it was used as the constructor for a new array.

So far we have been returning only one value, but the f we provide to map can return any number values because it is evaluated in list context. By this mechanism you can turn an array into a hash, which is probably the most common use of that.

my @words = qw(cat dog cow dog monkey);
my %words = map { $_ => 1 } @words;

The map block here returns the input word, $_, and 1. Remember that in this post we learned that => is semantically equivalent to , except with the quoting of the word to its left. Remember also that in the same post we learned that a hash is constructed from any even-sized list (or an odd-sized list, with a warning). So this block is returning two items for every one we put in, doubling the length of the list.

The use of the fat comma in a construct such as this is idiomatic: since it has no actual effect on the output of the map block it is really only there to hearken to the usual construction of a hash, which is a list that uses => a lot. The image conjured is of taking each element of @words and turning it into word => 1, joining it all together until we have an even-sized list thus:

my %words = (
    cat => 1,
    dog => 1,
    cow => 1,
    dog => 1,
    monkey => 1
);

Knowing the intermediate steps to get to that may help:

my %words = map { $_ => 1 } qw(cat dog cow dog monkey);
my %words = map { $_, 1 } qw(cat dog cow dog monkey);
my %words = ('cat', 1, 'dog', 1, 'cow', 1, 'dog', 1, 'monkey', 1);

Of course, in a true application, you would not set @words in this manner in the first place. @words will be computed, perhaps from a file passed in by name on the command line, which you have processed and turned into this array.

A little knowledge about hash construction is relevant here to know the actual purpose of this map block. If you understand that a repeated key in the constructing list is simply overwritten by the latest occurrence of that key, you will see that this has the effect of making the input list unique. Although the output list is exactly double the length of the input list, that list is immediately used to construct a hash. The hash itself then undertakes its usual behaviour, and in this example the repetition of "dog" in the original array is overwritten by its last appearance by the construction of that hash - which is fine in this instance because all the keys are associated with the value 1. Thus any repeated word is not counted.

You can also skip over part of the list as well. Perl makes no assumptions on the size of your returned list when running map. You can use this to omit elements from your output list while doubling the rest.

To do that, simply test $_ and return an empty list if you don't want the element:

map { test $_ ? ($_ => 1) : () } @things

You might return the empty list if the test returns true, for example if you are trying to transform elements you have not already transformed and remove those you have:

my %to_cache = map { in_cache $_ ? () : ($_ => transform $_) } @stuff;

For some test in_cache, which returns true if its argument is in the cache, and some function transform, which returns a transformed version of its argument, you can thus construct a hash of transformed stuff ready for caching. Of course, you don't have to double the list; you can use this simply to filter it. But if you're not going to transform $_ then it is more sensible to use grep

When Not To Use map

Now that you have a new toy you might be tempted to use it. It is simple nature. Well as with every tool, it is unwise to use it outside its purpose. There are four principles that you should follow when playing with map:

The Principle of Transformation. You can use map to test each item and conditionally return the empty list, thus shrinking the size of the list. But unless your map block transforms $_ in some other way (by returning an altered version, or a longer list), what you have actually got is grep. In some cases it may be easier to read if you use both map and grep, using the output of grep as the input to map.

The Principle of Immutability. You should never transform the input list. That seems contradictory, but by this I mean never alter $_ directly; you should always return an altered copy. It is not, in all cases, possible to change $_. Only when the input list is an array is it possible to overwrite $_ for all values you get from the list. When it is a list constructed by some other means, even a hash, the input values are often immutable, which means you can't change them so don't try. Always return a copy of $_, with transformations applied to the copy.

The Principle of Brevity. If your transformation is quite long-winded you should either a) put it in a sub or b) use a for loop and push onto another array. map is a construct of brevity; this should be honoured.

The Principle of Containment. If you need to do anything else while iterating over your list other than transforming x into y then you should use a for loop for that. Your map block should have no side effects, which means when the map has finished executing, everything should be the same as it was when it started, except now there is a new list in memory.

The general principle is that you are getting back from map a new list, and leaving the old one alone. Don't run map without using the result!

Understanding grep

grep is an operator that has many similarities to map. It takes an input list, and returns another list. You provide some function f, akin to that described for map earlier, and it is run with each successive item from the list.

The difference is simply that grep returns a list of equal size or shorter than the input list.

The f that you give to grep is a filter, not a transformation. It tests each $_ and returns trueness or falseness:

f(x) ∈ (1,0)

We are not looking in this case to return a new value, y, from our function, but rather any value that is true or not true. Reasonably one could point out that if you are outputting a truth value you are running a transformation. You are: but the result of grep itself is part of the input list. The result of map is the output list.

grep can be written in terms of map:

@filtered = grep { test } @things;
@filtered = map { test ? $_ : () } @things;

A common adage is "spelt grep, pronounced filter". That's a mnemonic, of course; it's pronounced grep.

A traditional use of grep is to find the defined items in an array:

grep { defined } @array

It is not uncommon to have produced an array or list with undefined elements, especially if that list came from a map block that could return undef in some cases. Another common idiom is to map and grep at the same time:

my @phone_numbers = map { $_->{phone} } grep { $_->{phone} } @people;

Here @people is assumed to be an array of hashrefs representing people. The grep filters out those for whom the key 'phone' returns a true value, and the map then returns the actual value.

We can thus draw this distinction between the two: map collects output values, and grep collects input values. You can see this is the case: the map block is assuming that the input list is a list of hashrefs, and extracts strings from them. We collect the list of things that map outputs. For that to work, it means that the output of grep has to be hashrefs; and since the input to grep is hashrefs but the grep block only returns a value from that hashref, we see that the output of grep is just a subset of the input.

When Not To Use grep

grep doesn't have the principle of transformation because you're not supposed to transform the list.

The Principle of Immutability - grep should not alter the input values; it should merely test them.

The Principle of Containment - nothing in the grep block should affect anything outside of the expression. Everything should be the same after the grep has finished running as it was beforehand, except now there is a new list in memory of (copies of) some or all of the input list.

The Principle of Brevity - if you need to do a long-winded process to find out whether or not you want to keep a particular $_, take it somewhere else, because it doesn't belong here. Make a sub and put it in that.

Similar Things

There is a simple concept here: You take a list, you apply a function to each element, and you get another list. map is the simplest example of this because the list you create is exactly the list that map returns. grep is arguably more complex because the list you create with this process (a list of true and false values) is itself used to alter the input list, so you get back a different list from the one you build.

sort

sort is another list operator. Like grep, the list it returns comprises elements of the input list. Unlike grep, the list it returns is the same length as the input list. Clearly, it sorts the input list, and returns a new one with the items in order.

An important thing to have already realised by now is that if you are sorting you have to do two variables at once, instead of one. Perl sorts, haha, this out for you by providing you $a and $b instead of $_. Your task is to tell Perl whether $a is greater than, less than, or equal to $b, by returning -1, 0 or 1 respectively from the block.

The operators <=> for numbers and cmp for strings do this easily for you, but this generalisation of the sorting process allows for you to run $a and $b through any algorithm in order to sort the list.

my @sorted = sort { length $a <=> length $b } @strings;

for

The for loop is much more than it is in other languages, especially when used as a statement modifier. That's when you put it at the end instead of putting it first and using braces. for is the operator you should use when you want to modify the array itself, rather than create a modified copy. Note you can only fully modify arrays and only the values of hashes.

$_++ for @array; # increment every value
$_++ for values %hash; # same
$_ .= "-old" for @filename_backups; # append "-old" to each filename

map is equivalent to doing a for loop over the old array, and pushing things onto a new array:

my @output;
push @output, $_ * 2 for @input;

Presumably map is more efficient, but part of Perl is that you have the option of being more expressive with your code, which is to say that you should write what you mean. If you mean to perform a map operation, use map; otherwise when someone else reads your code (i.e. you, next month) there will be no head-scratching wondering why you used a for loop.

Other functions in List::Util, List::MoreUtils and List::UtilsBy serve to run a function across all items of a list in the same general way, and return either a list or a single value.

List::Util

• first - Find the first item in a list; usually used to find any item that matches the criterion. Often therefore used to find whether any item matches the criterion.
  my $caps_key = first { uc $_ eq $_ } keys %hash;
• reduce - Collapse a list into one value, by repeatedly applying the function. This is called reduction, as in "map-reduce". Note the use of $a and $b, like sort: we are trying to reduce a list into one value so we have to do two values at once, rather than one.
  my $sum = reduce { $a + $b } @numbers;

List::MoreUtils

This module has many more than List::Util so I won't list them all.

• any, all, notall, none - Test the whole list. any is similar to first from List::Util; the difference being that it will return a true value if your search succeeds, whereas first could return a false value if you're looking for false values. These are essentially grep, except they will stop looking if they find the answer - grep will process the whole list in all cases.
  if (any { defined } @things) {}  # if any is defined
if (all { defined } @things) {} # if all are defined;
  if (notall { defined } @things) {} # if not all are defined
if (none { defined } @things) {} # if no thing is defined
• pairwise - This only works on arrays, but it takes one value from the first array and one value from the second, and gives them both to your function. Then it collects the outputs. It's like map, except it does two things at a time.
  my %hash = pairwise { $a => $b } @keys, @values;
my @totals = pairwise { $a * $b } @prices, @quantities

List::UtilsBy

This is a convenience module for all those cases where normally you'd do the same thing to both $a and $b. I will stick to a couple of examples.

• sort_by - Applies the procedure to both $a and $b and compares the results as strings (nsort_by for numbers). Saves typing.
  my @sorted = nsort_by { length } @strings;
my @sorted = nsort_by { $_->mother->mother->mother->age } @people;
• uniq_by - Makes the list unique based on the function.
  my @unique_by_colour = uniq_by { $_->colour } @fruit;

In Conclusion

I hope this has given you, the newcomer to Perl, a good idea of what map and grep do. I also hope it has given you an insight into the general concept of applying a function to a sequence of values, and doing something with the result. It is an operation that is much more common than you'd think, even in real life.

It is the basis of mail merge, for example, where you take a template and a list of names and you put each name in the template and, with the resulting list of letters, print and send them. That's map.

It is the basis of searching, where you have a quantity of items and you're looking for all of those that match a certain criterion. You apply your criterion to each and keep those that match. That's grep.

It is the basis of sorting things by height, or alphabetically, or by the number of times they've won the World Cup: you take the list of things, and two at a time you find out how many times they've won the World Cup and you sort based on that. That's sort (or sort_by or nsort_by).

Examining or modifying a list is an awfully common operation. One thing I cannot teach you, however, is how to recognise when you should use one. Feel free to ask!

---

¹ All right smart arse. Everything in list context is a list, even if it is only one scalar, because that's just a list with one item in it. The point is that this statement is true regardless of the length of the returned list.

Einstein's Constraint: Booleans

Everything should be kept as simple as possible, but no simpler.
Albert Einstein

Perl is a language that combines ideas from many other languages. It is a language designed by a linguist, and hence it uses principles from natural language. The design of Perl is therefore a combination of two things: convention set out by its muses and Larry Wall's desires.

This meant that Perl was free to pick and choose from different conventions or invent new techniques that solved problems inherent in others'. Application of Einstein's Constraint is clear in Perl 5 (as well as a few instances of failure to apply it!), and today we will look at booleans.

True and False

The most obvious starting point when deciding how to implement booleans is to decide what will be true and what will be false. Probably the most common falseness and trueness are zero and not-zero, respectively, a convention popularised—if not invented—by C. In some dialects of Lisp, an empty list is false, and in many languages, True and False are their own types.

JavaScript, PHP, Python: many languages have explicit types for true and false—global, singleton values that always represent trueness and falseness. This allows truth to be explicitly defined. But many languages these days, including PHP, JavaScript, Python and Perl, all employ a concept called coercion to switch between different types implicitly, i.e. swapping without you having to ask for it. When this is available, we also have to consider what other values have truth or falsehood.

Is it simpler to a) have True and False as separate values and coerce into them, or b) use existing values, and a rule?

Let's ask Einstein.

To answer, we have to consider usage. Perl draws a lot on the do-what-I-mean, or DWIM, philosophy of programming, a philosophy which naturally leads to Perl automatically and transparently switching between data types where possible (and in fact it is always possible, thanks to various operators). Strings and numbers are interchangeable; objects can be converted to strings; arrays to scalars or lists. Anything can be used anywhere and a rule is applied, consistently, so that the programmer knows what to expect perl¹ to do.

Observing this principle it seems like we're already tending towards b. Perl is, after all, already designed with rules in mind: a consistent (and concise) set of ground rules is the easiest way to understand what to expect in a given situation.

But if we follow the thought further we realise that once there is type coercion, the difference is moot. If we introduce a separate value for true and a separate one for false, this means we have to create a whole new set of rules for how to coerce other values into these two whenever there is a boolean test. If we already have to make that decision, it then follows that it doesn't matter what we coerce into: what matters is what values are false and what values are true.

Einstein's Constraint, then, says that since we have to implement b) anyway, and know the rules, it is tautologous to implement a) as well. Indeed, there may or may not be boolean data types internally to the Perl interpreter, but this makes no difference to Perl as a language. So, we can decide that we will simply choose values, rather than types, to be true or false. This is sensible, because Perl doesn't have types in that respect. True, false, 0 and 1 are all scalar values.

Deciding which values should be true and which should be false is the next logical step. Convention from C tells us that zero should be false; but Perl has list types, and Common Lisp suggests an empty list should be false. This nicely follows existing rules: since boolean values are scalar, and an array in scalar context is its length, then a zero-length array is naturally false because it is treated as zero. A list—however it is constructed—in scalar context returns its last item; an empty list will return no item. Nothingness, then, is falsehood.

Type coercion says that the string '0' is equal to 0, and hence false. We might consider that the strings '00' and '0E0'(0×10⁰) also numify as 0. But note that 0 will not stringify as either of these. Only the string '0' is fully equivalent to 0, because the conversion between these two will never produce a different value.

Furthermore, we are coercing to booleans, not integers: that is to say, we want to know whether it is true or not, and we are not using any specific type to represent truthiness. You may spot inconsistency. In this respect, you might say that Perl does have a boolean type, and you'd be right in a sense. However, there is no true and false; there is merely a state of trueness that a value can have. The value only has a value of truth when it is used as a truth value; in other languages, true and false always represent truth values. Truth is contextual. How recondite.

Perl therefore defines '0' to be false for consistency with 0, but any other string to be true, including '00' and the common zero-but-true value '0E0'. The exception is the empty string '', which falls into the "nothingness" category, since there's nothing in it, and is also false.

Finally, the undefined value is conceptually equivalent to an empty list: it is a scalar with no value. That, too, is a false value.

Removing the tautology has made this as simple as possible. There is no remaining tautology here.

Comparison Operators

Einstein's Constraint removed the need for extra values from Perl to represent truth and falsehood. The same mantra in fact extends Perl's collection of operators for comparison.

To explain why, we should take a look at languages that don't. JavaScript and PHP both use == to compare all things: strings, integers and objects. They both also have === to compare without coercion.

As mentioned, coercion is the practice of treating a variable as another type by silently converting from its current type to another. Using == to compare two types in JavaScript or PHP will coerce both, one or neither operand to a different type, based on rules documented somewhere.

Using == to compare two types in Perl will coerce both operands to numbers, and compare the results. Using eq will coerce both operands to strings, and compare the results.

Why?

This becomes easy to explain when we consider the types of comparison available to us. The two obvious ones are numerical and string comparison. Two numbers are equal if they represent the same platonic value—020, 16 and 0x10 are all equal. Two strings are equal when they contain the same characters in the same order.

Then you might suggest that two arrays are equal if they are the same length, which works for Perl. Or that they are equivalent: the same keys and the same values, which works for PHP. Or that they are the same actual array, which works for PHP and JavaScript.

What about two objects? Perl's objects are necessarily references, so referential equality seems reasonable—but Perl also has operator overload, so the decision could be given to the objects themselves. PHP has true objects, so you might suggest that an operator overload would be good, but PHP doesn't think you can be trusted with that so it compares them by comparing their attributes instead. JavaScript also uses referential equality, but doesn't allow for operator overloading either.

For PHP and JavaScript, == is actually an equivalence operator, and hence numerous rules are needed to determine what is equivalent to what. === is also an equivalence operator: it is still not an equality operator. It just happens that the equivalence has fewer rules, and in many cases equality is the only satisfactory state.

Also, we've been focusing on equality. What about the other operators, < and >? Strings can be compared lexically: there are rules for what is "less than" and what is "greater than". Objects, well, who knows? PHP's manual commits the fatal flaw of calling == the "comparison operator", whereas it is in fact a comparison operator called the equality (or equivalence!) operator—a mistake which allows the manual to conveniently omit the rules for the other ones. JavaScript takes a better approach and simply decides that objects are not comparable and returns false when you try a magnitude test.

But what do you do when the strings could be integers? Do you compare lexically, or numerically? Neither is incorrect. You would be upset if you were trying to sort a list by lexical analysis only to find that your language was assuming they were numbers and treating them numerically, and likewise the reverse. Is the string "10" less than or greater than the string "011"? If we weren't type-coercing we would know instantly: it's a string, so it's greater. But we are, so we don't.

Here is a generalised table over all of ==, <, <=, > and >= operators, showing you what coercion you can expect from the languages we've mentioned, on various operands. The result column is the result of the operator <, for reference. I chose that one because it performs the most erratically. In the example, [] are used to refer to real arrays in Perl, not arrayrefs.

Operands		Treated as			Result
L	R	PHP	JS	Perl	PHP	JS	Perl
0	1	Numbers	Numbers	Numbers	1	1	1
"0"	"1"	Numbers	Numbers	Numbers	1	1	1
"a"	"b"	Strings	Strings	Numbers	1	1	0*
"10"	11	Numbers	Numbers	Numbers	1	1	1
"10"	"011"	???	Strings	Numbers	0	0	1
"10b"	"11a"	Strings	Numbers	Numbers	0	1	1
[1, 2, 3]	[1, 2, 3, 4]	???	Objects**	Numbers	1	0	1
[1, 2, 3]	4	Array is always greater	Objects**	Numbers	0	0	1
false	true	Booleans	Numbers	Numbers2	1	1	-

* Non-numeric strings numify as zero, and a warning is cast that you numified a non-numeric string. ** < and > are defined always to return false; otherwise, true is returned if they refer to the same thing.

This is Einstein's Constraint again. Perl has made it as simple as possible, but no simpler. In PHP's case it is not defined generally over the five operators how the arguments will be treated. In JavaScript's case, each pair of operands is consistent across the five operators, but the language is inconsistent as the operands change. There are rules, but why should you have to remember them? In Perl's case they are always treated as numbers. It cannot be simpler without being more complex elsewhere.

Perl sidesteps the whole issue simply by stating that if you use any of <, <=, ==, >=, > or the special Perl-only <=>³ then they are treated as numbers; and if you use any of lt, lte, eq, gte, gt or cmp, then they are treated as strings. The mnemonic is simple: the mathematical operators are used on numbers, and the letters operators are used on letters.

Triple Equals

A hue and a cry! What audacity to not mention that PHP and JavaScript have the triple-equals operator, === that enforces type checking as well. With this magical operator, we solve the problem a different way. We can, in all cases, avoid the problem of type coercion by simply demanding that it not take place.

All cases? No. Since both languages have false as well as an undefined (null) value and zero, how do you test a string, read from standard input, for falsity? Or how do you compare a variable that exists, but is not defined or is false, and differentiate it from zero? And how many more rules and exceptions are there to this new operator, that can compare types as well? Are we forgetting the principle that we should be able to implicitly treat any type than any other type? Didn't we learn a lesson from the true/false thought experiment?

Perl's use of two types of operators for two types of comparison remains simpler, and the main reason is that all things are supposed to be coerced into all other things. That is a sound principle in Perl, but without these extra operators, other languages find a barrier preventing them from seamlessly implementing the philosophy.

That aside, there is not a triple-equals version of <= or >= is there? Those are the troublemakers, after all. Those are the ones that force us to sort our number-like strings the way they want to, not the way we want to. How do we prevent this behaviour on these other operators? Oh sod it all, let's just have separate comparisons for strings and numbers.

¹ By convention, Perl is the language and perl is the interpreter.
² The Constraint explained earlier shows why we got rid of the boolean type for Perl. While this row is correct for PHP's and JavaScript's two boolean values, all three languages will come a cropper if you try to compare a trueish value with a falsish one. Perl, again, simplifies it by not doing this, and therefore we can't say what Perl will treat true and false as because they don't exist. But it would be numbers.
³ The spaceship operator returns -1 if A is less than B; zero if they are equal; and 1 if A is greater than B. The same test is three lines of code, or two chained ternary conditional operators, in other languages: sort { $a <=> $b } ... is better than sort { $a == $b ? 0 : $a < $b ? -1 : 1 } ... because a) it is legible. cmp does the same, but for strings.