{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Julia 0.5 Highlights" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Link to the original blog post" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Julia 0.5 is a pivotal release. It introduces more transformative features than any release since the first official version. Moreover, several of these features set the stage for even more to come in the [lead up to Julia 1.0](https://www.youtube.com/watch?v=5gXMpbY1kJY). In this post, we’ll go through some of the major changes in 0.5, including improvements to functional programming, comprehensions, generators, arrays, strings, and more.\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO: Nothing to be done\n" ] } ], "source": [ "# We add the BenchmarkTools package and load it to demonstrate the new features in 0.5\n", "\n", "Pkg.add(\"BenchmarkTools\")\n", "using BenchmarkTools" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Julia has always had first-class functions that can be passed to and from other functions (i.e. higher-order functions) and full support for lambdas. Before this release, however, anonymous and higher-order functions came with a significant performance cost, and in a language that targets high-performance technical computing, that’s a serious limitation. So the Julia standard library and ecosystem have been rife with work-arounds to get the expressiveness of functional programming without the performance problems. But the right solution, of course, is to make functional programming fast – ideally just as fast as the optimal hand-written version of your code would be. In Julia 0.5, it is. And that changes everything.\n", "\n", "First, some definitions – they’re the same in both 0.4 and 0.5:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "double_it_map (generic function with 1 method)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v = rand(10^7); # 10 million random numbers\n", "double_it_vec(v) = 2v # vectorized doubling of input\n", "double_it_map(v) = map(x->2x, v) # map a lambda over input" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The version defined in terms of map is as fast as the vectorized method in 0.5. In this case, writing `2v` happens to be more convenient than writing `map(x->2x, v)` – and, of course, you can keep using vectorized forms when they’re convenient – but there are many cases where functional constructs are more general, clearer, and more convenient. Now they’re also fast." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "mean([@elapsed(double_it_vec(v)) for _=1:100]) # Vectorized version" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "mean([@elapsed(double_it_map(v)) for _=1:100]) # Higher order function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Ambiguous methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Julia 0.4 and earlier, the second method definition causes an ambiguity warning. n Julia 0.5 the existence of potential ambiguities is fine, but actually calling an ambiguous method is an immediate error. The above method definitions for f, which previously triggered a warning, are now silent, but calling f with two Int arguments is a method dispatch error: " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(a::Int, b::Real) = 1\n", "f(a::Real, b::Int) = 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(3,4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Return type annotations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A long-requested feature has been the ability to annotate method definitions with an explicit return type. This aids the clarity of code, serves as self-documentation, helps the compiler reason about code, and ensures that return types are what programmers intend them to be. In 0.5, you can annotate method definitions with a return type like so:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "function clip{T<:Real}(x::T, lo::Real, hi::Real)::T\n", " if x < lo\n", " return lo\n", " elseif x > hi\n", " return hi\n", " else\n", " return x\n", " end\n", "end" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip(0.5, 1, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip(1.5, 1, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip(2.5, 1, 2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You’ll note that the annotated return type here is `T`, which is a type parameter of the clip method. Not only is that allowed, but the return type can be an arbitrary expression of argument values, type parameters, and values from outer scopes. For example, here is a variation that promotes its arguments:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "function clip2(x::Real, lo::Real, hi::Real)::promote_type(typeof(x), typeof(lo), typeof(hi))\n", " if x < lo\n", " return lo\n", " elseif x > hi\n", " return hi\n", " else\n", " return x\n", " end\n", "end" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip2(2, 1, 3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip2(2, 1, 13//5)\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip2(2.5, 1, 13//5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Vectorized function calls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Julia 0.5 introduces the syntax `f.(A1, A2, ...)` for vectorized function calls. This syntax translates to `broadcast(f, A1, A2, ...)`, where broadcast is a higher-order function (introduced in 0.2), which generically implements the kind of broadcasting behavior found in Julia’s “dotted operators” such as `+, .-, .*, and ./.` Since higher-order functions are now efficient, writing `broadcast(f,v,w)` and `f.(v,w)` are both about as fast as loops specialized for the operation `f` and the shapes of `v` and `w`. This syntax lets you vectorize your scalar functions the way built-in vectorized functions like log, exp, and atan2 work. In fact, in the future, this syntax may replace the pre-vectorized methods of functions like exp and log, so that users will write `exp.(v)` to exponentiate a vector of values. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "v = randn(10)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clip.(v, -1, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second and third argument don’t need to be scalars – as with dotted operators, they can be vectors as well, and the clip operation will be applied to each corresponding triple of values:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ " clip.(v, repmat([-1,0.5],5), repmat([-0.5,1],5))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From this example, it may be unclear why this operation is called “broadcast”. The function gets its name from the following behavior: wherever one of its arguments has a singleton dimension, it “broadcasts” that value along the corresponding dimension of the other arguments when applying the operator. This behavior allows dotted operations to easily do handy tricks like quickly mean-centering the columns of a matrix:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "A = rand(3,4);\n", "B = A .- mean(A,1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "mean(B,1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The matrix `A` is `3×4` and `mean(A,1)` is `1×4` so the `.-` operator broadcasts each column’s mean along the values of the column in A, subtracting that mean from each element. Combining this broadcasting behavior with vectorized call syntax let’s us write some fairly fancy custom array operations very concisely:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ " clip.(B, [-0.3, -0.2, -0.1], [0.4, 0.3, 0.2, 0.1]')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This clips each element of `B` with its own specific `(hi,lo)` pair, as in this matrix:\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ " [(lo,hi) for lo=[-0.3, -0.2, -0.1], hi=[0.4, 0.3, 0.2, 0.1]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This computation avoids allocating any intermediate arrays and performs the entire vectorized computation all at once into the result array. We can see this difference in allocation when we benchmark these expressions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "X, Y = rand(1000,1000), rand(1000,1000);" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@benchmark atan2(cos(X), sin(Y))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@benchmark atan2.(cos.(X), sin.(Y))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With in-place vectorized assignment, we can fill the pre-allocated array, `Z`, without doing any allocation (the 96 bytes is an artifact)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "Z = zeros(X);\n", "@benchmark Z .= atan2.(cos.(X), sin.(Y))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Comprehensions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Julia's array comprehensions have always supported some advanced features such as iterating with several variables to produce multidimensional arrays.\n", "This release rounds out the functionality of comprehensions with two additional features: nested generation with multiple `for` clauses, and filtering with a trailing `if` clause.\n", "To demonstrate these features, consider making a dollar (100¢) using quarters (25¢), dimes (10¢), nickels (5¢) and pennies (1¢).\n", "We can generate an array of tuples of total values in each kind of coin by using a comprehension with nested `for` clauses:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "change = [(q,d,n,p) for q=0:25:100 for d=0:10:100-q for n=0:5:100-q-d for p=100-q-d-n]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are a few notable differences from the multidimensional array syntax:\n", "\n", "- Each iteration is a new `for` clause, rather than a single compound iteration separated by commas.\n", "- Each successive `for` clause *can* refer to variables from the previous clauses.\n", "- The result is a single flat vector regardless of how many `for` clauses there are.\n", "\n", "The tuple `(q,d,n,p)` in the comprehension body is a breakdown of monetary value into quarters, dimes, nickels and pennies.\n", "Note that the iteration range for `p` isn't a range at all, it's a single value, `100-q-d-n`, the unique number guaranteeing that each tuple adds up to a dollar.\n", "(This relies on the fact that a number behaves like an immutable zero-dimensional container, holding only itself, a behavior which is sometimes convenient but which has been the subject of significant debate. As of 0.5 it still works.)\n", "\n", "We can verify that each tuple adds up to 100 using a comprehension to make an array of the sums of each tuple and passing that array to the `extrema` function, to find the minimum and maximum over all sums:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "extrema([sum(t) for t in change])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since the minimum and maximum are both 100, the sums must all be exactly 100.\n", "So, we know there are 242 ways to make a dollar with common coins.\n", "But suppose we want to ensure that the value in pennies is less than the value in nickels, and so forth.\n", "By adding a filter clause, we can do this easily too:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[(q,d,n,p) for q=0:25:100 for d=0:10:100-q for n=0:5:100-q-d for p=100-q-d-n if p < n < d < q]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Generators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the previous section we used an array comprehension to take the sum of each tuple, save the sums as an array, and then pass that array of sums to the `extrema` function to find the largest and smallest sum (they should all be 100):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@time extrema([sum(t) for t in change])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since `extrema` works with arbitrary iterable objects – including generators – expressing an interleaved calculation using constant memory is now as simple as deleting `[` and `]`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@time extrema(sum(t) for t in change)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This avoids allocating a temporary array of sums entirely, instead computing the next tuple sum only when the `extrema` function is ready to accept a new value.\n", "In this case, the allocation saved is trivial, but it transforms this code's memory overhead from *O(n)* in the number of samples to *O(1)*.\n", "It's not hard to imagine situations where such a reduction in asymptotic memory usage is crucial.\n", "The similar syntax between array comprehensions and generator expressions makes it trivial to move back and forth between the two styles of computation as needed.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initializing collections" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The new generator syntax dovetails particularly nicely with Julia's convention for constructing collections – to make a new collection, you call the constructor with a single iterable argument, which yields the values you want in the new collection.\n", "In its simplest form, this looks something like:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "IntSet([1, 4, 9, 16, 25, 36, 49, 64])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can create the same `IntSet` using a comprehension that creates an Array, or a generator expression. The generator version is much more concise, just as clear, and (as we're about to see) faster – let's benchmark these two constructions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@benchmark IntSet([k^2 for k = 1:8])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@benchmark IntSet(k^2 for k = 1:8) # no temporary array!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Constructing dictionaries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generators can be used to construct dictionaries too, and this use case deserves some special attention since it com\n", "pletes a multi-release process of putting user-defined dictionary types on an equal footing with the built-in `Dict`\n", " type.\n", "In Julia 0.3, the `=>` operator only existed as part of syntax for constructing `Dict` objects – e.g. `[k₁ => v₁, k₂\n", " => v₂]` or `[k(i) => v(i) for i = c]`.\n", "This design was based on other dynamic languages where dictionaries are among a small set of built-in types, that ha\n", "ve special syntax and are deeply integrated into the system.\n", "As Julia's ecosystem has matured, however, it has become apparent that Julia is actually more like Java or C++ in th\n", "is respect than it is like Python or Lua: the `Dict` type isn't that special – it happens to be defined in the stand\n", "ard library, but is otherwise quite ordinary.\n", "Many programs use other dictionary implementations: for example, the tree-based `SortedDict` type, which sorts value\n", "s by key; or `OrderedDict`, which maintains keys in the order they are inserted.\n", "Having special syntax only for `Dict` makes using other dictionary implementations problematic.\n", "In 0.3, there was no good syntax for constructing values of these dictionaries – the best one could do was to invoke\n", " a constructor with an array of two-tuples:\n", "\n", " SortedDict([(k₁, v₁), (k₂, v₂)]) # fixed-size dictionaries\n", " SortedDict([(k(i), v(i)) for i in c]) # dictionary comprehensions\n", "\n", "Not only are these constructions inconvenient and ugly, they're also inefficient since they create temporary heap-allocated arrays of heap-allocated tuples of key-value pairs.\n", "In Julia 0.5, with much relief, we can now instead write:\n", "\n", " SortedDict(k₁ => v₁, k₂ => v₂) # fixed-size dictionaries, since 0.4\n", " SortedDict(k(i) => v(i) for i = c) # dictionary comprehensions, since 0.5\n", " \n", " \n", "This syntax combines two orthogonal features introduced in 0.4 and 0.5, respectively: `k => v` as standalone syntax for a `Pair` object and generator expressions.\n", "The `Dict` type is now constructed in exactly the same way:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "Dict(\"foo\" => 1, \"bar\" => 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ " Dict(\"*\"^k => k for k = 1:10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Arrays\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All previous versions of Julia have dropped trailing scalar slices when performing multidimensional array slicing.\n", "That is, when an array was sliced with multiple indices, the resulting array had the number of dimensions of the original array minus the number of trailing scalar slices.\n", "Thus, when you sliced a column out of a matrix the result was 1-dimensional, but when you sliced a row the result was a 2-dimensional row matrix. This rule is handy for linear algebra since row and column slices have distinct types and different orientations, bu\n", "t its complexity, asymmetry, and lack of generality make it less than ideal for arrays as general purpose containers.\n", "\n", "By comparison, the new slicing behavior in 0.5 is simple, systematic, and symmetrical.\n", "(And not original by any means – APL pioneered this array slicing scheme in the 1960s.)\n", "In Julia 0.5, when an array is sliced, the dimension of the result is the sum of the dimensions of the slices, and the dimension sizes of the result are the concatenation of the sizes of the slices.\n", "Thus, row slices and column slices are both vectors.\n", "\n", "Consider an N-d example:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "T = [i+j^2+k^3 for i=1:3, j=1:4, k=1:2]\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Slicing a 3-dimensional array with scalars in all but one dimension also produces a vector:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "T[2,:,2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "T[:,4,:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are a number of other changes for array views and many other improvements, which are noted in the 0.5 highlights blog post." ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.5.0", "language": "julia", "name": "julia-0.5" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.5.0" } }, "nbformat": 4, "nbformat_minor": 0 }