{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Contributed packages worth knowing about" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Packages from [JuliaStats](https://github.com/JuliaStats/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The `Distributions` package" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `Distributions` package provides a comprehensive list of univariate, multivariate and matrix distributions and a set of generic functions that can be applied to them. See the [documentation](http://distributionsjl.readthedocs.org/en/latest/) for details on the coverage.\n", "\n", "Note that some methods refer to the distribution __type__ (e.g. `Normal`) instead of a distribution __instance__ (e.g. `Normal(0.,1.)`)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "Distributions.Normal(μ=0.0, σ=1.0)" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using Distributions\n", "d = Normal() # standard normal (Gaussian) distribution" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "20-element Array{Float64,1}:\n", " -1.37163 \n", " 0.108703\n", " -0.614373\n", " 0.908453\n", " -0.616237\n", " 1.30055 \n", " 1.82147 \n", " 1.56462 \n", " 1.27457 \n", " -0.407733\n", " -0.691858\n", " -1.49719 \n", " 0.850856\n", " -0.250366\n", " 1.22701 \n", " -1.0103 \n", " -0.47893 \n", " 0.512367\n", " -0.433555\n", " 0.818587" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = rand(d,20) # sample of size 20 from a standard normal" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "Distributions.Normal(μ=0.15075083842683862, σ=0.9912678731753393)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "de = fit_mle(Normal,x) # maximum likelihood estimates of normal dist. pars." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.15075083842683862" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean(de)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.9912678731753393" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "std(de)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.9826119963895605" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "var(de)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "kurtosis(de)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.0" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "skewness(de)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "-28.203361159103117" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loglikelihood(de,x)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "Distributions.NormalStats(3.0150167685367726,0.15075083842683862,19.65223992779121,20.0)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ss = suffstats(Normal,x)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1x4 Array{Symbol,2}:\n", " :s :m :s2 :tw" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fieldnames(ss)'" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "Distributions.Normal(μ=0.15075083842683862, σ=0.9912678731753393)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fit_mle(Normal,ss) # can fit from sufficient statistics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is also possible to do `map` (maximum a posteriori) estimation using a conjugate prior. Again, see the documentation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The `DataArrays` package" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a lightweight package to define types that can contain `NA`'s. It was split off from the `DataFrames` package because loading the whole of `DataFrames` takes a while. (This will change when pre-compiled packages are more easily constructed.)\n", "\n", "The basic types defined in `DataArrays` are `NA`, used for literal NA input, the `DataArray`, like a numeric or integer vector in `R`, and the `PooledDataVector`, like a `factor` in `R`.\n", "\n", "The concept of an `NA` is built into many of the `R` types. In `Julia` a `DataArray` or `PooledDataArray` is composed of the data and a separate `bitarray` of indicators of missingness." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "4-element DataArrays.DataArray{Int64,1}:\n", " 2 \n", " 1 \n", " NA\n", " 4 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using DataArrays\n", "v = @data([2,1,NA,4])" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "2-element Array{Symbol,1}:\n", " :data\n", " :na " ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fieldnames(v)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "svec(Array{Int64,1},BitArray{1})" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "typeof(v).types" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "4-element Array{Int64,1}:\n", " 2\n", " 1\n", " 2\n", " 4" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v.data" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "4-element BitArray{1}:\n", " false\n", " false\n", " true\n", " false" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v.na" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "4-element BitArray{1}:\n", " false\n", " false\n", " true\n", " false" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "isna(v)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "3-element Array{Int64,1}:\n", " 2\n", " 1\n", " 4" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dropna(v)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `PooledDataArray`, generated with the `@pdata` macro call, consists of a `pool` array (similar to the `levels` in an `R` `factor` object) and a unsigned integer vector `refs`." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1000-element DataArrays.PooledDataArray{Char,UInt32,1}:\n", " 'M'\n", " 'F'\n", " 'M'\n", " 'M'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " ⋮ \n", " 'M'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "d = Bernoulli()\n", "sex = @pdata [rand(d) ≠ 0 ? 'F' : 'M' for i in 1:1000]" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "2-element Array{Char,1}:\n", " 'F'\n", " 'M'" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sex.pool" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1000-element Array{UInt32,1}:\n", " 0x00000002\n", " 0x00000001\n", " 0x00000002\n", " 0x00000002\n", " 0x00000002\n", " 0x00000001\n", " 0x00000001\n", " 0x00000001\n", " 0x00000002\n", " 0x00000001\n", " 0x00000002\n", " 0x00000001\n", " 0x00000001\n", " ⋮\n", " 0x00000002\n", " 0x00000001\n", " 0x00000002\n", " 0x00000001\n", " 0x00000001\n", " 0x00000001\n", " 0x00000002\n", " 0x00000001\n", " 0x00000001\n", " 0x00000001\n", " 0x00000001\n", " 0x00000002" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sex.refs" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1000-element DataArrays.PooledDataArray{Char,UInt8,1}:\n", " 'M'\n", " 'F'\n", " 'M'\n", " 'M'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " ⋮ \n", " 'M'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'F'\n", " 'M'" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sex = compact(sex) # provides a more compact representation, if possible" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1000-element Array{UInt8,1}:\n", " 0x02\n", " 0x01\n", " 0x02\n", " 0x02\n", " 0x02\n", " 0x01\n", " 0x01\n", " 0x01\n", " 0x02\n", " 0x01\n", " 0x02\n", " 0x01\n", " 0x01\n", " ⋮\n", " 0x02\n", " 0x01\n", " 0x02\n", " 0x01\n", " 0x01\n", " 0x01\n", " 0x02\n", " 0x01\n", " 0x01\n", " 0x01\n", " 0x01\n", " 0x02" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sex.refs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A few other functions common to an `R` programmer, like `rep`, are in the `DataArrays` package." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " / 38 KB Function\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: both StatsBase and Base export \"histrange\"; uses of it in module DataArrays must be qualified\n", "WARNING: both StatsBase and Base export \"midpoints\"; uses of it in module DataArrays must be qualified\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " @data 1533 bytes Function\n", " @pdata 1249 bytes Function\n", " AbstractDataArray 188 bytes DataType\n", " AbstractDataMatrix 80 bytes TypeConstructor\n", " AbstractDataVector 80 bytes TypeConstructor\n", " AbstractHistogram 228 bytes DataType\n", " CoefTable 284 bytes DataType\n", " DataArray 220 bytes DataType\n", " DataArrays 674 KB Module\n", " DataMatrix 80 bytes TypeConstructor\n", " DataVector 80 bytes TypeConstructor\n", " EachDropNA 168 bytes DataType\n", " EachFailNA 168 bytes DataType\n", " EachReplaceNA 220 bytes DataType\n", " FastPerm 284 bytes DataType\n", " Histogram 272 bytes DataType\n", " L1dist 1279 bytes Function\n", " L2dist 577 bytes Function\n", " Linfdist 1450 bytes Function\n", " NA 0 bytes DataArrays.NAtype\n", " NAException 112 bytes DataType\n", " NAtype 92 bytes DataType\n", " PooledDataArray 260 bytes DataType\n", " PooledDataMatrix 120 bytes TypeConstructor\n", " PooledDataVecs 8507 bytes Function\n", " PooledDataVector 120 bytes TypeConstructor\n", " RegressionModel 92 bytes DataType\n", " StatisticalModel 92 bytes DataType\n", " StatsBase 476 KB Module\n", " WeightVec 284 bytes DataType\n", " addcounts! 14 KB Function\n", " allna 1744 bytes Function\n", " anyna 1708 bytes Function\n", " array 5595 bytes Function\n", " autocor 1089 bytes Function\n", " autocor! 3572 bytes Function\n", " autocov 4814 bytes Function\n", " autocov! 3572 bytes Function\n", " coef 516 bytes Function\n", " coeftable 516 bytes Function\n", " compact 1438 bytes Function\n", " competerank 649 bytes Function\n", " confint 516 bytes Function\n", " corkendall 5773 bytes Function\n", " corspearman 3260 bytes Function\n", " counteq 1287 bytes Function\n", " countmap 2176 bytes Function\n", " countne 1287 bytes Function\n", " counts 8052 bytes Function\n", " crosscor 8202 bytes Function\n", " crosscor! 6952 bytes Function\n", " crosscov 8202 bytes Function\n", " crosscov! 6952 bytes Function\n", " crossentropy 2134 bytes Function\n", " cut 3992 bytes Function\n", " data 504 bytes Function\n", " denserank 647 bytes Function\n", " describe 560 bytes Function\n", " deviance 516 bytes Function\n", " df_residual 516 bytes Function\n", " dropna 2555 bytes Function\n", " each_dropna 557 bytes Function\n", " each_failNA 617 bytes Function\n", " each_failna 557 bytes Function\n", " each_replaceNA 629 bytes Function\n", " each_replacena 1015 bytes Function\n", " ecdf 948 bytes Function\n", " entropy 1799 bytes Function\n", " findat 538 bytes Function\n", " fit 13 KB Function\n", " fit! 548 bytes Function\n", " fitted 516 bytes Function\n", " geomean 1163 bytes Function\n", " getpoolidx 2027 bytes Function\n", " gkldiv 1559 bytes Function\n", " gl 2271 bytes Function\n", " harmmean 1153 bytes Function\n", " head 592 bytes Function\n", " hist 2683 bytes Function\n", " indexmap 1184 bytes Function\n", " indicatormat 4556 bytes Function\n", " inverse_rle 1942 bytes Function\n", " invsoftplus 5037 bytes Function\n", " iqr 608 bytes Function\n", " isna 4945 bytes Function\n", " kldivergence 2138 bytes Function\n", " kurtosis 5279 bytes Function\n", " levels 1717 bytes Function\n", " levelsmap 1320 bytes Function\n", " logistic 4468 bytes Function\n", " logit 4441 bytes Function\n", " loglikelihood 516 bytes Function\n", " logsumexp 2785 bytes Function\n", " mad 3645 bytes Function\n", " maxad 570 bytes Function\n", " mean_and_cov 2938 bytes Function\n", " mean_and_std 2120 bytes Function\n", " mean_and_var 2120 bytes Function\n", " meanad 585 bytes Function\n", " middle 3053 bytes Function\n", " mode 4046 bytes Function\n", " model_response 516 bytes Function\n", " modes 4963 bytes Function\n", " moment 2360 bytes Function\n", " msd 587 bytes Function\n", " nobs 516 bytes Function\n", " nquantile 590 bytes Function\n", " ordinalrank 645 bytes Function\n", " pacf 3226 bytes Function\n", " pacf! 1893 bytes Function\n", " padNA 1240 bytes Function\n", " pdata 516 bytes Function\n", " percent_change 1169 bytes Function\n", " percentile 583 bytes Function\n", " predict 516 bytes Function\n", " predict! 516 bytes Function\n", " proportionmap 1003 bytes Function\n", " proportions 7036 bytes Function\n", " psnr 655 bytes Function\n", " reldiff 1140 bytes Function\n", " removeNA 601 bytes Function\n", " reorder 1040 bytes Function\n", " rep 6024 bytes Function\n", " replace! 4684 bytes Function\n", " residuals 516 bytes Function\n", " rle 6285 bytes Function\n", " rmsd 1691 bytes Function\n", " sample 9170 bytes Function\n", " sample! 3510 bytes Function\n", " samplepair 1447 bytes Function\n", " scattermat 3208 bytes Function\n", " sem 568 bytes Function\n", " set_levels 611 bytes Function\n", " set_levels! 615 bytes Function\n", " setlevels 3164 bytes Function\n", " setlevels! 4791 bytes Function\n", " skewness 5199 bytes Function\n", " softmax 560 bytes Function\n", " softmax! 2515 bytes Function\n", " softplus 5021 bytes Function\n", " span 742 bytes Function\n", " sqL2dist 1276 bytes Function\n", " stderr 517 bytes Function\n", " summarystats 1004 bytes Function\n", " tail 607 bytes Function\n", " tiedrank 646 bytes Function\n", " trimmean 1904 bytes Function\n", " variation 1035 bytes Function\n", " vcov 516 bytes Function\n", " view 4098 bytes Function\n", " weights 1086 bytes Function\n", " wmean 723 bytes Function\n", " wmedian 1071 bytes Function\n", " wquantile 2130 bytes Function\n", " wsample 4621 bytes Function\n", " wsample! 1945 bytes Function\n", " wsum 1775 bytes Function\n", " wsum! 1904 bytes Function\n", " xlogx 4467 bytes Function\n", " xlogy 5141 bytes Function\n", " xtab 180 bytes DataType\n", " xtabs 1041 bytes Function\n", " zscore 3008 bytes Function\n", " zscore! 2904 bytes Function\n" ] } ], "source": [ "whos(DataArrays)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The `DataFrames` package" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `DataFrame` type and methods for working with it are defined in the `DataFrames` package. There is online documentation but, at least in my browser, the formatting is horrible. I would recommend reading the PDF file instead.\n", "\n", "The `DataFrames` package is where the formula language and types like `ModelFrame` and `ModelMatrix` are defined. Many familiar examples of data frames are available in the `RDatasets` package." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
BatchYield
1A1545
2A1440
3A1440
4A1520
5A1580
6B1540
7B1555
8B1490
9B1560
10B1495
11C1595
12C1550
13C1605
14C1510
15C1560
16D1445
17D1440
18D1595
19D1465
20D1545
21E1595
22E1630
23E1515
24E1635
25E1625
26F1520
27F1455
28F1450
29F1480
30F1445
" ], "text/plain": [ "30x2 DataFrames.DataFrame\n", "| Row | Batch | Yield |\n", "|-----|-------|-------|\n", "| 1 | \"A\" | 1545 |\n", "| 2 | \"A\" | 1440 |\n", "| 3 | \"A\" | 1440 |\n", "| 4 | \"A\" | 1520 |\n", "| 5 | \"A\" | 1580 |\n", "| 6 | \"B\" | 1540 |\n", "| 7 | \"B\" | 1555 |\n", "| 8 | \"B\" | 1490 |\n", "| 9 | \"B\" | 1560 |\n", "| 10 | \"B\" | 1495 |\n", "| 11 | \"C\" | 1595 |\n", "⋮\n", "| 19 | \"D\" | 1465 |\n", "| 20 | \"D\" | 1545 |\n", "| 21 | \"E\" | 1595 |\n", "| 22 | \"E\" | 1630 |\n", "| 23 | \"E\" | 1515 |\n", "| 24 | \"E\" | 1635 |\n", "| 25 | \"E\" | 1625 |\n", "| 26 | \"F\" | 1520 |\n", "| 27 | \"F\" | 1455 |\n", "| 28 | \"F\" | 1450 |\n", "| 29 | \"F\" | 1480 |\n", "| 30 | \"F\" | 1445 |" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ " @data 1533 bytes Function\n", " @pdata 1545 bytes Function\n", " AbstractDataArray 188 bytes DataType\n", " AbstractDataMatrix 80 bytes TypeConstructor\n", " AbstractDataVector 80 bytes TypeConstructor\n", " AbstractHistogram 228 bytes DataType\n", " CoefTable 284 bytes DataType\n", " DataArray 220 bytes DataType\n", " DataArrays 803 KB Module\n", " DataMatrix 80 bytes TypeConstructor\n", " DataVector 80 bytes TypeConstructor\n", " EachDropNA 168 bytes DataType\n", " EachFailNA 168 bytes DataType\n", " EachReplaceNA 220 bytes DataType\n", " FastPerm 284 bytes DataType\n", " Histogram 272 bytes DataType\n", " L1dist 1279 bytes Function\n", " L2dist 577 bytes Function\n", " Linfdist 1450 bytes Function\n", " NA 0 bytes DataArrays.NAtype\n", " NAException 112 bytes DataType\n", " NAtype 92 bytes DataType\n", " PooledDataArray 260 bytes DataType\n", " PooledDataMatrix 120 bytes TypeConstructor\n", " PooledDataVecs 8507 bytes Function\n", " PooledDataVector 120 bytes TypeConstructor\n", " RegressionModel 92 bytes DataType\n", " StatisticalModel 92 bytes DataType\n", " StatsBase 601 KB Module\n", " WeightVec 284 bytes DataType\n", " addcounts! 14 KB Function\n", " allna 1744 bytes Function\n", " anyna 1708 bytes Function\n", " array 5595 bytes Function\n", " autocor 1089 bytes Function\n", " autocor! 3572 bytes Function\n", " autocov 4814 bytes Function\n", " autocov! 3572 bytes Function\n", " coef 516 bytes Function\n", " coeftable 516 bytes Function\n", " compact 3324 bytes Function\n", " competerank 649 bytes Function\n", " confint 516 bytes Function\n", " corkendall 5773 bytes Function\n", " corspearman 3260 bytes Function\n", " counteq 1287 bytes Function\n", " countmap 2176 bytes Function\n", " countne 1287 bytes Function\n", " counts 8052 bytes Function\n", " crosscor 8202 bytes Function\n", " crosscor! 6952 bytes Function\n", " crosscov 8202 bytes Function\n", " crosscov! 6952 bytes Function\n", " crossentropy 2134 bytes Function\n", " cut 3992 bytes Function\n", " data 504 bytes Function\n", " denserank 647 bytes Function\n", " describe 560 bytes Function\n", " deviance 516 bytes Function\n", " df_residual 516 bytes Function\n", " dropna 3439 bytes Function\n", " each_dropna 557 bytes Function\n", " each_failNA 617 bytes Function\n", " each_failna 557 bytes Function\n", " each_replaceNA 629 bytes Function\n", " each_replacena 1015 bytes Function\n", " ecdf 948 bytes Function\n", " entropy 31 KB Function\n", " findat 538 bytes Function\n", " fit 18 KB Function\n", " fit! 548 bytes Function\n", " fitted 516 bytes Function\n", " geomean 1163 bytes Function\n", " getpoolidx 2027 bytes Function\n", " gkldiv 1559 bytes Function\n", " gl 2271 bytes Function\n", " harmmean 1153 bytes Function\n", " head 592 bytes Function\n", " hist 2683 bytes Function\n", " indexmap 1184 bytes Function\n", " indicatormat 4556 bytes Function\n", " inverse_rle 1942 bytes Function\n", " invsoftplus 5037 bytes Function\n", " iqr 608 bytes Function\n", " isna 5623 bytes Function\n", " kldivergence 2638 bytes Function\n", " kurtosis 38 KB Function\n", " levels 1717 bytes Function\n", " levelsmap 1320 bytes Function\n", " logistic 4468 bytes Function\n", " logit 4441 bytes Function\n", " loglikelihood 2394 bytes Function\n", " logsumexp 2785 bytes Function\n", " mad 3645 bytes Function\n", " maxad 570 bytes Function\n", " mean_and_cov 2938 bytes Function\n", " mean_and_std 2120 bytes Function\n", " mean_and_var 2120 bytes Function\n", " meanad 585 bytes Function\n", " middle 3053 bytes Function\n", " mode 31 KB Function\n", " model_response 516 bytes Function\n", " modes 14 KB Function\n", " moment 2360 bytes Function\n", " msd 587 bytes Function\n", " nobs 516 bytes Function\n", " nquantile 590 bytes Function\n", " ordinalrank 645 bytes Function\n", " pacf 3226 bytes Function\n", " pacf! 1893 bytes Function\n", " padNA 1240 bytes Function\n", " pdata 516 bytes Function\n", " percent_change 1169 bytes Function\n", " percentile 583 bytes Function\n", " predict 516 bytes Function\n", " predict! 516 bytes Function\n", " proportionmap 1003 bytes Function\n", " proportions 7036 bytes Function\n", " psnr 655 bytes Function\n", " reldiff 1140 bytes Function\n", " removeNA 601 bytes Function\n", " reorder 1040 bytes Function\n", " rep 6024 bytes Function\n", " replace! 4684 bytes Function\n", " residuals 516 bytes Function\n", " rle 6285 bytes Function\n", " rmsd 1691 bytes Function\n", " sample 9170 bytes Function\n", " sample! 3510 bytes Function\n", " samplepair 1447 bytes Function\n", " scattermat 3208 bytes Function\n", " sem 568 bytes Function\n", " set_levels 611 bytes Function\n", " set_levels! 615 bytes Function\n", " setlevels 3164 bytes Function\n", " setlevels! 4791 bytes Function\n", " skewness 36 KB Function\n", " softmax 560 bytes Function\n", " softmax! 2515 bytes Function\n", " softplus 5021 bytes Function\n", " span 742 bytes Function\n", " sqL2dist 1276 bytes Function\n", " stderr 517 bytes Function\n", " summarystats 1004 bytes Function\n", " tail 607 bytes Function\n", " tiedrank 646 bytes Function\n", " trimmean 1904 bytes Function\n", " variation 1035 bytes Function\n", " vcov 516 bytes Function\n", " view 4098 bytes Function\n", " weights 1086 bytes Function\n", " wmean 723 bytes Function\n", " wmedian 1071 bytes Function\n", " wquantile 2130 bytes Function\n", " wsample 4621 bytes Function\n", " wsample! 1945 bytes Function\n", " wsum 1775 bytes Function\n", " wsum! 1904 bytes Function\n", " xlogx 4467 bytes Function\n", " xlogy 5141 bytes Function\n", " xtab 180 bytes DataType\n", " xtabs 1041 bytes Function\n", " zscore 3008 bytes Function\n", " zscore! 2904 bytes Function\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/bates/.julia/v0.4/RDatasets/src/dataset.jl:1\n", "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/bates/.julia/v0.4/RDatasets/src/dataset.jl:1\n", "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/bates/.julia/v0.4/RDatasets/src/datasets.jl:1\n" ] } ], "source": [ "using DataFrames, RDatasets\n", "ds = dataset(\"lme4\",\"Dyestuff\")" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "2-element Array{Symbol,1}:\n", " :Batch\n", " :Yield" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "names(ds)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Indivual columns can be accessed by name using symbols (e.g. `:Yield`). This means that the column names should be valid Julia identifiers. Among other things, they cannot contain the dot or period character (`.`)." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "30-element DataArrays.DataArray{Int32,1}:\n", " 1545\n", " 1440\n", " 1440\n", " 1520\n", " 1580\n", " 1540\n", " 1555\n", " 1490\n", " 1560\n", " 1495\n", " 1595\n", " 1550\n", " 1605\n", " ⋮\n", " 1465\n", " 1545\n", " 1595\n", " 1630\n", " 1515\n", " 1635\n", " 1625\n", " 1520\n", " 1455\n", " 1450\n", " 1480\n", " 1445" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds[:Yield]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `DataFrame` constructor can be given `=` pairs." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
xytruey
11.05.3000000000000015.4266459954573465
22.06.46.547618585779938
33.07.57.4126925420952325
44.08.6000000000000018.502478506094308
55.09.79.459142933970893
66.010.810.798058744556116
77.011.90000000000000211.791803744236113
88.013.013.131877566162375
99.014.10000000000000114.145992654678988
1010.015.215.120811027815394
" ], "text/plain": [ "10x3 DataFrames.DataFrame\n", "| Row | x | ytrue | y |\n", "|-----|------|-------|---------|\n", "| 1 | 1.0 | 5.3 | 5.42665 |\n", "| 2 | 2.0 | 6.4 | 6.54762 |\n", "| 3 | 3.0 | 7.5 | 7.41269 |\n", "| 4 | 4.0 | 8.6 | 8.50248 |\n", "| 5 | 5.0 | 9.7 | 9.45914 |\n", "| 6 | 6.0 | 10.8 | 10.7981 |\n", "| 7 | 7.0 | 11.9 | 11.7918 |\n", "| 8 | 8.0 | 13.0 | 13.1319 |\n", "| 9 | 9.0 | 14.1 | 14.146 |\n", "| 10 | 10.0 | 15.2 | 15.1208 |" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = 1.:10.;\n", "ϵ = rand(Normal(0.,0.1),length(x));\n", "β = [4.2,1.1];\n", "ytrue = [ones(length(x)) x]*β;\n", "dd = DataFrame(x=x,ytrue = ytrue, y = ytrue + ϵ)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In `R` many modeling functions that use a formula/data representation first apply `model.frame` then `model.matrix`. In the `DataFrames` package these are `ModelFrame` and `ModelMatrix`. A `ModelFrame` is the reduction of the original `DataFrame` to only those columns that are used in the model and after application of the NA action. It includes a `Terms` object, which describes the terms in the formula, again after some reduction and expansion. Finally, a record is kept of which rows in the original data frame are represented in the derived frame." ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "DataFrames.ModelFrame(10x2 DataFrames.DataFrame\n", "| Row | y | x |\n", "|-----|---------|------|\n", "| 1 | 5.42665 | 1.0 |\n", "| 2 | 6.54762 | 2.0 |\n", "| 3 | 7.41269 | 3.0 |\n", "| 4 | 8.50248 | 4.0 |\n", "| 5 | 9.45914 | 5.0 |\n", "| 6 | 10.7981 | 6.0 |\n", "| 7 | 11.7918 | 7.0 |\n", "| 8 | 13.1319 | 8.0 |\n", "| 9 | 14.146 | 9.0 |\n", "| 10 | 15.1208 | 10.0 |,DataFrames.Terms(Any[:x],Any[:y,:x],2x2 Array{Int8,2}:\n", " 1 0\n", " 0 1,[1,1],true,true),Bool[true,true,true,true,true,true,true,true,true,true])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mf = ModelFrame(y ~ x, dd)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `ModelMatrix` is constructed from the `ModelFrame`." ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "DataFrames.ModelMatrix{Float64}(10x2 Array{Float64,2}:\n", " 1.0 1.0\n", " 1.0 2.0\n", " 1.0 3.0\n", " 1.0 4.0\n", " 1.0 5.0\n", " 1.0 6.0\n", " 1.0 7.0\n", " 1.0 8.0\n", " 1.0 9.0\n", " 1.0 10.0,[0,1])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mm = ModelMatrix(mf)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `assign` vector in this object maps columns to terms. It is used when performing hypothesis tests, like `anova`. At present the `model_response` function returns the value of the expression on the left-hand side of the formula." ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "10-element Array{Float64,1}:\n", " 5.42665\n", " 6.54762\n", " 7.41269\n", " 8.50248\n", " 9.45914\n", " 10.7981 \n", " 11.7918 \n", " 13.1319 \n", " 14.146 \n", " 15.1208 " ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model_response(mf)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These facilities are not developed as fully as those in `R`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The GLM Package" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `GLM` package provides functions to fit and analyse the linear models and generalized linear models." ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "DataFrames.DataFrameRegressionModel{GLM.LinearModel{GLM.LmResp{Array{Float64,1}},GLM.DensePredQR{Float64}},Float64}:\n", "\n", "Coefficients:\n", " Estimate Std.Error t value Pr(>|t|)\n", "(Intercept) 4.22575 0.0913301 46.269 <1e-10\n", "x 1.09236 0.0147192 74.2132 <1e-11\n" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using GLM\n", "fm = lm(y ~ x, dd)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "DataFrames.DataFrameRegressionModel{GLM.LinearModel{GLM.LmResp{Array{Float64,1}},GLM.DensePredQR{Float64}},Float64}:\n", "\n", "Coefficients:\n", " Estimate Std.Error t value Pr(>|t|)\n", "(Intercept) 4.22575 0.0913301 46.269 <1e-10\n", "x 1.09236 0.0147192 74.2132 <1e-11\n" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fm = fit(LinearModel,y ~ x,dd)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### StatsBase" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `StatsBase` package contains functions for sample statistics and many utilities. There is online documentation. Much of the design and implementation is by Dahua Lin who is a stickler for extracting every last ounce of performance." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " AbstractHistogram 228 bytes DataType\n", " CoefTable 284 bytes DataType\n", " Histogram 272 bytes DataType\n", " L1dist 1279 bytes Function\n", " L2dist 577 bytes Function\n", " Linfdist 1450 bytes Function\n", " RegressionModel 92 bytes DataType\n", " StatisticalModel 92 bytes DataType\n", " StatsBase 470 KB Module\n", " WeightVec 284 bytes DataType\n", " addcounts! 11 KB Function\n", " autocor 4814 bytes Function\n", " autocor! 3572 bytes Function\n", " autocov 4814 bytes Function\n", " autocov! 3572 bytes Function\n", " coef 516 bytes Function\n", " coeftable 516 bytes Function\n", " competerank 649 bytes Function\n", " confint 516 bytes Function\n", " corkendall 5773 bytes Function\n", " corspearman 2532 bytes Function\n", " counteq 1287 bytes Function\n", " countmap 1118 bytes Function\n", " countne 1287 bytes Function\n", " counts 8052 bytes Function\n", " crosscor 8202 bytes Function\n", " crosscor! 6952 bytes Function\n", " crosscov 8202 bytes Function\n", " crosscov! 6952 bytes Function\n", " crossentropy 2134 bytes Function\n", " denserank 647 bytes Function\n", " describe 560 bytes Function\n", " deviance 516 bytes Function\n", " df_residual 516 bytes Function\n", " ecdf 948 bytes Function\n", " entropy 1799 bytes Function\n", " findat 538 bytes Function\n", " fit 13 KB Function\n", " fit! 548 bytes Function\n", " fitted 516 bytes Function\n", " geomean 1163 bytes Function\n", " gkldiv 1559 bytes Function\n", " harmmean 1153 bytes Function\n", " hist 2683 bytes Function\n", " histrange 5943 bytes Function\n", " indexmap 1184 bytes Function\n", " indicatormat 4556 bytes Function\n", " inverse_rle 1798 bytes Function\n", " invsoftplus 5037 bytes Function\n", " iqr 608 bytes Function\n", " kldivergence 2138 bytes Function\n", " kurtosis 4745 bytes Function\n", " levelsmap 1320 bytes Function\n", " logistic 4468 bytes Function\n", " logit 4441 bytes Function\n", " loglikelihood 516 bytes Function\n", " logsumexp 2785 bytes Function\n", " mad 3116 bytes Function\n", " maxad 570 bytes Function\n", " mean_and_cov 2938 bytes Function\n", " mean_and_std 2120 bytes Function\n", " mean_and_var 2120 bytes Function\n", " meanad 585 bytes Function\n", " middle 3053 bytes Function\n", " midpoints 1657 bytes Function\n", " mode 4046 bytes Function\n", " model_response 516 bytes Function\n", " modes 4963 bytes Function\n", " moment 2360 bytes Function\n", " msd 587 bytes Function\n", " nobs 516 bytes Function\n", " nquantile 590 bytes Function\n", " ordinalrank 645 bytes Function\n", " pacf 3226 bytes Function\n", " pacf! 1893 bytes Function\n", " percentile 583 bytes Function\n", " predict 516 bytes Function\n", " predict! 516 bytes Function\n", " proportionmap 1003 bytes Function\n", " proportions 7036 bytes Function\n", " psnr 655 bytes Function\n", " residuals 516 bytes Function\n", " rle 1824 bytes Function\n", " rmsd 1691 bytes Function\n", " sample 9170 bytes Function\n", " sample! 3510 bytes Function\n", " samplepair 1447 bytes Function\n", " scattermat 3208 bytes Function\n", " sem 568 bytes Function\n", " skewness 4665 bytes Function\n", " softmax 560 bytes Function\n", " softmax! 2515 bytes Function\n", " softplus 5021 bytes Function\n", " span 742 bytes Function\n", " sqL2dist 1276 bytes Function\n", " stderr 517 bytes Function\n", " summarystats 1004 bytes Function\n", " tiedrank 646 bytes Function\n", " trimmean 1904 bytes Function\n", " variation 1035 bytes Function\n", " vcov 516 bytes Function\n", " view 4098 bytes Function\n", " weights 1086 bytes Function\n", " wmean 723 bytes Function\n", " wmedian 1071 bytes Function\n", " wquantile 2130 bytes Function\n", " wsample 4621 bytes Function\n", " wsample! 1945 bytes Function\n", " wsum 1775 bytes Function\n", " wsum! 1904 bytes Function\n", " xlogx 4467 bytes Function\n", " xlogy 5141 bytes Function\n", " zscore 3008 bytes Function\n", " zscore! 2904 bytes Function\n" ] } ], "source": [ "using StatsBase\n", "whos(StatsBase)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### MLBase" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `MLBase` package contains many functions for data manipulation and reduction. It uses the Machine Learning (ML) terminology." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/modeltune.jl:5\n", "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/modeltune.jl:5\n", "WARNING: Base.String is deprecated, use AbstractString instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/modeltune.jl:5\n", "WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/deprecated/datapre.jl:104\n", "WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/deprecated/datapre.jl:105\n", "WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/deprecated/datapre.jl:163\n", "WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/deprecated/datapre.jl:163\n", "WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.\n", " likely near /home/juser/.julia/v0.4/MLBase/src/deprecated/datapre.jl:163\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " AbstractHistogram 228 bytes DataType\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: both StatsBase and Base export \"histrange\"; uses of it in module MLBase must be qualified\n", "WARNING: both StatsBase and Base export \"midpoints\"; uses of it in module MLBase must be qualified\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " CoefTable 284 bytes DataType\n", " CrossValGenerator 92 bytes DataType\n", " Forward 0 bytes Base.Order.ForwardOrdering\n", " Histogram 272 bytes DataType\n", " Kfold 136 bytes DataType\n", " L1dist 1279 bytes Function\n", " L2dist 577 bytes Function\n", " LOOCV 112 bytes DataType\n", " LabelMap 180 bytes DataType\n", " Linfdist 1450 bytes Function\n", " MLBase 369 KB Module\n", " ROCNums 228 bytes DataType\n", " RandomSub 136 bytes DataType\n", " RegressionModel 92 bytes DataType\n", " Reverse 0 bytes Base.Order.ReverseOrdering{Base.Or…\n", " Standardize 136 bytes DataType\n", " StatisticalModel 92 bytes DataType\n", " StatsBase 470 KB Module\n", " StratifiedKfold 148 bytes DataType\n", " StratifiedRandomSub 148 bytes DataType\n", " WeightVec 284 bytes DataType\n", " addcounts! 11 KB Function\n", " autocor 4814 bytes Function\n", " autocor! 3572 bytes Function\n", " autocov 4814 bytes Function\n", " autocov! 3572 bytes Function\n", " classify 4926 bytes Function\n", " classify! 3564 bytes Function\n", " classify_withscore 1934 bytes Function\n", " classify_withscores 1325 bytes Function\n", " classify_withscores! 2395 bytes Function\n", " coef 516 bytes Function\n", " coeftable 516 bytes Function\n", " competerank 649 bytes Function\n", " confint 516 bytes Function\n", " confusmat 1518 bytes Function\n", " corkendall 5773 bytes Function\n", " correctrate 611 bytes Function\n", " corspearman 2532 bytes Function\n", " counteq 1287 bytes Function\n", " counthits 4733 bytes Function\n", " countmap 1118 bytes Function\n", " countne 1287 bytes Function\n", " counts 8052 bytes Function\n", " cross_validate 2492 bytes Function\n", " crosscor 8202 bytes Function\n", " crosscor! 6952 bytes Function\n", " crosscov 8202 bytes Function\n", " crosscov! 6952 bytes Function\n", " crossentropy 2134 bytes Function\n", " denserank 647 bytes Function\n", " describe 560 bytes Function\n", " deviance 516 bytes Function\n", " df_residual 516 bytes Function\n", " ecdf 948 bytes Function\n", " entropy 1799 bytes Function\n", " errorrate 611 bytes Function\n", " f1score 604 bytes Function\n", " false_negative 496 bytes Function\n", " false_negative_rate 521 bytes Function\n", " false_positive 496 bytes Function\n", " false_positive_rate 521 bytes Function\n", " findat 538 bytes Function\n", " fit 13 KB Function\n", " fit! 548 bytes Function\n", " fitted 516 bytes Function\n", " geomean 1163 bytes Function\n", " gkldiv 1559 bytes Function\n", " gridtune 2063 bytes Function\n", " groupindices 3382 bytes Function\n", " harmmean 1153 bytes Function\n", " hist 2683 bytes Function\n", " hitrate 731 bytes Function\n", " hitrates 1422 bytes Function\n", " indexmap 1184 bytes Function\n", " indicatormat 4556 bytes Function\n", " inverse_rle 1798 bytes Function\n", " invsoftplus 5037 bytes Function\n", " iqr 608 bytes Function\n", " kldivergence 2138 bytes Function\n", " kurtosis 4745 bytes Function\n", " labeldecode 1648 bytes Function\n", " labelencode 1657 bytes Function\n", " labelmap 1417 bytes Function\n", " levelsmap 1320 bytes Function\n", " logistic 4468 bytes Function\n", " logit 4441 bytes Function\n", " loglikelihood 516 bytes Function\n", " logsumexp 2785 bytes Function\n", " mad 3116 bytes Function\n", " maxad 570 bytes Function\n", " mean_and_cov 2938 bytes Function\n", " mean_and_std 2120 bytes Function\n", " mean_and_var 2120 bytes Function\n", " meanad 585 bytes Function\n", " middle 3053 bytes Function\n", " mode 4046 bytes Function\n", " model_response 516 bytes Function\n", " modes 4963 bytes Function\n", " moment 2360 bytes Function\n", " msd 587 bytes Function\n", " nobs 516 bytes Function\n", " nquantile 590 bytes Function\n", " ordinalrank 645 bytes Function\n", " pacf 3226 bytes Function\n", " pacf! 1893 bytes Function\n", " percentile 583 bytes Function\n", " precision 3287 bytes Function\n", " predict 516 bytes Function\n", " predict! 516 bytes Function\n", " proportionmap 1003 bytes Function\n", " proportions 7036 bytes Function\n", " psnr 655 bytes Function\n", " recall 506 bytes Function\n", " repeach 3277 bytes Function\n", " repeachcol 3354 bytes Function\n", " repeachrow 4030 bytes Function\n", " residuals 516 bytes Function\n", " rle 1824 bytes Function\n", " rmsd 1691 bytes Function\n", " roc 15 KB Function\n", " sample 9170 bytes Function\n", " sample! 3510 bytes Function\n", " samplepair 1447 bytes Function\n", " scattermat 3208 bytes Function\n", " sem 568 bytes Function\n", " skewness 4665 bytes Function\n", " softmax 560 bytes Function\n", " softmax! 2515 bytes Function\n", " softplus 5021 bytes Function\n", " span 742 bytes Function\n", " sqL2dist 1276 bytes Function\n", " standardize 1819 bytes Function\n", " standardize! 1821 bytes Function\n", " stderr 517 bytes Function\n", " summarystats 1004 bytes Function\n", " tiedrank 646 bytes Function\n", " transform 1086 bytes Function\n", " trimmean 1904 bytes Function\n", " true_negative 496 bytes Function\n", " true_negative_rate 521 bytes Function\n", " true_positive 496 bytes Function\n", " true_positive_rate 521 bytes Function\n", " variation 1035 bytes Function\n", " vcov 516 bytes Function\n", " view 4098 bytes Function\n", " weights 1086 bytes Function\n", " wmean 723 bytes Function\n", " wmedian 1071 bytes Function\n", " wquantile 2130 bytes Function\n", " wsample 4621 bytes Function\n", " wsample! 1945 bytes Function\n", " wsum 1775 bytes Function\n", " wsum! 1904 bytes Function\n", " xlogx 4467 bytes Function\n", " xlogy 5141 bytes Function\n", " zscore 3008 bytes Function\n", " zscore! 2904 bytes Function\n" ] } ], "source": [ "using MLBase\n", "whos(MLBase)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### RCall" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [], "source": [ "using RCall" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
carboptden
10.10.086
20.30.269
30.50.446
40.60.538
50.70.626
60.90.782
" ], "text/plain": [ "6x2 DataFrames.DataFrame\n", "| Row | carb | optden |\n", "|-----|------|--------|\n", "| 1 | 0.1 | 0.086 |\n", "| 2 | 0.3 | 0.269 |\n", "| 3 | 0.5 | 0.446 |\n", "| 4 | 0.6 | 0.538 |\n", "| 5 | 0.7 | 0.626 |\n", "| 6 | 0.9 | 0.782 |" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "form = rcopy(\"Formaldehyde\")" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: RCall.jl Loading required package: Matrix\n" ] } ], "source": [ "@rimport lme4" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Arabidopsis 8 bytes RCall.RObject{RCall.VecSxp}\n", " Cv_to_Sv 8 bytes RCall.RObject{RCall.ClosSxp}\n", " Cv_to_Vv 8 bytes RCall.RObject{RCall.ClosSxp}\n", " Dyestuff 8 bytes RCall.RObject{RCall.VecSxp}\n", " Dyestuff2 8 bytes RCall.RObject{RCall.VecSxp}\n", " GHrule 8 bytes RCall.RObject{RCall.ClosSxp}\n", " GQN 8 bytes RCall.RObject{RCall.VecSxp}\n", " GQdk 8 bytes RCall.RObject{RCall.ClosSxp}\n", " InstEval 8 bytes RCall.RObject{RCall.VecSxp}\n", " NelderMead 8 bytes RCall.RObject{RCall.ClosSxp}\n", " Nelder_Mead 8 bytes RCall.RObject{RCall.ClosSxp}\n", " Pastes 8 bytes RCall.RObject{RCall.VecSxp}\n", " Penicillin 8 bytes RCall.RObject{RCall.VecSxp}\n", " REMLcrit 8 bytes RCall.RObject{RCall.ClosSxp}\n", " Sv_to_Cv 8 bytes RCall.RObject{RCall.ClosSxp}\n", " VarCorr 8 bytes RCall.RObject{RCall.ClosSxp}\n", " VerbAgg 8 bytes RCall.RObject{RCall.VecSxp}\n", " Vv_to_Cv 8 bytes RCall.RObject{RCall.ClosSxp}\n", " bootMer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " cake 8 bytes RCall.RObject{RCall.VecSxp}\n", " cbpp 8 bytes RCall.RObject{RCall.VecSxp}\n", " confint.merMod 8 bytes RCall.RObject{RCall.ClosSxp}\n", " cov2sdcor 8 bytes RCall.RObject{RCall.ClosSxp}\n", " devcomp 8 bytes RCall.RObject{RCall.ClosSxp}\n", " dummy 8 bytes RCall.RObject{RCall.ClosSxp}\n", " expandDoubleVerts 8 bytes RCall.RObject{RCall.ClosSxp}\n", " factorize 8 bytes RCall.RObject{RCall.ClosSxp}\n", " findbars 8 bytes RCall.RObject{RCall.ClosSxp}\n", " fixef 8 bytes RCall.RObject{RCall.ClosSxp}\n", " formatVC 8 bytes RCall.RObject{RCall.ClosSxp}\n", " fortify.merMod 8 bytes RCall.RObject{RCall.ClosSxp}\n", " getL 8 bytes RCall.RObject{RCall.ClosSxp}\n", " getME 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glFormula 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmFamily 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmResp 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmer.nb 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmerControl 8 bytes RCall.RObject{RCall.ClosSxp}\n", " glmerLaplaceHandle 8 bytes RCall.RObject{RCall.ClosSxp}\n", " golden 8 bytes RCall.RObject{RCall.ClosSxp}\n", " grouseticks 8 bytes RCall.RObject{RCall.VecSxp}\n", " grouseticks_agg 8 bytes RCall.RObject{RCall.VecSxp}\n", " isGLMM 8 bytes RCall.RObject{RCall.ClosSxp}\n", " isLMM 8 bytes RCall.RObject{RCall.ClosSxp}\n", " isNLMM 8 bytes RCall.RObject{RCall.ClosSxp}\n", " isNested 8 bytes RCall.RObject{RCall.ClosSxp}\n", " isREML 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lFormula 8 bytes RCall.RObject{RCall.ClosSxp}\n", " llikAIC 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lmList 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lmResp 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lme4 756 bytes Module\n", " lmer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lmerControl 8 bytes RCall.RObject{RCall.ClosSxp}\n", " lmerResp 8 bytes RCall.RObject{RCall.ClosSxp}\n", " logProf 8 bytes RCall.RObject{RCall.ClosSxp}\n", " merPredD 8 bytes RCall.RObject{RCall.ClosSxp}\n", " methTitle 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkDataTemplate 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkGlmerDevfun 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkLmerDevfun 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkMerMod 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkParsTemplate 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkReTrms 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkRespMod 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mkVarCorr 8 bytes RCall.RObject{RCall.ClosSxp}\n", " mlist2vec 8 bytes RCall.RObject{RCall.ClosSxp}\n", " negative.binomial 8 bytes RCall.RObject{RCall.ClosSxp}\n", " ngrps 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nlformula 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nlmer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nlmerControl 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nloptwrap 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nlsResp 8 bytes RCall.RObject{RCall.ClosSxp}\n", " nobars 8 bytes RCall.RObject{RCall.ClosSxp}\n", " optimizeGlmer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " optimizeLmer 8 bytes RCall.RObject{RCall.ClosSxp}\n", " ranef 8 bytes RCall.RObject{RCall.ClosSxp}\n", " rePos 8 bytes RCall.RObject{RCall.ClosSxp}\n", " refit 8 bytes RCall.RObject{RCall.ClosSxp}\n", " refitML 8 bytes RCall.RObject{RCall.ClosSxp}\n", " sdcor2cov 8 bytes RCall.RObject{RCall.ClosSxp}\n", " show 8 bytes RCall.RObject{RCall.ClosSxp}\n", " sigma 8 bytes RCall.RObject{RCall.ClosSxp}\n", " sleepstudy 8 bytes RCall.RObject{RCall.VecSxp}\n", " subbars 8 bytes RCall.RObject{RCall.ClosSxp}\n", " updateGlmerDevfun 8 bytes RCall.RObject{RCall.ClosSxp}\n", " varianceProf 8 bytes RCall.RObject{RCall.ClosSxp}\n", " vcov.merMod 8 bytes RCall.RObject{RCall.ClosSxp}\n", " vec2STlist 8 bytes RCall.RObject{RCall.ClosSxp}\n", " vec2mlist 8 bytes RCall.RObject{RCall.ClosSxp}\n" ] } ], "source": [ "whos(lme4)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "RCall.RObject{RCall.VecSxp}\n", " INDEX TICKS BROOD HEIGHT YEAR LOCATION cHEIGHT\n", "1 1 0 501 465 95 32 2.759305\n", "2 2 0 501 465 95 32 2.759305\n", "3 3 0 502 472 95 36 9.759305\n", "4 4 0 503 475 95 37 12.759305\n", "5 5 0 503 475 95 37 12.759305\n", "6 6 3 503 475 95 37 12.759305\n", "7 7 2 503 475 95 37 12.759305\n", "8 8 0 504 488 95 44 25.759305\n", "9 9 0 504 488 95 44 25.759305\n", "10 10 2 504 488 95 44 25.759305\n", "11 11 0 505 492 95 47 29.759305\n", "12 12 0 505 492 95 47 29.759305\n", "13 13 0 505 492 95 47 29.759305\n", "14 14 0 506 490 95 45 27.759305\n", "15 15 0 506 490 95 45 27.759305\n", "16 16 0 506 490 95 45 27.759305\n", "17 17 0 507 464 95 31 1.759305\n", "18 18 0 507 464 95 31 1.759305\n", "19 19 0 507 464 95 31 1.759305\n", "20 20 1 507 464 95 31 1.759305\n", "21 21 2 507 464 95 31 1.759305\n", "22 22 1 509 457 95 28 -5.240695\n", "23 23 0 510 457 95 28 -5.240695\n", "24 24 0 511 457 95 28 -5.240695\n", "25 25 5 511 457 95 28 -5.240695\n", "26 26 8 512 451 95 26 -11.240695\n", "27 27 3 512 451 95 26 -11.240695\n", "28 28 4 512 451 95 26 -11.240695\n", "29 29 7 513 437 95 17 -25.240695\n", "30 30 0 513 437 95 17 -25.240695\n", "31 31 4 513 437 95 17 -25.240695\n", "32 32 4 514 430 95 14 -32.240695\n", "33 33 1 514 430 95 14 -32.240695\n", "34 34 0 514 430 95 14 -32.240695\n", "35 35 0 514 430 95 14 -32.240695\n", "36 36 3 514 430 95 14 -32.240695\n", "37 37 1 514 430 95 14 -32.240695\n", "38 38 6 515 427 95 13 -35.240695\n", "39 39 0 515 427 95 13 -35.240695\n", "40 40 1 515 427 95 13 -35.240695\n", "41 41 0 515 427 95 13 -35.240695\n", "42 42 2 516 419 95 7 -43.240695\n", "43 43 7 516 419 95 7 -43.240695\n", "44 44 31 516 419 95 7 -43.240695\n", "45 45 34 517 411 95 4 -51.240695\n", "46 46 17 517 411 95 4 -51.240695\n", "47 47 16 517 411 95 4 -51.240695\n", "48 48 66 518 419 95 7 -43.240695\n", "49 49 49 518 419 95 7 -43.240695\n", "50 50 82 518 419 95 7 -43.240695\n", "51 51 85 518 419 95 7 -43.240695\n", "52 52 64 518 419 95 7 -43.240695\n", "53 53 11 519 424 95 11 -38.240695\n", "54 54 14 519 424 95 11 -38.240695\n", "55 55 4 519 424 95 11 -38.240695\n", "56 56 10 519 424 95 11 -38.240695\n", "57 57 3 520 427 95 13 -35.240695\n", "58 58 15 520 427 95 13 -35.240695\n", "59 59 8 520 427 95 13 -35.240695\n", "60 60 9 521 422 95 9 -40.240695\n", "61 61 11 521 422 95 9 -40.240695\n", "62 62 7 521 422 95 9 -40.240695\n", "63 63 13 521 422 95 9 -40.240695\n", "64 64 3 522 503 95 53 40.759305\n", "65 65 0 523 496 95 51 33.759305\n", "66 66 0 523 496 95 51 33.759305\n", "67 67 1 523 496 95 51 33.759305\n", "68 68 1 523 496 95 51 33.759305\n", "69 69 0 525 507 95 54 44.759305\n", "70 70 0 525 507 95 54 44.759305\n", "71 71 0 525 507 95 54 44.759305\n", "72 72 0 525 507 95 54 44.759305\n", "73 73 0 526 496 95 51 33.759305\n", "74 74 0 526 496 95 51 33.759305\n", "75 75 0 526 496 95 51 33.759305\n", "76 76 1 526 496 95 51 33.759305\n", "77 77 1 526 496 95 51 33.759305\n", "78 78 0 526 496 95 51 33.759305\n", "79 79 2 528 466 95 33 3.759305\n", "80 80 0 528 466 95 33 3.759305\n", "81 81 3 528 466 95 33 3.759305\n", "82 82 1 531 488 95 44 25.759305\n", "83 83 7 533 442 95 19 -20.240695\n", "84 84 2 533 442 95 19 -20.240695\n", "85 85 16 533 442 95 19 -20.240695\n", "86 86 12 533 442 95 19 -20.240695\n", "87 87 0 533 442 95 19 -20.240695\n", "88 88 1 535 442 95 19 -20.240695\n", "89 89 0 537 533 95 63 70.759305\n", "90 90 0 537 533 95 63 70.759305\n", "91 91 1 537 533 95 63 70.759305\n", "92 92 0 537 533 95 63 70.759305\n", "93 93 0 537 533 95 63 70.759305\n", "94 94 1 537 533 95 63 70.759305\n", "95 95 0 537 533 95 63 70.759305\n", "96 96 0 538 533 95 63 70.759305\n", "97 97 0 539 515 95 59 52.759305\n", "98 98 0 539 515 95 59 52.759305\n", "99 99 0 539 515 95 59 52.759305\n", "100 100 0 539 515 95 59 52.759305\n", "101 101 5 540 518 95 60 55.759305\n", "102 102 2 540 518 95 60 55.759305\n", "103 103 2 542 493 95 48 30.759305\n", "104 104 1 542 493 95 48 30.759305\n", "105 105 1 542 493 95 48 30.759305\n", "106 106 0 548 468 95 34 5.759305\n", "107 107 0 548 468 95 34 5.759305\n", "108 108 1 548 468 95 34 5.759305\n", "109 109 1 549 476 95 38 13.759305\n", "110 110 1 549 476 95 38 13.759305\n", "111 111 0 549 476 95 38 13.759305\n", "112 112 0 549 476 95 38 13.759305\n", "113 113 5 550 446 95 22 -16.240695\n", "114 114 3 550 446 95 22 -16.240695\n", "115 115 2 553 460 95 30 -2.240695\n", "116 116 2 559 525 95 62 62.759305\n", "117 117 1 559 525 95 62 62.759305\n", "118 118 1 601 410 96 3 -52.240695\n", "119 119 0 601 410 96 3 -52.240695\n", "120 120 2 601 410 96 3 -52.240695\n", "121 121 5 601 410 96 3 -52.240695\n", "122 122 1 601 410 96 3 -52.240695\n", "123 123 2 601 410 96 3 -52.240695\n", "124 124 2 601 410 96 3 -52.240695\n", "125 125 3 602 417 96 6 -45.240695\n", "126 126 14 602 417 96 6 -45.240695\n", "127 127 11 602 417 96 6 -45.240695\n", "128 128 9 602 417 96 6 -45.240695\n", "129 129 4 602 417 96 6 -45.240695\n", "130 130 10 602 417 96 6 -45.240695\n", "131 131 33 602 417 96 6 -45.240695\n", "132 132 19 602 417 96 6 -45.240695\n", "133 133 16 602 417 96 6 -45.240695\n", "134 134 16 603 430 96 14 -32.240695\n", "135 135 13 603 430 96 14 -32.240695\n", "136 136 11 603 430 96 14 -32.240695\n", "137 137 7 603 430 96 14 -32.240695\n", "138 138 4 603 430 96 14 -32.240695\n", "139 139 11 603 430 96 14 -32.240695\n", "140 140 1 604 456 96 27 -6.240695\n", "141 141 1 604 456 96 27 -6.240695\n", "142 142 4 604 456 96 27 -6.240695\n", "143 143 6 605 457 96 28 -5.240695\n", "144 144 2 605 457 96 28 -5.240695\n", "145 145 7 605 457 96 28 -5.240695\n", "146 146 8 605 457 96 28 -5.240695\n", "147 147 14 605 457 96 28 -5.240695\n", "148 148 6 606 430 96 14 -32.240695\n", "149 149 13 606 430 96 14 -32.240695\n", "150 150 5 606 430 96 14 -32.240695\n", "151 151 8 606 430 96 14 -32.240695\n", "152 152 13 606 430 96 14 -32.240695\n", "153 153 17 606 430 96 14 -32.240695\n", "154 154 5 606 430 96 14 -32.240695\n", "155 155 1 606 430 96 14 -32.240695\n", "156 156 1 606 430 96 14 -32.240695\n", "157 157 2 606 430 96 14 -32.240695\n", "158 158 7 607 423 96 10 -39.240695\n", "159 159 7 608 421 96 8 -41.240695\n", "160 160 11 608 421 96 8 -41.240695\n", "161 161 1 609 525 96 62 62.759305\n", "162 162 0 609 525 96 62 62.759305\n", "163 163 5 610 509 96 55 46.759305\n", "164 164 4 610 509 96 55 46.759305\n", "165 165 0 611 499 96 52 36.759305\n", "166 166 0 611 499 96 52 36.759305\n", "167 167 0 611 499 96 52 36.759305\n", "168 168 7 612 503 96 53 40.759305\n", "169 169 5 612 503 96 53 40.759305\n", "170 170 3 612 503 96 53 40.759305\n", "171 171 1 612 503 96 53 40.759305\n", "172 172 6 612 503 96 53 40.759305\n", "173 173 1 614 492 96 47 29.759305\n", "174 174 2 614 492 96 47 29.759305\n", "175 175 14 615 491 96 46 28.759305\n", "176 176 5 615 491 96 46 28.759305\n", "177 177 27 615 491 96 46 28.759305\n", "178 178 1 616 475 96 37 12.759305\n", "179 179 2 616 475 96 37 12.759305\n", "180 180 3 616 475 96 37 12.759305\n", "181 181 0 616 475 96 37 12.759305\n", "182 182 1 617 479 96 40 16.759305\n", "183 183 0 617 479 96 40 16.759305\n", "184 184 5 617 479 96 40 16.759305\n", "185 185 5 617 479 96 40 16.759305\n", "186 186 8 617 479 96 40 16.759305\n", "187 187 21 617 479 96 40 16.759305\n", "188 188 15 618 472 96 36 9.759305\n", "189 189 15 618 472 96 36 9.759305\n", "190 190 6 618 472 96 36 9.759305\n", "191 191 19 618 472 96 36 9.759305\n", "192 192 14 618 472 96 36 9.759305\n", "193 193 1 621 485 96 42 22.759305\n", "194 194 1 621 485 96 42 22.759305\n", "195 195 3 621 485 96 42 22.759305\n", "196 196 2 621 485 96 42 22.759305\n", "197 197 3 621 485 96 42 22.759305\n", "198 198 2 623 495 96 50 32.759305\n", "199 199 5 623 495 96 50 32.759305\n", "200 200 0 624 472 96 36 9.759305\n", "201 201 6 624 472 96 36 9.759305\n", "202 202 3 624 472 96 36 9.759305\n", "203 203 1 625 458 96 29 -4.240695\n", "204 204 0 625 458 96 29 -4.240695\n", "205 205 1 625 458 96 29 -4.240695\n", "206 206 6 625 458 96 29 -4.240695\n", "207 207 1 625 458 96 29 -4.240695\n", "208 208 85 626 449 96 24 -13.240695\n", "209 209 45 626 449 96 24 -13.240695\n", "210 210 68 626 449 96 24 -13.240695\n", "211 211 84 626 449 96 24 -13.240695\n", "212 212 50 626 449 96 24 -13.240695\n", "213 213 13 628 442 96 19 -20.240695\n", "214 214 1 628 442 96 19 -20.240695\n", "215 215 19 629 448 96 23 -14.240695\n", "216 216 26 629 448 96 23 -14.240695\n", "217 217 9 629 448 96 23 -14.240695\n", "218 218 2 629 448 96 23 -14.240695\n", "219 219 4 629 448 96 23 -14.240695\n", "220 220 3 629 448 96 23 -14.240695\n", "221 221 22 630 448 96 23 -14.240695\n", "222 222 32 630 448 96 23 -14.240695\n", "223 223 5 631 403 96 1 -59.240695\n", "224 224 21 631 403 96 1 -59.240695\n", "225 225 26 631 403 96 1 -59.240695\n", "226 226 13 631 403 96 1 -59.240695\n", "227 227 23 631 403 96 1 -59.240695\n", "228 228 42 632 411 96 4 -51.240695\n", "229 229 38 632 411 96 4 -51.240695\n", "230 230 61 632 411 96 4 -51.240695\n", "231 231 79 632 411 96 4 -51.240695\n", "232 232 39 632 411 96 4 -51.240695\n", "233 233 41 632 411 96 4 -51.240695\n", "234 234 15 634 415 96 5 -47.240695\n", "235 235 23 634 415 96 5 -47.240695\n", "236 236 14 634 415 96 5 -47.240695\n", "237 237 7 635 427 96 13 -35.240695\n", "238 238 24 636 424 96 11 -38.240695\n", "239 239 3 638 525 96 62 62.759305\n", "240 240 1 638 525 96 62 62.759305\n", "241 241 2 640 521 96 61 58.759305\n", "242 242 1 640 521 96 61 58.759305\n", "243 243 0 640 521 96 61 58.759305\n", "244 244 3 641 518 96 60 55.759305\n", "245 245 8 641 518 96 60 55.759305\n", "246 246 1 642 495 96 50 32.759305\n", "247 247 2 642 495 96 50 32.759305\n", "248 248 0 642 495 96 50 32.759305\n", "249 249 8 643 495 96 50 32.759305\n", "250 250 3 643 495 96 50 32.759305\n", "251 251 14 643 495 96 50 32.759305\n", "252 252 16 643 495 96 50 32.759305\n", "253 253 18 643 495 96 50 32.759305\n", "254 254 11 643 495 96 50 32.759305\n", "255 255 13 643 495 96 50 32.759305\n", "256 256 6 645 460 96 30 -2.240695\n", "257 257 7 645 460 96 30 -2.240695\n", "258 258 10 645 460 96 30 -2.240695\n", "259 259 5 647 442 96 19 -20.240695\n", "260 260 7 647 442 96 19 -20.240695\n", "261 261 25 648 443 96 20 -19.240695\n", "262 262 11 648 443 96 20 -19.240695\n", "263 263 6 648 443 96 20 -19.240695\n", "264 264 4 648 443 96 20 -19.240695\n", "265 265 7 648 443 96 20 -19.240695\n", "266 266 4 650 425 96 12 -37.240695\n", "267 267 6 650 425 96 12 -37.240695\n", "268 268 2 650 425 96 12 -37.240695\n", "269 269 5 651 439 96 18 -23.240695\n", "270 270 3 651 439 96 18 -23.240695\n", "271 271 7 651 439 96 18 -23.240695\n", "272 272 3 652 444 96 21 -18.240695\n", "273 273 1 701 450 97 25 -12.240695\n", "274 274 4 701 450 97 25 -12.240695\n", "275 275 4 701 450 97 25 -12.240695\n", "276 276 2 701 450 97 25 -12.240695\n", "277 277 5 701 450 97 25 -12.240695\n", "278 278 3 702 446 97 22 -16.240695\n", "279 279 0 702 446 97 22 -16.240695\n", "280 280 3 702 446 97 22 -16.240695\n", "281 281 1 702 446 97 22 -16.240695\n", "282 282 2 702 446 97 22 -16.240695\n", "283 283 3 702 446 97 22 -16.240695\n", "284 284 1 704 472 97 36 9.759305\n", "285 285 0 704 472 97 36 9.759305\n", "286 286 4 704 472 97 36 9.759305\n", "287 287 0 704 472 97 36 9.759305\n", "288 288 0 704 472 97 36 9.759305\n", "289 289 0 705 472 97 36 9.759305\n", "290 290 0 706 460 97 30 -2.240695\n", "291 291 0 706 460 97 30 -2.240695\n", "292 292 0 706 460 97 30 -2.240695\n", "293 293 1 708 442 97 19 -20.240695\n", "294 294 3 708 442 97 19 -20.240695\n", "295 295 4 708 442 97 19 -20.240695\n", "296 296 0 708 442 97 19 -20.240695\n", "297 297 4 708 442 97 19 -20.240695\n", "298 298 2 708 442 97 19 -20.240695\n", "299 299 0 709 525 97 62 62.759305\n", "300 300 0 709 525 97 62 62.759305\n", "301 301 1 709 525 97 62 62.759305\n", "302 302 0 710 533 97 63 70.759305\n", "303 303 1 710 533 97 63 70.759305\n", "304 304 2 710 533 97 63 70.759305\n", "305 305 0 710 533 97 63 70.759305\n", "306 306 0 710 533 97 63 70.759305\n", "307 307 0 710 533 97 63 70.759305\n", "308 308 1 711 513 97 57 50.759305\n", "309 309 0 711 513 97 57 50.759305\n", "310 310 0 711 513 97 57 50.759305\n", "311 311 1 711 513 97 57 50.759305\n", "312 312 1 711 513 97 57 50.759305\n", "313 313 0 711 513 97 57 50.759305\n", "314 314 0 711 513 97 57 50.759305\n", "315 315 0 712 514 97 58 51.759305\n", "316 316 0 712 514 97 58 51.759305\n", "317 317 0 713 511 97 56 48.759305\n", "318 318 0 713 511 97 56 48.759305\n", "319 319 1 713 511 97 56 48.759305\n", "320 320 0 713 511 97 56 48.759305\n", "321 321 0 713 511 97 56 48.759305\n", "322 322 0 714 511 97 56 48.759305\n", "323 323 0 714 511 97 56 48.759305\n", "324 324 1 714 511 97 56 48.759305\n", "325 325 0 714 511 97 56 48.759305\n", "326 326 0 714 511 97 56 48.759305\n", "327 327 0 714 511 97 56 48.759305\n", "328 328 0 715 496 97 51 33.759305\n", "329 329 0 715 496 97 51 33.759305\n", "330 330 0 715 496 97 51 33.759305\n", "331 331 0 715 496 97 51 33.759305\n", "332 332 1 716 494 97 49 31.759305\n", "333 333 0 716 494 97 49 31.759305\n", "334 334 0 717 494 97 49 31.759305\n", "335 335 0 717 494 97 49 31.759305\n", "336 336 0 717 494 97 49 31.759305\n", "337 337 2 718 411 97 4 -51.240695\n", "338 338 4 718 411 97 4 -51.240695\n", "339 339 4 718 411 97 4 -51.240695\n", "340 340 1 718 411 97 4 -51.240695\n", "341 341 2 718 411 97 4 -51.240695\n", "342 342 2 719 411 97 4 -51.240695\n", "343 343 1 719 411 97 4 -51.240695\n", "344 344 4 719 411 97 4 -51.240695\n", "345 345 1 719 411 97 4 -51.240695\n", "346 346 3 719 411 97 4 -51.240695\n", "347 347 3 719 411 97 4 -51.240695\n", "348 348 2 720 423 97 10 -39.240695\n", "349 349 0 720 423 97 10 -39.240695\n", "350 350 3 721 424 97 11 -38.240695\n", "351 351 2 722 403 97 1 -59.240695\n", "352 352 0 722 403 97 1 -59.240695\n", "353 353 1 722 403 97 1 -59.240695\n", "354 354 0 723 409 97 2 -53.240695\n", "355 355 3 723 409 97 2 -53.240695\n", "356 356 1 723 409 97 2 -53.240695\n", "357 357 10 724 434 97 16 -28.240695\n", "358 358 4 724 434 97 16 -28.240695\n", "359 359 1 724 434 97 16 -28.240695\n", "360 360 2 725 477 97 39 14.759305\n", "361 361 0 725 477 97 39 14.759305\n", "362 362 1 725 477 97 39 14.759305\n", "363 363 0 727 472 97 36 9.759305\n", "364 364 0 728 468 97 34 5.759305\n", "365 365 0 728 468 97 34 5.759305\n", "366 366 0 729 470 97 35 7.759305\n", "367 367 0 730 486 97 43 23.759305\n", "368 368 0 731 495 97 50 32.759305\n", "369 369 0 731 495 97 50 32.759305\n", "370 370 0 731 495 97 50 32.759305\n", "371 371 0 731 495 97 50 32.759305\n", "372 372 0 731 495 97 50 32.759305\n", "373 373 0 732 483 97 41 20.759305\n", "374 374 0 732 483 97 41 20.759305\n", "375 375 2 732 483 97 41 20.759305\n", "376 376 2 733 442 97 19 -20.240695\n", "377 377 2 733 442 97 19 -20.240695\n", "378 378 0 734 457 97 28 -5.240695\n", "379 379 4 736 457 97 28 -5.240695\n", "380 380 5 736 457 97 28 -5.240695\n", "381 381 2 737 457 97 28 -5.240695\n", "382 382 3 737 457 97 28 -5.240695\n", "383 383 2 737 457 97 28 -5.240695\n", "384 384 2 737 457 97 28 -5.240695\n", "385 385 1 737 457 97 28 -5.240695\n", "386 386 2 737 457 97 28 -5.240695\n", "387 387 0 737 457 97 28 -5.240695\n", "388 388 2 738 464 97 31 1.759305\n", "389 389 0 738 464 97 31 1.759305\n", "390 390 1 738 464 97 31 1.759305\n", "391 391 0 739 433 97 15 -29.240695\n", "392 392 3 739 433 97 15 -29.240695\n", "393 393 1 739 433 97 15 -29.240695\n", "394 394 0 739 433 97 15 -29.240695\n", "395 395 0 740 442 97 19 -20.240695\n", "396 396 0 740 442 97 19 -20.240695\n", "397 397 0 741 433 97 15 -29.240695\n", "398 398 1 741 433 97 15 -29.240695\n", "399 399 0 741 433 97 15 -29.240695\n", "400 400 0 742 430 97 14 -32.240695\n", "401 401 0 742 430 97 14 -32.240695\n", "402 402 2 743 450 97 25 -12.240695\n", "403 403 0 743 450 97 25 -12.240695\n" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lme4.grouseticks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [JuliaOpt](http://www.juliaopt.org) packages" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the areas in which Julia shines is optimization packages. I mostly do nonlinear optimization subject to box constraints and use the `NLopt` package. Many other types of optimization problems can be addressed with the `JuMP` package.\n", "\n", "A recent addition is the [JuliaDiff](http://www.juliadiff.org) organization that provides several types of automatic differentiation packages for Julia." ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.4.3", "language": "julia", "name": "julia-0.4" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.4.3" } }, "nbformat": 4, "nbformat_minor": 0 }