{"paragraphs":[{"text":"%md\n\n### Setup\n\nAdd the following properties to the Spark interpreter (or create a new interpretter with the following properties added)\n\n```\nspark.kryo.registrator ->\torg.apache.mahout.sparkbindings.io.MahoutKryoRegistrator\nspark.serializer ->\torg.apache.spark.serializer.KryoSerializer\n```\n\nRun the following (is this nessecary?): \n```\n$MAHOUT_HOME/bin/mahout-load-spark-env.sh \n```\n\nRun the following (This is nessecary):\n```\nexport MAHOUT_HOME=[directory into which you checked out Mahout]\nexport SPARK_HOME=[directory where you unpacked Spark]\nexport MASTER=[url of the Spark master]\n```","authenticationInfo":{},"dateUpdated":"May 18, 2016 9:23:42 AM","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/markdown","editorHide":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1463541352726_-393281136","id":"20160517-221552_572661682","result":{"code":"SUCCESS","type":"HTML","msg":"
Add the following properties to the Spark interpreter (or create a new interpretter with the following properties added)
\nspark.kryo.registrator -> org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator\nspark.serializer -> org.apache.spark.serializer.KryoSerializer\n
\nRun the following (is this nessecary?):
\n$MAHOUT_HOME/bin/mahout-load-spark-env.sh \n
\nRun the following (This is nessecary):
\nexport MAHOUT_HOME=[directory into which you checked out Mahout]\nexport SPARK_HOME=[directory where you unpacked Spark]\nexport MASTER=[url of the Spark master]\n
\n"},"dateCreated":"May 17, 2016 10:15:52 PM","dateStarted":"May 18, 2016 9:23:34 AM","dateFinished":"May 18, 2016 9:23:34 AM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:261","focus":true},{"text":"%spark\n// things that get loaded by the shell\n\nimport org.apache.mahout.sparkbindings.shell.Main\nimport org.apache.mahout.drivers.ItemSimilarityDriver\nimport org.apache.mahout.drivers.RowSimilarityDriver\nimport org.apache.mahout.drivers.TrainNBDriver\nimport org.apache.mahout.drivers.TestNBDriver\n\n\n//import org.apache.mahout.driver.MahoutDriver","authenticationInfo":{},"dateUpdated":"May 18, 2016 8:39:07 AM","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala","title":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1463426249157_-920488739","id":"20160516-141729_2021992019","result":{"code":"SUCCESS","type":"TEXT","msg":"import org.apache.mahout.sparkbindings.shell.Main\nimport org.apache.mahout.drivers.ItemSimilarityDriver\nimport org.apache.mahout.drivers.RowSimilarityDriver\nimport org.apache.mahout.drivers.TrainNBDriver\nimport org.apache.mahout.drivers.TestNBDriver\n"},"dateCreated":"May 16, 2016 2:17:29 PM","dateStarted":"May 18, 2016 8:26:02 AM","dateFinished":"May 18, 2016 8:26:04 AM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:262","title":"Shell Drivers (redundant?) "},{"text":"%spark\n\n// https://github.com/apache/mahout/blob/master/examples/bin/spark-shell-plot.mscala\nimport org.apache.mahout.math._\nimport org.apache.mahout.math.scalabindings._\nimport org.apache.mahout.math.drm._\nimport org.apache.mahout.math.scalabindings.RLikeOps._\nimport org.apache.mahout.math.drm.RLikeDrmOps._\nimport org.apache.mahout.sparkbindings._\n\nimplicit val sdc: org.apache.mahout.sparkbindings.SparkDistributedContext = sc2sdc(sc)\n\nval mxRnd = Matrices.symmetricUniformView(5000, 2, 1234)\nval drmRand = drmParallelize(mxRnd)\n\n\nval drmSin = drmRand.mapBlock() {case (keys, block) => \n val blockB = block.like()\n for (i <- 0 until block.nrow) {\n blockB(i, 0) = block(i, 0) \n blockB(i, 1) = Math.sin((block(i, 0) * 8))\n }\n keys -> blockB\n}\n\n// make some 2d plots here sample 1000 units\n/*\nval mxRnd3d = Matrices.symmetricUniformView(50000, 3, 1234)\nval drmRand3d = drmParallelize(mxRnd3d)\n\nval drmGauss = drmRand3d.mapBlock() {case (keys, block) =>\n val blockB = block.like()\n for (i <- 0 until block.nrow) {\n val x: Double = block(i, 0)\n val y: Double = block(i, 1)\n val z: Double = block(i, 2)\n\n blockB(i, 0) = x\n blockB(i, 1) = y\n blockB(i, 2) = Math.exp(-((Math.pow(x, 2)) + (Math.pow(y, 2)))/2)\n }\n keys -> blockB\n}\n*/","authenticationInfo":{},"dateUpdated":"May 18, 2016 8:38:39 AM","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala","editorHide":false,"title":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1463429883345_-1761895189","id":"20160516-151803_2005744291","result":{"code":"SUCCESS","type":"TEXT","msg":"import org.apache.mahout.math._\nimport org.apache.mahout.math.scalabindings._\nimport org.apache.mahout.math.drm._\nimport org.apache.mahout.math.scalabindings.RLikeOps._\nimport org.apache.mahout.math.drm.RLikeDrmOps._\nimport org.apache.mahout.sparkbindings._\nsdc: org.apache.mahout.sparkbindings.SparkDistributedContext = org.apache.mahout.sparkbindings.SparkDistributedContext@73ffd2b4\nmxRnd: org.apache.mahout.math.Matrix = \n{\n 0 =>\t{0:0.4586377101191827,1:0.07261898163580698}\n 1 =>\t{0:0.48977896201757654,1:0.2695201068510176}\n 2 =>\t{0:0.33215452109376786,1:0.2148377346657124}\n 3 =>\t{0:0.4497098649240723,1:0.4331127334380502}\n 4 =>\t{0:-0.03782634247193647,1:-0.32353833540588983}\n 5 =>\t{0:0.15137106418749705,1:0.422446220403861}\n 6 =>\t{0:0.2714115385692545,1:-0.4495233989067956}\n 7 =>\t{0:0.02468155133492185,1:0.49474128114887833}\n 8 =>\t{0:-0.2269662536373416,1:-0.14808249195411455}\n 9 =>\t{0:0.050870692759856756,1:-0.4797329808849356}\n... }\ndrmRand: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@66f9cc0e\ndrmSin: org.apache.mahout.math.drm.DrmLike[Int] = OpMapBlock(org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@66f9cc0e,