{"paragraphs":[{"text":"%md\n\n### Getting credentialed through Facebook\n\n[Facebook App Dashboard](https://developers.facebook.com/apps/)\n\nAdd new App (top right)\n\nAfter created click \"Settings\" to right. Click show to see \"App Secret\"\n\nGo to [Graph API Explorer](https://developers.facebook.com/tools/explorer/) to get Token (they expire regularly)\n\n### Setting up the Stream\n\nFirst you'll need to add some dependencies, including the Facebook **provider** jar:\n\n","dateUpdated":"2016-11-03T22:08:59+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{},"map":{"baseMapType":"Streets","isOnline":true,"pinCols":[]}},"enabled":true,"editorHide":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1478208205252_343847036","id":"20161103-212325_-892897887","result":{"code":"SUCCESS","type":"HTML","msg":"
Add new App (top right)
\nAfter created click “Settings” to right. Click show to see “App Secret”
\nGo to Graph API Explorer to get Token (they expire regularly)
\nFirst you'll need to add some dependencies, including the Facebook provider jar:
\n"},"dateCreated":"2016-11-03T21:23:25+0000","dateStarted":"2016-11-03T22:09:00+0000","dateFinished":"2016-11-03T22:09:00+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:1781"},{"title":"Add Deps","text":"%spark.dep\nz.reset()\nz.addRepo(\"apache-snapshots\").url(\"https://repository.apache.org/content/repositories/snapshots\").snapshot()\nz.load(\"org.apache.streams:streams-provider-facebook:0.4-incubating-SNAPSHOT\")\nz.load(\"org.apache.streams:streams-converters:0.4-incubating-SNAPSHOT\")","dateUpdated":"2016-11-03T22:08:51+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{},"map":{"baseMapType":"Streets","isOnline":true,"pinCols":[]}},"enabled":true,"editorMode":"ace/mode/scala","title":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1477439051383_1294290529","id":"20161025-234411_1038490081","result":{"code":"ERROR","type":"TEXT","msg":"Must be used before SparkInterpreter (%spark) initialized\nHint: put this paragraph before any Spark code and restart Zeppelin/Interpreter"},"dateCreated":"2016-10-25T11:44:11+0000","dateStarted":"2016-11-03T22:08:51+0000","dateFinished":"2016-11-03T22:08:51+0000","status":"ERROR","progressUpdateIntervalMs":500,"$$hashKey":"object:1782"},{"title":"Imports and Credentials","text":"%spark.spark\n\nimport com.typesafe.config._\n\nimport org.apache.streams.config._\nimport org.apache.streams.converter.ActivityConverterProcessor\nimport org.apache.streams.core.StreamsProcessor\nimport org.apache.streams.core._\nimport org.apache.streams.facebook._\nimport org.apache.streams.facebook.FacebookUserInformationConfiguration\nimport org.apache.streams.facebook.FacebookUserstreamConfiguration\nimport org.apache.streams.facebook.Page\nimport org.apache.streams.facebook.Post\nimport org.apache.streams.facebook.processor.FacebookTypeConverter\nimport org.apache.streams.facebook.provider.FacebookFriendFeedProvider\nimport org.apache.streams.jackson.StreamsJacksonMapper;\nimport org.apache.streams.pojo.json.Activity\n\nimport scala.collection.JavaConverters\nimport scala.collection.JavaConversions._\nimport java.util.Iterator\n\nval appId = z.input(\"appId\", \"\") //\nval appSecret = z.input(\"appSecret\", \"\") // \nval userAccessToken = z.input(\"userAccessToken\", \"\") // \n","dateUpdated":"2016-11-03T22:17:17+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{},"map":{"baseMapType":"Streets","isOnline":true,"pinCols":[]}},"enabled":true,"editorMode":"ace/mode/scala","title":true,"editorHide":false},"settings":{"params":{"appId":"","appSecret":"","userAccessToken":"EAACEdEose0cBADsCnKb3Y55nGqo14AmXE7DuPZCFlYz1YD0KQpJxtcpiOAyZBERxNfeBBOtikyZALdZAecR3HucYJkmhHKX89P06XA1HwkGu27ZB8GzjGMvSyZAIgDImqkkGS60eX0ZBzTbWspH44FG25e4H1YNhZBX080ZAUSS4YcgZDZD"},"forms":{"appId":{"name":"appId","displayName":"appId","type":"input","defaultValue":"","hidden":false},"appSecret":{"name":"appSecret","displayName":"appSecret","type":"input","defaultValue":"","hidden":false},"userAccessToken":{"name":"userAccessToken","displayName":"userAccessToken","type":"input","defaultValue":"","hidden":false}}},"apps":[],"jobName":"paragraph_1478208269219_-1037542722","id":"20161103-212429_-781954795","result":{"code":"SUCCESS","type":"TEXT","msg":"\nimport com.typesafe.config._\n\nimport org.apache.streams.config._\n\nimport org.apache.streams.converter.ActivityConverterProcessor\n\nimport org.apache.streams.core.StreamsProcessor\n\nimport org.apache.streams.core._\n\nimport org.apache.streams.facebook._\n\nimport org.apache.streams.facebook.FacebookUserInformationConfiguration\n\nimport org.apache.streams.facebook.FacebookUserstreamConfiguration\n\nimport org.apache.streams.facebook.Page\n\nimport org.apache.streams.facebook.Post\n\nimport org.apache.streams.facebook.processor.FacebookTypeConverter\n\nimport org.apache.streams.facebook.provider.FacebookFriendFeedProvider\n\nimport org.apache.streams.jackson.StreamsJacksonMapper\n\nimport org.apache.streams.pojo.json.Activity\n\nimport scala.collection.JavaConverters\n\nimport scala.collection.JavaConversions._\n\nimport java.util.Iterator\n\nappId: Object = \n\nappSecret: Object = \n\nuserAccessToken: Object = EAACEdEose0cBADsCnKb3Y55nGqo14AmXE7DuPZCFlYz1YD0KQpJxtcpiOAyZBERxNfeBBOtikyZALdZAecR3HucYJkmhHKX89P06XA1HwkGu27ZB8GzjGMvSyZAIgDImqkkGS60eX0ZBzTbWspH44FG25e4H1YNhZBX080ZAUSS4YcgZDZD\n"},"dateCreated":"2016-11-03T21:24:29+0000","dateStarted":"2016-11-03T22:09:07+0000","dateFinished":"2016-11-03T22:09:12+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1783"},{"text":"%spark\n\n\nval credentials =\n s\"\"\"\n {\n facebook {\n oauth {\n appId = \"$appId\"\n appSecret = $appSecret\n userAccessToken = $userAccessToken\n }\n \n info = [\n rawkintrevo\n ]\n \n }\n }\n \"\"\"\nval credentialsConfig = ConfigFactory.parseString(credentials)\n\nval typesafe = ConfigFactory.parseString(credentials)\nval config = new ComponentConfigurator(classOf[FacebookUserstreamConfiguration]).detectConfiguration(typesafe, \"facebook\");\nval provider = new FacebookFriendFeedProvider(config, classOf[org.apache.streams.facebook.Post] );\n\nval timeline_buf = scala.collection.mutable.ArrayBuffer.empty[Object]\n\nprovider.prepare()\nprovider.startStream()\n//while(provider.isRunning()) {\nfor (i <- 0 to 20) {\n val resultSet = provider.readCurrent()\n resultSet.size()\n val iterator = resultSet.iterator();\n while(iterator.hasNext()) {\n val datum = iterator.next();\n //println(datum.getDocument)\n timeline_buf += datum.getDocument\n } \n println(s\"Iteration $i\")\n}\nprovider.cleanUp()\n\n//val converter = new ActivityConverterProcessor()\nval converter = new FacebookTypeConverter(classOf[Post], classOf[Activity])\n\nconverter.prepare()\n\nval status_datums = timeline_buf.map(x => new StreamsDatum(x))\nval activity_datums = status_datums.flatMap(x => converter.process(x)).map(x => x.getDocument.asInstanceOf[Activity])\n\nval mapper = StreamsJacksonMapper.getInstance();\nval activitiesRDD = sc.parallelize(activity_datums.map(o => mapper.writeValueAsString(o)))\n\nval activitiesDF = sqlContext.read.json(activitiesRDD)\n\nactivitiesDF.registerTempTable(\"activities\")","dateUpdated":"2016-11-03T22:09:17+0000","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{},"map":{"baseMapType":"Streets","isOnline":true,"pinCols":[]}},"enabled":true,"editorMode":"ace/mode/scala","editorHide":false,"tableHide":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1477440105858_-1402710081","id":"20161026-000145_-964668043","result":{"code":"SUCCESS","type":"TEXT","msg":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\ncredentials: String = \n\"\n {\n facebook {\n oauth {\n appId = \"\"\n appSecret = \n userAccessToken = EAACEdEose0cBADsCnKb3Y55nGqo14AmXE7DuPZCFlYz1YD0KQpJxtcpiOAyZBERxNfeBBOtikyZALdZAecR3HucYJkmhHKX89P06XA1HwkGu27ZB8GzjGMvSyZAIgDImqkkGS60eX0ZBzTbWspH44FG25e4H1YNhZBX080ZAUSS4YcgZDZD\n }\n \n info = [\n rawkintrevo\n ]\n \n }\n }\n \"\n\ncredentialsConfig: com.typesafe.config.Config = Config(SimpleConfigObject({\"facebook\":{\"info\":[\"rawkintrevo\"],\"oauth\":{\"appId\":\"\",\"appSecret\":\"\",\"userAccessToken\":\"EAACEdEose0cBADsCnKb3Y55nGqo14AmXE7DuPZCFlYz1YD0KQpJxtcpiOAyZBERxNfeBBOtikyZALdZAecR3HucYJkmhHKX89P06XA1HwkGu27ZB8GzjGMvSyZAIgDImqkkGS60eX0ZBzTbWspH44FG25e4H1YNhZBX080ZAUSS4YcgZDZD\"}}}))\n\ntypesafe: com.typesafe.config.Config = Config(SimpleConfigObject({\"facebook\":{\"info\":[\"rawkintrevo\"],\"oauth\":{\"appId\":\"\",\"appSecret\":\"\",\"userAccessToken\":\"EAACEdEose0cBADsCnKb3Y55nGqo14AmXE7DuPZCFlYz1YD0KQpJxtcpiOAyZBERxNfeBBOtikyZALdZAecR3HucYJkmhHKX89P06XA1HwkGu27ZB8GzjGMvSyZAIgDImqkkGS60eX0ZBzTbWspH44FG25e4H1YNhZBX080ZAUSS4YcgZDZD\"}}}))\n\nconfig: org.apache.streams.facebook.FacebookUserstreamConfiguration = org.apache.streams.facebook.FacebookUserstreamConfiguration@6b54bb81[info=[rawkintrevo],pollIntervalMillis=60000,additionalProperties={},protocol=