{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ARC5 - Import et formatage de données de graphe\n",
"\n",
"D'après la liste des projets, allocations de recherche (ADR) et des acteurs (laboratoires, écoles doctorales, partenaires socio-économiques...), nous allons calculer la structure du réseau des partenariats de l'ARC5. \n",
"\n",
"Nous procédons d'abord à **l'import des données** :\n",
"\n",
"1. import de l'ensemble des données concernant les différents acteurs (noms, catégories, etc.)\n",
"2. import de la liste des projets et allocations de recherche\n",
"3. import de la liste des partenariats\n",
"\n",
"Ensuite, nous convertissons les **données de réseaux** :\n",
"\n",
"1. convertir les personnes en liens\n",
"2. convertir les projets et ADR en liens\n",
"\n",
"Enfin, nous **exportons ces données** sous plusieurs formes\n",
"\n",
"1. une **carte interactive** de réseau grâce à l'application en ligne [*Topogram*](http://topogram.io)\n",
"2. un fichier de données de réseaux qui sera réutilisé pour effectuer différents calculs\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#!/usr/bin/env python\n",
"# -*- coding: utf-8 -*-\n",
"\n",
"import csv\n",
"import os\n",
"import json\n",
"import itertools\n",
"from collections import Counter\n",
"\n",
"from slugify import slugify\n",
"\n",
"import networkx as nx\n",
"from networkx.readwrite import write_gpickle\n",
"\n",
"data_dir = os.getcwd()\n",
"\n",
"fichier_projets = \"../final/ARC5-Final - Projets (tous).csv\"\n",
"fichier_partenaires = \"../final/ARC5-Final - Partenariats (OK).csv\"\n",
"fichier_nodes = \"../final/ARC5-Final - Noms (tous).csv\"\n",
"\n",
"# parsing helpers\n",
"project_types = {\n",
" \"ADR\" : \"Thèse\",\n",
" \"projet\" : \"Projet de recherche\",\n",
" \"postdoc\" : \"Recherche post-doctorale\"\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"L'ensemble de fonctions ci-dessous est utilisé pour créer et montrer les différentes données: "
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from IPython.display import display, Markdown, Latex\n",
"\n",
"def show_table( title, array):\n",
" \"\"\"print a table using markdown\"\"\"\n",
" md_table = \"\"\n",
" display(Markdown(\"### \"+title))\n",
" md_table += \"| Ecole Doctorale | Nombre de thèses |\\n\"\n",
" md_table += \"| --- | --- |\\n\"\n",
" for c in Counter(array).most_common():\n",
" md_table +=\"| %s | %s | \\n\"%(c[0], c[1])\n",
" display(Markdown(md_table))\n",
"\n",
"def get_slug(name, type):\n",
" \"\"\" get a clean string ID from name and type\"\"\"\n",
" return \"%s-%s\"%(slugify( name.decode('utf-8') ),type.decode('utf-8'))\n",
"\n",
"def get_project(name, type): \n",
" \"\"\" Retrieve a project based on his name and type\"\"\"\n",
" slug = get_slug(name, type)\n",
" try :\n",
" return G.node[slug]\n",
" except KeyError:\n",
" n=stored_projects[slug]\n",
" node = create_node(n[\"name\"], n[\"type\"], n[\"start\"], n[\"end\"], orga=n[\"orga\"])\n",
" return G.node[slug]\n",
" \n",
"def create_node(name, type, start, end, orga=None, info=None) : \n",
" \"\"\"create the node at the right format in the main graph\"\"\"\n",
" slug = get_slug(name, type)\n",
" \n",
" try :\n",
" if start > G.node[slug][\"start\"] : start = G.node[slug][\"start\"]\n",
" if end > G.node[slug][\"end\"] : start = G.node[slug][\"end\"]\n",
" except:\n",
" start = start\n",
" end = end\n",
" \n",
" node = {}\n",
" node[\"id\"] = slug\n",
" node[\"type\"] = type\n",
" node[\"orga\"] = orga # cluster or ARC ?\n",
" node[\"name\"] = name\n",
" node[\"start\"] = start\n",
" node[\"end\"] = end\n",
" \n",
" if info :\n",
" node[\"info\"]=info\n",
" \n",
" G.add_node(node[\"id\"], node)\n",
" return node[\"id\"]\n",
"\n",
"def merge_edge_data(Graph, e, data):\n",
" \"\"\"\n",
" merge data properly :prevent data within existing edges to be erased\n",
" \"\"\"\n",
" try : \n",
" Graph.edge[e[0]][e[1]]\n",
" except KeyError:\n",
" Graph.add_edge(e[0], e[1])\n",
" \n",
" try:\n",
" Graph.edge[e[0]][e[1]][\"edge_types\"].append(data)\n",
" except KeyError:\n",
" Graph.edge[e[0]][e[1]][\"edge_types\"] = [data]\n",
"\n",
"def save_graph(graph, path):\n",
" \"\"\"save pickle graph for later use\"\"\"\n",
" print \"graph saved at : %s\"%path\n",
" write_gpickle(graph, path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Importer le fichier contenant tous les noms de tous les acteurs et organisations"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"../final/ARC5-Final - Noms (tous).csv\n",
"279 nodes\n",
"0 edges\n"
]
},
{
"data": {
"text/markdown": [
"### Types de nodes dans le fichier d'origine "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| laboratoire | 62 | \n",
"| médiation | 52 | \n",
"| patrimoine | 45 | \n",
"| création | 37 | \n",
"| localité | 36 | \n",
"| etablissement | 14 | \n",
"| enseignement | 11 | \n",
"| ecole-doctorale | 10 | \n",
"| économique | 6 | \n",
"| cst | 6 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"print fichier_nodes\n",
"\n",
"partenaires_types= {}\n",
"\n",
"G = nx.Graph()\n",
"\n",
"with open( os.path.join(data_dir, fichier_nodes), \"r\") as f :\n",
" reader = csv.DictReader(f)\n",
" for line in reader :\n",
" start = int(line[\"Début\"])\n",
" end = int(line[\"Fin\"])\n",
" \n",
" info = {\n",
" \"ville\" : line[\"Ville\"],\n",
" \"lien\" : line[\"Lien\"]\n",
" }\n",
" \n",
" node = create_node(line[\"Nom\"], line[\"Type\"], start, end, info=info)\n",
" partenaires_types[slugify(line[\"Nom\"].decode('utf-8'))] = line[\"Type\"]\n",
"\n",
"print \"%s nodes\"%len(G.nodes())\n",
"print \"%s edges\"%len(G.edges())\n",
"\n",
"show_table( \"Types de nodes dans le fichier d'origine \", [n[1][\"type\"] for n in G.nodes(data=True)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Importer le fichier contenant toutes les allocations de thèses et les projets de recherche"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"../final/ARC5-Final - Projets (tous).csv\n",
"482 nodes\n",
"438 edges\n"
]
},
{
"data": {
"text/markdown": [
"### Types de nodes "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| personne | 91 | \n",
"| laboratoire | 62 | \n",
"| ADR | 57 | \n",
"| projet | 54 | \n",
"| médiation | 52 | \n",
"| patrimoine | 45 | \n",
"| création | 37 | \n",
"| localité | 36 | \n",
"| etablissement | 14 | \n",
"| enseignement | 11 | \n",
"| ecole-doctorale | 10 | \n",
"| économique | 6 | \n",
"| cst | 6 | \n",
"| postdoc | 1 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### Projets par organisation "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| None | 370 | \n",
"| ARC5 | 112 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"stored_projects={}\n",
"\n",
"print fichier_projets\n",
"\n",
"with open( os.path.join(data_dir, fichier_projets), \"r\") as f :\n",
" reader = csv.DictReader(f)\n",
" for line in reader :\n",
" \n",
" if line[\"Nom Projet\"] and line[\"Orga\"] != \"13\" and line[\"Orga\"] != \"14\":\n",
"\n",
" start = int(line[\"Année\"])\n",
" end = start+3\n",
" \n",
" # create project\n",
" projet = create_node(line[\"Nom Projet\"], line[\"Type\"], start, end, orga=line[\"Orga\"]) \n",
" \n",
" # porteur de projet\n",
" porteur = create_node(line[\"Porteurs (nom)\"], \"personne\", start, end)\n",
" \n",
" # get existing\n",
" etablissement = G.node[get_slug(line[\"Etablissement\"], \"etablissement\")][\"id\"]\n",
" laboratoire = G.node[get_slug(line[\"Labo\"], \"laboratoire\")][\"id\"]\n",
" \n",
" # TODOs : ville !\n",
"# ville = G.node[get_slug(line[\"Ville\"], \"localite\")]\n",
"\n",
" edges = [] \n",
" edges.append((projet, etablissement))\n",
" edges.append((projet, laboratoire))\n",
" edges.append((projet, porteur))\n",
" edges.append((laboratoire, porteur))\n",
" \n",
"# edges.append((etablissement, ville))\n",
"# edges.append((laboratoire, ville))\n",
"\n",
" for e in edges :\n",
" merge_edge_data(G, e, { \"type\" : line[\"Type\"], \"name\" : line[\"Nom Projet\"] })\n",
" \n",
" elif line[\"Orga\"] == \"13\" or line[\"Orga\"] == \"14\":\n",
" start = int(line[\"Année\"])\n",
" end = start+3\n",
" stored_projects[get_slug(line[\"Nom Projet\"], line[\"Type\"])] = { \"name\" : line[\"Nom Projet\"], \"type\": line[\"Type\"], \"start\" : start, \"end\" : end, \"orga\" : line[\"Orga\"] }\n",
"\n",
"print \"%s nodes\"%len(G.nodes())\n",
"print \"%s edges\"%len(G.edges())\n",
"show_table( \"Types de nodes \", [n[1][\"type\"] for n in G.nodes(data=True)])\n",
"show_table( \"Projets par organisation \", [n[1][\"orga\"] for n in G.nodes(data=True)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Importer le fichier contenant les partenariats\n",
"\n",
"Nous procédons maintenant à l'import du fichier contenant les partenariats. Chaque ligne contient un partenariat, organisé comme suit :\n",
"\n",
"| Structure\t| Projet\t| début\t| fin\t| type de projet | \n",
"| ---| ---| ---| ---| ---| \n",
"| Académie de Savoie\t| Chaînes Éditoriales Patrimoniales : Corpus Électroniques et Papier (CEP2)\t| 2013| \t2015\t| projet | \n",
"| Académie de Savoie\t| CLELIA 2 : du fonds de manuscrits de Stendhal à d’autres corpus rhône-alpins, valorisation d’une mémoire culturelle collective par l’édition électronique. \t| 2012\t| 2014\t| projet| \n",
"| Acrimed 69 \t | Le passage au numérique des médias locaux entre mutations médiatiques et mutations territoriales : du bouleversement des pratiques professionnelles à la reconfiguration des identités locales\t| 2014\t| 2017\t| ADR | "
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"../final/ARC5-Final - Partenariats (OK).csv\n",
"491 nodes\n",
"713 edges\n"
]
},
{
"data": {
"text/markdown": [
"### Types de nodes après avoir ajouté les partenariats"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| personne | 91 | \n",
"| ADR | 65 | \n",
"| laboratoire | 62 | \n",
"| projet | 55 | \n",
"| médiation | 52 | \n",
"| patrimoine | 45 | \n",
"| création | 37 | \n",
"| localité | 36 | \n",
"| etablissement | 14 | \n",
"| enseignement | 11 | \n",
"| ecole-doctorale | 10 | \n",
"| économique | 6 | \n",
"| cst | 6 | \n",
"| postdoc | 1 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"print fichier_partenaires\n",
"\n",
"\n",
"with open( os.path.join(data_dir, fichier_partenaires), \"r\") as f :\n",
" reader = csv.DictReader(f) \n",
" for i, line in enumerate(reader):\n",
" if line[\"Projet\"] and line[\"Structure\"] : \n",
"\n",
" start = int(line[\"début\"])\n",
" end = int(line[\"fin\"])\n",
" \n",
" type = partenaires_types[slugify(line[\"Structure\"].decode('utf-8'))]\n",
" \n",
" partenaire = G.node[ get_slug( line[\"Structure\"], type)]\n",
" \n",
" # TODO : ville\n",
" # ville = create_node(line[\"Ville\"], \"ville\", start, end)\n",
"\n",
" # get project (only those with partners)\n",
" projet = get_project(line[\"Projet\"], line[\"Type\"])\n",
"\n",
" e = (partenaire[\"id\"], projet[\"id\"])\n",
" merge_edge_data(G, e, { \"type\" : projet[\"type\"], \"name\" : projet[\"name\"] })\n",
"\n",
"print \"%s nodes\"%len(G.nodes())\n",
"print \"%s edges\"%len(G.edges())\n",
"show_table( \"Types de nodes après avoir ajouté les partenariats\", [n[1][\"type\"] for n in G.nodes(data=True)])\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Convertir les personnes en liens\n",
"\n",
"Plutôt que de conserver les personnes (et leurs noms) dans le graphe, nous allons désormais les transformer en liens entre les organisations. Chaque personne ayant des liens entre deux organisations créera donc un lien entre elles puis sera supprimée du graphe. "
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"before : 491 nodes / 713 edges\n",
"after : 400 nodes / 560 edges\n"
]
},
{
"data": {
"text/markdown": [
"### Types de nodes (sans les personnes) "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| ADR | 65 | \n",
"| laboratoire | 62 | \n",
"| projet | 55 | \n",
"| médiation | 52 | \n",
"| patrimoine | 45 | \n",
"| création | 37 | \n",
"| localité | 36 | \n",
"| etablissement | 14 | \n",
"| enseignement | 11 | \n",
"| ecole-doctorale | 10 | \n",
"| économique | 6 | \n",
"| cst | 6 | \n",
"| postdoc | 1 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"graph saved at : ../final/ARC5-nx-with-projects.pickle\n"
]
}
],
"source": [
"print \"before : %s nodes / %s edges\"%(len(G.nodes()),len(G.edges()))\n",
"\n",
"G_without_people = G.copy()\n",
"\n",
"# get all persons in the graph\n",
"persons = [node[0] for node in G_without_people.nodes(data=True) if node[1][\"type\"] == \"personne\"]\n",
"\n",
"for person in persons:\n",
"\n",
" # edges for a single person\n",
" person_edges = G_without_people.edges(person)\n",
" \n",
" # get all nodes linked by a single person\n",
" list_of_person_nodes = []; map(list_of_person_nodes.extend, map(list,person_edges))\n",
" assert len(list_of_person_nodes) == len(person_edges)*2 # make sure we have all nodes\n",
" \n",
" clean_nodes = [n for n in list_of_person_nodes if n != person]\n",
"\n",
" if len(person_edges) > 2 : # if have less than degree of 1 then remove node\n",
"\n",
" # get data from the node to add to the edge\n",
" data = G_without_people.node[person]\n",
" \n",
" # create new edges between all those\n",
" new_edges = list(itertools.combinations(clean_nodes, 2))\n",
"\n",
" # create new edges with merge data info\n",
" for e in new_edges:\n",
" merge_edge_data(G_without_people, e, { \"type\" : \"personne\", \"name\" : None })\n",
"\n",
" # remove person from the graph\n",
" G_without_people.remove_node(person)\n",
"\n",
"print \"after : %s nodes / %s edges\"%(len(G_without_people.nodes()),len(G_without_people.edges()))\n",
"\n",
"show_table( \"Types de nodes (sans les personnes) \", [n[1][\"type\"] for n in G_without_people.nodes(data=True)])\n",
"\n",
"# save graph without people inside\n",
"save_graph(G_without_people, '../final/ARC5-nx-with-projects.pickle')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Convertir les projets et allocations de recherche en liens\n",
"\n",
"De la même façon, les projets et allocations de recherches (ADR) vont désormais être convertis en liens dans le graphe. Les liens ainsi créés vont relier les différentes organisations ayant pris par au projet, puis les projets (ou ADRs) seront supprimés du graphe.\n",
"\n",
"Les titres des projets et ADRs seront stockés dans les liens, afin de pouvoir être consultable ensuite. \n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"before : 400 nodes / 560 edges\n",
"after : 279 nodes / 1021 edges\n"
]
},
{
"data": {
"text/markdown": [
"### Types de nodes (sans personnes ni projets) "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"| Ecole Doctorale | Nombre de thèses |\n",
"| --- | --- |\n",
"| laboratoire | 62 | \n",
"| médiation | 52 | \n",
"| patrimoine | 45 | \n",
"| création | 37 | \n",
"| localité | 36 | \n",
"| etablissement | 14 | \n",
"| enseignement | 11 | \n",
"| ecole-doctorale | 10 | \n",
"| économique | 6 | \n",
"| cst | 6 | \n"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"graph saved at : ../final/ARC5-nx-without-projects.pickle\n"
]
}
],
"source": [
"print \"before : %s nodes / %s edges\"%(len(G_without_people.nodes()),len(G_without_people.edges()))\n",
"\n",
"G_without_people_and_projects = G_without_people.copy()\n",
"\n",
"# get all projects in the graph\n",
"projects = [node[0] for node in G_without_people_and_projects.nodes(data=True) if node[1][\"type\"] == \"projet\" or node[1][\"type\"] == \"ADR\" or node[1][\"type\"] == \"postdoc\" ]\n",
"\n",
"for project in projects:\n",
"\n",
" # edges for a single person\n",
" project_edges = G_without_people_and_projects.edges(project)\n",
" \n",
" # get all nodes linked by a single person\n",
" list_of_project_nodes = []; map(list_of_project_nodes.extend, map(list, project_edges))\n",
" assert len(list_of_project_nodes) == len(project_edges)*2 # make sure we have all nodes\n",
" \n",
" clean_nodes = [n for n in list_of_project_nodes if n != project]\n",
"\n",
" if len(project_edges) > 2 : # if have less than degree of 1 then remove node\n",
"\n",
" # get data from the node to add to the edge\n",
" data = G_without_people_and_projects.node[project]\n",
" \n",
" # create new edges between all those\n",
" new_edges = list(itertools.combinations(clean_nodes, 2))\n",
" \n",
" # parse text properly\n",
" \n",
" # merge data into edge info\n",
" for e in new_edges:\n",
" \n",
" proj=G.node[project]\n",
" notes = { \"type\" : proj[\"type\"], \"name\" : proj[\"name\"]}\n",
" merge_edge_data(G_without_people_and_projects, e, notes) \n",
"\n",
" # remove person from the graph\n",
" G_without_people_and_projects.remove_node(project)\n",
"\n",
"\n",
"print \"after : %s nodes / %s edges\"%(len(G_without_people_and_projects.nodes()),len(G_without_people_and_projects.edges()))\n",
"\n",
"show_table( \"Types de nodes (sans personnes ni projets) \", [n[1][\"type\"] for n in G_without_people_and_projects.nodes(data=True)])\n",
"\n",
"# save graph without projects\n",
"save_graph(G_without_people_and_projects, '../final/ARC5-nx-without-projects.pickle')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Obtenir le graphe final\n",
"\n",
"Dans le graph final, nous supprimons les noeuds ayant un degré nul (ceux qui n'ont aucune connection), car il n'apporte que très peu d'information. Egalement, nous attribuons aux liens un poids égal au nombre de projets, personnel ou ADRs en commun. \n",
"\n",
"Une dernière étape constite à convertir les données stockées dans les liens (liste de projets et ADRs) en une forme agréable à lire qui pourra ensuite être affichée dans l'interface de navigation du logiciel *Topogram*. "
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"210 nodes\n",
"1021 edges\n"
]
}
],
"source": [
"# create the graph\n",
"\n",
"nodes = []\n",
"for n in G_without_people_and_projects.nodes(data=True): \n",
" if G_without_people_and_projects.degree(n[0]) > 0: # ignore singletons\n",
" node = n[1]\n",
" node[\"id\"] = n[0]\n",
" node[\"group\"] = n[1][\"type\"]\n",
" \n",
" # add website and city\n",
" node[\"additionalInfo\"] = \"**Ville** : %s \\n\\n \"%node[\"info\"][\"ville\"]\n",
" node[\"additionalInfo\"] += \"[Consulter le site](%s)\"%node[\"info\"][\"lien\"]\n",
" nodes.append(node)\n",
"\n",
"print \"%s nodes\"%len(nodes)\n",
" \n",
"edges = []\n",
"for i, e in enumerate(G_without_people_and_projects.edges(data=True)): \n",
" \n",
" edge = e[2]\n",
" \n",
" # calculate edge weight\n",
" edge[\"weight\"] = len(edge[\"edge_types\"])\n",
" \n",
" notes = \"\"\n",
" team = 0\n",
" \n",
" for t in edge[\"edge_types\"]:\n",
" if t[\"type\"] == \"ADR\" or t[\"type\"] == \"projet\" or t[\"type\"] == \"postdoc\" : \n",
" notes = notes + \"* **%s** : %s \\n\"%(project_types[t[\"type\"]], t[\"name\"])\n",
" elif t[\"type\"] == \"personne\":\n",
" team = team + 1\n",
" \n",
" if team != 0 : \n",
" notes = \"* Membres d'équipe en commun \\n\" + notes\n",
" \n",
" edge[\"additionalInfo\"] = notes \n",
" \n",
" edge[\"source\"] = e[0]\n",
" edge[\"target\"] = e[1]\n",
" \n",
" edges.append(edge)\n",
"\n",
"print \"%s edges\"%len(edges)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Creér le graphe final sur Topogram\n",
"\n",
"Maintenant que toutes nos données ont été traitées et formattées correctement, nous pouvons créer ou mettre à jour la visualisation de notre graphe, rendu disponible en ligne grâce au logiciel [*Topogram*](http://topogram.io).\n",
"\n",
"Pour écrire la carte depuis ce script, nous utilisons le [client API Python](https://github.com/topogram/topogram-api-client) qui nous permet de manipuler les graphes présents dans le service *Topogram* depuis une machine tierce. La mise à jour se fait donc de façon programmatique, après voir supprimé le contenu existant de la carte.\n"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"A topogram with the same name already exists\n",
"1024 existing edges, 210 existing nodes\n",
"nodes deleted\n",
"edges deleted\n",
"210 nodes created.\n",
"1021 edges created.\n",
"done. Topogram is online at https://app.topogram.io/topograms/3Fep7oZAFjqBnHLQR/view\n"
]
}
],
"source": [
"from topogram_client import TopogramAPIClient\n",
"import logging \n",
"import datetime\n",
"\n",
"now=datetime.datetime.now().strftime(\"%Y-%m-%d_%H:%M:%S\")\n",
"\n",
"# passwords\n",
"TOPOGRAM_URL = \"https://app.topogram.io\" # http://localhost:3000\n",
"USER = \"***\"\n",
"PASSWORD = \"***\"\n",
"\n",
"# connect to the topogram instance \n",
"topogram = TopogramAPIClient(TOPOGRAM_URL)\n",
"\n",
"# topogram.create_user(USER, PASSWORD)\n",
"topogram.user_login(USER, PASSWORD)\n",
"\n",
"r = topogram.create_topogram(\"ARC 5 - Collaborations Culture / Recherche en Rhône-Alpes\")\n",
"print r[\"message\"]\n",
"topogram_ID = r[\"data\"][0][\"_id\"]\n",
"\n",
"# get and backup existing nodes and edges\n",
"existing_nodes = topogram.get_nodes(topogram_ID)[\"data\"]\n",
"url = slugify(TOPOGRAM_URL.decode('utf-8'))\n",
"with open('data/ARC5-%s-nodes-%s.json'%(url,now), 'w') as outfile:\n",
" json.dump(existing_nodes, outfile)\n",
"\n",
"existing_edges = topogram.get_edges(topogram_ID)[\"data\"]\n",
"with open('data/ARC5-%s-edges-%s.json'%(url,now), 'w') as outfile:\n",
" json.dump(existing_edges, outfile)\n",
"\n",
"print \"%s existing edges, %s existing nodes\"%(len(existing_edges), len(existing_nodes))\n",
"\n",
"# clear existing graph\n",
"topogram.delete_nodes([n[\"_id\"] for n in existing_nodes])\n",
"print \"nodes deleted\"\n",
"topogram.delete_edges([n[\"_id\"] for n in existing_edges])\n",
"print \"edges deleted\"\n",
"\n",
"r = topogram.create_nodes(topogram_ID, nodes)\n",
"print \"%s nodes created.\"%len(r[\"data\"])\n",
"r = topogram.create_edges(topogram_ID, edges)\n",
"print \"%s edges created.\"%len(r[\"data\"])\n",
"\n",
"print \"done. Topogram is online at %s/topograms/%s/view\"%(TOPOGRAM_URL, topogram_ID)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"CodeCell": {
"cm_config": {
"lineWrapping": true
}
},
"MarkdownCell": {
"cm_config": {
"lineWrapping": true
}
},
"anaconda-cloud": {},
"celltoolbar": "Raw Cell Format",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}