{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"kernel": "scala"
},
"source": [
"# Imperative vs functional programming style\n",
"\n",
"In this notebook, the same task is implemented in three different programming languages using two different programming styles. The task in question is figuring out from a [letter dataset](letters.csv) all the authors who write more than 10 letters, and ordering them in decreasing order by their letter count.\n",
"\n",
"## Imperative\n",
"\n",
"Imperative programming is a style where you go through items, gathering new information bit by bit into new data structures."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"kernel": "python3"
},
"outputs": [
{
"data": {
"text/plain": [
"[('Maria Celeste Galilei', 49),\n",
" ('Geri Bocchineri', 49),\n",
" ('Mario Guiducci', 30),\n",
" ('Francesco Niccolini', 16)]"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Python\n",
"import pandas\n",
"dt = pandas.read_csv(\"letters.csv\").to_dict(orient='records')\n",
"# First, create an author name to letter count dictionary\n",
"m = dict()\n",
"# Now, go through all the letters, increasing the by author counts each time for the relevant author\n",
"for row in dt:\n",
" m[row['Author']] = m.get(row['Author'],0) + 1\n",
"# Then, create a second dictionary, and copy into that only the authors and counts where the counts are over 10\n",
"n = dict()\n",
"for key,value in m.items():\n",
" if (value>10):\n",
" n[key] = value\n",
"# Finally, sort the authors by decreasing counts\n",
"sorted(n.items(), key=lambda p: (p[1],p[0]), reverse=True)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"kernel": "ir"
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\t- Geri Bocchineri
\n",
"\t\t- 49
\n",
"\t- Maria Celeste Galilei
\n",
"\t\t- 49
\n",
"\t- Mario Guiducci
\n",
"\t\t- 30
\n",
"\t- Francesco Niccolini
\n",
"\t\t- 16
\n",
"
\n"
],
"text/latex": [
"\\begin{description*}\n",
"\\item[Geri Bocchineri] 49\n",
"\\item[Maria Celeste Galilei] 49\n",
"\\item[Mario Guiducci] 30\n",
"\\item[Francesco Niccolini] 16\n",
"\\end{description*}\n"
],
"text/markdown": [
"Geri Bocchineri\n",
": 49Maria Celeste Galilei\n",
": 49Mario Guiducci\n",
": 30Francesco Niccolini\n",
": 16\n",
"\n"
],
"text/plain": [
" Geri Bocchineri Maria Celeste Galilei Mario Guiducci \n",
" 49 49 30 \n",
" Francesco Niccolini \n",
" 16 "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# R\n",
"library(tidyverse)\n",
"dt <- read_csv(\"letters.csv\",col_types = cols())\n",
"# First, create an author name to letter count dictionary\n",
"m <- numeric(0)\n",
"# Now, go through all the letters, increasing the by author counts each time for the relevant author\n",
"for (i in 1:nrow(dt)) { \n",
" row <- dt[i,]\n",
" tmp <- m[row$Author]\n",
" if (is.na(tmp)) tmp <- 0\n",
" m[row$Author]=tmp+1\n",
"}\n",
"# Then, create a second dictionary, and copy into that only the authors and counts where the counts are over 10\n",
"n <- numeric(0)\n",
"for (i in names(m))\n",
" if (m[i]>10) n[i] <- m[i]\n",
"# Finally, sort the authors by decreasing counts\n",
"sort(n, decreasing = TRUE)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"kernel": "scala"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ArrayBuffer((Geri Bocchineri,49), (Maria Celeste Galilei,49), (Mario Guiducci,30), (Francesco Niccolini,16))\n"
]
},
{
"data": {
"text/plain": [
"\u001b[32mimport \u001b[39m\u001b[36m$ivy.$ \n",
"\u001b[39m\n",
"\u001b[32mimport \u001b[39m\u001b[36mcom.github.tototoshi.csv._\n",
"\u001b[39m\n",
"\u001b[32mimport \u001b[39m\u001b[36mjava.io.File\n",
"\u001b[39m\n",
"\u001b[32mimport \u001b[39m\u001b[36mscala.collection.mutable.HashMap\n",
"\n",
"// First, create an author name to letter count dictionary\n",
"\u001b[39m\n",
"\u001b[36mm\u001b[39m: \u001b[32mHashMap\u001b[39m[\u001b[32mString\u001b[39m, \u001b[32mInt\u001b[39m] = \u001b[33mMap\u001b[39m(\n",
" \u001b[32m\"Gabriello Riccardi\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Antonio Nardi\"\u001b[39m -> \u001b[32m5\u001b[39m,\n",
" \u001b[32m\"Raffaello Magiotti\"\u001b[39m -> \u001b[32m6\u001b[39m,\n",
" \u001b[32m\"Orazio Cavalcanti\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Giovanni Ciampoli\"\u001b[39m -> \u001b[32m4\u001b[39m,\n",
" \u001b[32m\"Francesco Stelluti\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Sebastiano Venier\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Gio. Battista Doni\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Bonaventura Calvalcanti\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Raffaello Visconti\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Mario Guiducci\"\u001b[39m -> \u001b[32m30\u001b[39m,\n",
" \u001b[32m\"Baldassarre Nardi\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Francesco Niccolini\"\u001b[39m -> \u001b[32m16\u001b[39m,\n",
" \u001b[32m\"Gian Giacomo Bouchard\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Benedetto Millini\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Ascanio Piccolomini\"\u001b[39m -> \u001b[32m5\u001b[39m,\n",
" \u001b[32m\"Gio. Camillo Gloriosi\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Carlo Rinuccini\"\u001b[39m -> \u001b[32m3\u001b[39m,\n",
" \u001b[32m\"Francesco Maria Fiorentini\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Cassiano dal Pozzo\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Filippo Magalotti\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Bernardo Conti\"\u001b[39m -> \u001b[32m3\u001b[39m,\n",
" \u001b[32m\"Giovanfrancesco Buonamici\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Luca Degli Albizzi\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Pier Francesco Rinuccini\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Matthias Bernegger\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Giulio Ninci\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Antonio de Ville\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Geri Bocchineri\"\u001b[39m -> \u001b[32m49\u001b[39m,\n",
" \u001b[32m\"Gio. Francesco Tolomei\"\u001b[39m -> \u001b[32m8\u001b[39m,\n",
" \u001b[32m\"Giovanni Vannuccini\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Vincenzo Galilei, Jr.\"\u001b[39m -> \u001b[32m4\u001b[39m,\n",
" \u001b[32m\"Maria Tedaldi\"\u001b[39m -> \u001b[32m6\u001b[39m,\n",
" \u001b[32m\"Benedetto Castelli\"\u001b[39m -> \u001b[32m10\u001b[39m,\n",
" \u001b[32m\"Marcantonio Pieralli\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Girolamo da Sommaia\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
" \u001b[32m\"Antonio Quaratesi\"\u001b[39m -> \u001b[32m2\u001b[39m,\n",
" \u001b[32m\"Pietro Mazzei\"\u001b[39m -> \u001b[32m1\u001b[39m,\n",
"...\n",
"\u001b[36mn\u001b[39m: \u001b[32mHashMap\u001b[39m[\u001b[32mString\u001b[39m, \u001b[32mInt\u001b[39m] = \u001b[33mMap\u001b[39m(\n",
" \u001b[32m\"Francesco Niccolini\"\u001b[39m -> \u001b[32m16\u001b[39m,\n",
" \u001b[32m\"Mario Guiducci\"\u001b[39m -> \u001b[32m30\u001b[39m,\n",
" \u001b[32m\"Geri Bocchineri\"\u001b[39m -> \u001b[32m49\u001b[39m,\n",
" \u001b[32m\"Maria Celeste Galilei\"\u001b[39m -> \u001b[32m49\u001b[39m\n",
")"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"// Scala\n",
"import $ivy.`com.github.tototoshi::scala-csv:1.3.5`\n",
"import com.github.tototoshi.csv._\n",
"import java.io.File\n",
"import scala.collection.mutable.HashMap\n",
"\n",
"// First, create an author name to letter count dictionary\n",
"var m = HashMap[String,Int]()\n",
"// Now, go through all the letters, increasing the by author counts each time for the relevant author\n",
"for (entry <- CSVReader.open(new File(\"letters.csv\")).allWithHeaders)\n",
" m(entry(\"Author\")) = m.getOrElse(entry(\"Author\"),0) + 1\n",
"// Then, create a second dictionary, and copy into that only the authors and counts where the counts are over 10\n",
"var n = HashMap[String,Int]()\n",
"for ((name,count) <- m)\n",
" if (count>10) n(name) = count\n",
"// Finally, sort the authors by decreasing counts\n",
"println(n.toSeq.sortBy(_._2)(Ordering[Int].reverse))"
]
},
{
"cell_type": "markdown",
"metadata": {
"kernel": "scala"
},
"source": [
"## Functional\n",
"\n",
"In functional programming, you chain together applications of functions that each take in some data, transform it in some way, and then pass that transformed data into the next function. Through this chaining of well-thought transformation functions, you end up with the result you desired. Here, the transformations to be done are as follows: \n",
" 1. First, transform the list of database rows into a dictionary so that each author is associated with a list of rows where they feature. For example, you go from `(\"Bocchineri,Rome\",\"Niccolini,Rome\",\"Bocchineri,Florence\")` to `Bocchineri->(\"Bocchineri,Rome\",\"Bocchineri,Florence\");Niccolini->(\"Niccolini,Rome\")`.\n",
" 2. Then, transform the dictionary into one where the grouped lists of rows are replaced by the lengths of the lists. So, from `Bocchineri->(\"Bocchineri,Rome\",\"Bocchineri,Florence\");Niccolini->(\"Niccolini,Rome\")` to `Bocchineri->2;Niccolini->1`.\n",
" 3. Then, you transform this dictionary to one only retaining the entries that have a value more than 10. In the example, you'd be left with no rows, but if you'd limit to more than 1, you'd go from `Bocchineri->2;Niccolini->1` to `Bocchineri->2`.\n",
" 4. Finally, you transform the dictionary into a sequence where the entries are in the proper order according to value.\n",
"\n",
"Often, the transformation functions implemented in ready libraries are what one calls \"higher order functions\", in that what they do depend on code blocks passed to them as parameters (frequently defined inline using language capabilities for anonymous or *lambda* functions). So, there is for example a general function to filter a collection, but you have to tell it yourself using code that the criteria here is to retain only people who've written more than 10 letters.\n",
"\n",
"How well functional programming is supported differs a lot between languages, as seen below:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"kernel": "scala"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vector((Maria Celeste Galilei,49), (Geri Bocchineri,49), (Mario Guiducci,30), (Francesco Niccolini,16))\n"
]
},
{
"data": {
"text/plain": [
"\u001b[32mimport \u001b[39m\u001b[36m$ivy.$ \n",
"\u001b[39m\n",
"\u001b[32mimport \u001b[39m\u001b[36mcom.github.tototoshi.csv._\n",
"\u001b[39m\n",
"\u001b[32mimport \u001b[39m\u001b[36mjava.io.File\n",
"\n",
"/* Scala has very good support for functional programming on all levels, \n",
" with most collection classes defining all the sensible higher order transformation functions, \n",
" as well as robust support for using (anonymous, shorthand) functions as parameters. */\n",
"\n",
"\u001b[39m"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"// Scala\n",
"import $ivy.`com.github.tototoshi::scala-csv:1.3.5`\n",
"import com.github.tototoshi.csv._\n",
"import java.io.File\n",
"\n",
"/* Scala has very good support for functional programming on all levels, \n",
" with most collection classes defining all the sensible higher order transformation functions, \n",
" as well as robust support for using (anonymous, shorthand) functions as parameters. */\n",
"\n",
"println(CSVReader.open(new File(\"letters.csv\")).allWithHeaders.groupBy(_(\"Author\"))\n",
" .mapValues(_.length)\n",
" .filter(_._2>10)\n",
" .toSeq\n",
" .sortBy(_._2)(Ordering[Int].reverse))"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"kernel": "python3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Author\n",
"Maria Celeste Galilei 49.0\n",
"Geri Bocchineri 49.0\n",
"Mario Guiducci 30.0\n",
"Francesco Niccolini 16.0\n",
"dtype: float64\n",
"[('Geri Bocchineri', 49), ('Maria Celeste Galilei', 49), ('Mario Guiducci', 30), ('Francesco Niccolini', 16)]\n",
"[('Geri Bocchineri', 49), ('Maria Celeste Galilei', 49), ('Mario Guiducci', 30), ('Francesco Niccolini', 16)]\n",
"[('Geri Bocchineri', 49), ('Maria Celeste Galilei', 49), ('Mario Guiducci', 30), ('Francesco Niccolini', 16)]\n"
]
}
],
"source": [
"# Python\n",
"import pandas\n",
"\n",
"dt = pandas.read_csv(\"letters.csv\")\n",
"\n",
"# With regard to data wrangling, the Pandas library (https://pandas.pydata.org/) defines commonly \n",
"# needed functions for grouping and filtering in an object oriented way suitable for chaining.\n",
"# However, these do not consistently use function parameters for the conditions (which is why\n",
"# the below uses where() followed by dropna() instead of filter()).\n",
"print((dt.groupby('Author')\n",
" .size()\n",
" .where(lambda x: x>10).dropna()\n",
" .sort_values(ascending=False)))\n",
"\n",
"# Plain Python also has some general machinery for functional programming. However, because \n",
"# 1) this is kept separate from the core collection classes and 2) plain Python isn't object \n",
"# oriented, it doesn't support method chaining, resulting in functional code often being \n",
"# difficult to use and understand. For example, note how here the order in which the functions \n",
"# appear is inverted from the order of application, which isn't as easy to follow as the method \n",
"# chaining approach possible with object oriented approaches\n",
"from itertools import groupby\n",
"dt2 = dt.to_dict(orient='records')\n",
"print(sorted(\n",
" filter(\n",
" lambda k: k[1]>10,\n",
" map(\n",
" lambda k: (k[0],len(list(k[1]))),\n",
" groupby(sorted(dt2,key=lambda p: p['Author']),lambda p: p['Author']) # Core Python groupby only works on already sorted collections, necessitating the additional sort here\n",
" )\n",
" ),\n",
" key=lambda k: k[1],\n",
" reverse = True\n",
"))\n",
"\n",
"# The above conceptual ordering problem can be sidestepped using temporary variables, but it does \n",
"# result in more typing:\n",
"tmp = groupby(sorted(dt2,key=lambda p: p['Author']),lambda p: p['Author'])\n",
"tmp = map(lambda k: (k[0],len(list(k[1]))),tmp)\n",
"tmp = filter(lambda k: k[1]>10,tmp)\n",
"print(sorted(tmp,key=lambda k: k[1],reverse = True))\n",
"\n",
"# Finally, note that Python has language support for writing certain types of list and dictionary \n",
"# transformations as so called compherension expressions (https://en.wikipedia.org/wiki/Set-builder_notation#Parallels_in_programming_languages),\n",
"# which can be used in parts of the process:\n",
"tmp = groupby(sorted(dt2,key=lambda p: p['Author']),lambda p: p['Author'])\n",
"tmp = {k:len(list(v)) for (k,v) in tmp} # dictionary comprehension\n",
"tmp = [(k,v) for (k,v) in tmp.items() if v>10 ] # list comprehension\n",
"print(sorted(tmp,key=lambda k: k[1],reverse = True))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"kernel": "ir"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# A tibble: 4 x 2\n",
" Author n\n",
" \n",
"1 Geri Bocchineri 49\n",
"2 Maria Celeste Galilei 49\n",
"3 Mario Guiducci 30\n",
"4 Francesco Niccolini 16\n",
"# A tibble: 4 x 2\n",
" Author n\n",
" \n",
"1 Geri Bocchineri 49\n",
"2 Maria Celeste Galilei 49\n",
"3 Mario Guiducci 30\n",
"4 Francesco Niccolini 16\n"
]
}
],
"source": [
"# R\n",
"library(tidyverse)\n",
"\n",
"dt <- read_csv(\"letters.csv\",col_types = cols())\n",
"\n",
"# R in general doesn't have shorthand lambda expressions or consistent definitions for the \n",
"# common higher order functions, and isn't object oriented, so cannot benefit from automatic method\n",
"# chaining.\n",
"#\n",
"# However, inside the tidyverse group of libraries (https://www.tidyverse.org), both of these \n",
"# issues are transcended. First, chaining issolved through the definition of the %>% operator, which \n",
"# alters the following function call by inserting its left hand argument as the first parameter. \n",
"# Second, because R as a language allows methods to access their arguments before evaluation, the \n",
"# library is able to extract and evaluate code parameters without needing them to be explicitly defined\n",
"# as functions. Therefore, the following:\n",
"\n",
"print(dt %>% \n",
" group_by(Author) %>% \n",
" tally() %>% \n",
" filter(n>10) %>% \n",
" arrange(desc(n)))\n",
"\n",
"# is exactly same as the following:\n",
"\n",
"print(\n",
" arrange_at(\n",
" filter_at(\n",
" tally(\n",
" group_by_at(dt,\"Author\")\n",
" ),\n",
" \"n\",\n",
" ~.>10 # a formula, something akin to a (lambda) function\n",
" ),\n",
" \"n\",\n",
" ~desc(.) # a formula, something akin to a (lambda) function\n",
" )\n",
")\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "SoS",
"language": "sos",
"name": "sos"
},
"language_info": {
"codemirror_mode": "sos",
"file_extension": ".sos",
"mimetype": "text/x-sos",
"name": "sos",
"nbconvert_exporter": "sos_notebook.converter.SoS_Exporter",
"pygments_lexer": "sos"
},
"sos": {
"kernels": [
[
"python3",
"python3",
"python3",
"#FFD91A"
],
[
"ir",
"ir",
"ir",
"#DCDCDA"
],
[
"scala",
"scala",
"",
""
]
],
"version": "0.17.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}