{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "[![Binder](img/badge-binder.svg)](https://mybinder.org/v2/gh/nhirschey/teaching/gh-pages?filepath=signal-portfolio.ipynb)\u0026emsp;\n",
    "[![Script](img/badge-script.svg)](/Teaching//signal-portfolio.fsx)\u0026emsp;\n",
    "[![Notebook](img/badge-notebook.svg)](/Teaching//signal-portfolio.ipynb)\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "#r \"nuget: FSharp.Data, 5.0.2\"\n",
    "#r \"nuget: FSharp.Stats, 0.5.0\"\n",
    "#r \"nuget: Plotly.NET, 3.*\"\n",
    "#r \"nuget: Plotly.NET.Interactive, 3.*\"\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "open System\n",
    "open FSharp.Data\n",
    "open Plotly.NET\n",
    "open FSharp.Stats\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "## Load Data\n",
    "\n",
    "First, make sure that you\u0027re referencing the correct files.\n",
    "\n",
    "Here I\u0027m assuming that you have a class folder with this `signal-exploration.ipynb` notebook and a `data` folder inside of it. The folder hierarchy would look like below where you\n",
    "have the below files and folders accessible:\n",
    "\n",
    "```code\n",
    "/class\n",
    "    signal-portfolio.ipynb\n",
    "    id_and_return_data.csv\n",
    "    zero_trades_252d.csv\n",
    "    \n",
    "```\n",
    "\n",
    "First, make sure that our working directory is the source file directory.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let [\u003cLiteral\u003e] ResolutionFolder = __SOURCE_DIRECTORY__\n",
    "Environment.CurrentDirectory \u003c- ResolutionFolder\n",
    "#endif // ipynb\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "### We will use the portfolio module\n",
    "\n",
    "We will use the [Portfolio module](https://github.com/nhirschey/NovaSBE.Finance/blob/main/src/NovaSBE.Finance/Portfolio.fs). Make sure that you load this code\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "#r \"nuget: NovaSBE.Finance, 0.5.0\"\n",
    "open NovaSBE.Finance\n",
    "open NovaSBE.Finance.Portfolio\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "### Data files\n",
    "\n",
    "We assume the `id_and_return_data.csv` file and the signal csv file are in the same folder as the notebook. In this example the signal file is `be_me.csv`. You should replace that file name with your signal file name.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let [\u003cLiteral\u003e] IdAndReturnsFilePath = \"id_and_return_data.csv\"\n",
    "let [\u003cLiteral\u003e] MySignalFilePath = \"be_me.csv\"\n",
    "let strategyName = \"book to market\"\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "If my paths are correct, then this code should read the first few lines of the files.\n",
    "If it doesn\u0027t show the first few lines, fix the above file paths.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "IO.File.ReadLines(IdAndReturnsFilePath) |\u003e Seq.truncate 5\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "IO.File.ReadLines(MySignalFilePath) |\u003e Seq.truncate 5\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Assuming the paths are defined correctly and you saw the first 5 rows above,\n",
    "we can now read the data using the CSV provider that parses the fields in the file.\n",
    "\n",
    "First define the Csv types from the sample files:\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "type IdAndReturnsType = \n",
    "    CsvProvider\u003cSample=IdAndReturnsFilePath,\n",
    "                ResolutionFolder=ResolutionFolder\u003e\n",
    "\n",
    "type MySignalType = \n",
    "    CsvProvider\u003cMySignalFilePath,\n",
    "                ResolutionFolder=ResolutionFolder\u003e\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Now read in the data.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let idAndReturnsCsv = IdAndReturnsType.GetSample()\n",
    "\n",
    "let mySignalCsv = MySignalType.GetSample()\n",
    "    \n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Columns in the `idAndReturnsCsv` are:\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "idAndReturnsCsv.Headers\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Columns in the `mySignalCsv` are:\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "mySignalCsv.Headers\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "There are a lot of columns in the id and returns csv. You can look at the data documentation to figure out what they are.\n",
    "\n",
    "Put the rows into a list (we\u0027re more familiar with lists).\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let idAndReturnsRows = idAndReturnsCsv.Rows |\u003e Seq.toList\n",
    "let mySignalRows = mySignalCsv.Rows |\u003e Seq.toList\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "We want to be able to look up idAndReturn data\n",
    "and signal data using a security\u0027s ID and month.\n",
    "To do that, we create a Map collection where the key\n",
    "is a tuple of the security id and month.\n",
    "\n",
    "* In this dataset, we\u0027ll use `row.Id` as the identifier. We\u0027ll assign it to\n",
    "the `Other` SecurityId case, because it\u0027s a dataset specific one.\n",
    "\n",
    "* In this dataset, the Eom variable defines the \"end of month\".\n",
    "\n",
    "* The returns are for the month ending in EOM.\n",
    "\n",
    "* The signals are \"known\" as of EOM. So you can use them on/after EOM. We\u0027ll\n",
    "form portfolios in the month ending EOM; that\u0027s the `FormationMonth`.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let msfBySecurityIdAndMonth =\n",
    "    idAndReturnsRows\n",
    "    |\u003e List.map(fun row -\u003e \n",
    "        let id = Other row.Id\n",
    "        let month = DateTime(row.Eom.Year,row.Eom.Month,1)\n",
    "        let key = id, month\n",
    "        key, row)\n",
    "    |\u003e Map    \n",
    "\n",
    "let signalBySecurityIdAndMonth =\n",
    "    mySignalRows\n",
    "    |\u003e List.choose(fun row -\u003e \n",
    "        // we\u0027ll use choose to drop the security if the signal is None.\n",
    "        // The signal is None when it is missing.\n",
    "        match row.Signal with\n",
    "        | None -\u003e None // choose will drop these None observations\n",
    "        | Some signal -\u003e\n",
    "            let id = Other row.Id\n",
    "            let month = DateTime(row.Eom.Year,row.Eom.Month,1)\n",
    "            let key = id, month\n",
    "            // choose will convert Some(key,signal) into\n",
    "            // (key,signal) and keep that.\n",
    "            Some (key, signal))\n",
    "    |\u003e Map    \n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "The `securitiesByFormationMonth` that we\u0027ll use to define our investment universe.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let securitiesByFormationMonth =\n",
    "    idAndReturnsRows\n",
    "    |\u003e List.groupBy(fun x -\u003e DateTime(x.Eom.Year, x.Eom.Month,1))\n",
    "    |\u003e List.map(fun (ym, obsThisMonth) -\u003e \n",
    "        let idsThisMonth = [ for x in obsThisMonth do Other x.Id ]\n",
    "        ym, idsThisMonth)\n",
    "    |\u003e Map\n",
    "\n",
    "let getInvestmentUniverse formationMonth =\n",
    "    match Map.tryFind formationMonth securitiesByFormationMonth with\n",
    "    | Some securities -\u003e \n",
    "        { FormationMonth = formationMonth \n",
    "          Securities = securities }\n",
    "    | None -\u003e failwith $\"{formationMonth} is not in the date range\"\n",
    "   \n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Checking universe\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let testUniverseObs = getInvestmentUniverse (DateTime(2015,4,1))\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Formation month.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "testUniverseObs.FormationMonth\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "First few securities\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "testUniverseObs.Securities[0..4]\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Now I want to be able to get my signal.\n",
    "We\u0027re going to assume here that a \"high\" signal\n",
    "predicts high returns. If you have a signal where\n",
    "a \"high\" signal predicts low returns, you can\n",
    "multiply the signal by `-1.0` below.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let getMySignal (securityId, formationMonth) =\n",
    "    match Map.tryFind (securityId, formationMonth) signalBySecurityIdAndMonth with\n",
    "    | None -\u003e None\n",
    "    | Some signal -\u003e\n",
    "        Some { SecurityId = securityId \n",
    "               // if a high signal means low returns,\n",
    "               // use `-signal` here instead of `signal`\n",
    "               Signal = signal }\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Test it\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "[ for securityId in testUniverseObs.Securities[0..4] do\n",
    "    let testObs = (securityId, testUniverseObs.FormationMonth)\n",
    "    getMySignal testObs ]\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "A function to do it for the whole investment universe.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let getMySignals (investmentUniverse: InvestmentUniverse) =\n",
    "    let listOfSecuritySignals =\n",
    "        investmentUniverse.Securities\n",
    "        |\u003e List.choose(fun security -\u003e \n",
    "            getMySignal (security, investmentUniverse.FormationMonth))    \n",
    "    \n",
    "    { FormationMonth = investmentUniverse.FormationMonth \n",
    "      Signals = listOfSecuritySignals }\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "And I should be able to get my market capitalization\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "msfBySecurityIdAndMonth\n",
    "|\u003e Map.toList\n",
    "|\u003e List.take 3\n",
    "|\u003e List.map (fun ((id, month), row) -\u003e id, row.MarketEquity)\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Now a function to do this.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let getMarketCap (security, formationMonth) =\n",
    "    match Map.tryFind (security, formationMonth) msfBySecurityIdAndMonth with\n",
    "    | None -\u003e None\n",
    "    | Some row -\u003e \n",
    "        match row.MarketEquity with\n",
    "        | None -\u003e None\n",
    "        | Some me -\u003e Some (security, me)\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "And I should be able to get my returns.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let getSecurityReturn (security, formationMonth) =\n",
    "    // If the security has a missing return, assume that we got 0.0.\n",
    "    // Note: If we were doing excess returns, we would need 0.0 - rf.\n",
    "    let missingReturn = 0.0\n",
    "    match Map.tryFind (security, formationMonth) msfBySecurityIdAndMonth with\n",
    "    | None -\u003e security, missingReturn\n",
    "    | Some x -\u003e  \n",
    "        match x.Ret with \n",
    "        | None -\u003e security, missingReturn\n",
    "        | Some r -\u003e security, r\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "The data is already filtered to valid securities based on the data documentation\n",
    "section 1.2, \"How to use the data\", from the paper the data came from.\n",
    "\n",
    "Define sample months\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let startSample = \n",
    "    idAndReturnsRows\n",
    "    |\u003e List.map(fun row -\u003e DateTime(row.Eom.Year,row.Eom.Month,1))\n",
    "    |\u003e List.min\n",
    "\n",
    "let endSample = \n",
    "    let lastMonthWithData = \n",
    "        idAndReturnsRows\n",
    "        |\u003e Seq.map(fun row -\u003e DateTime(row.Eom.Year,row.Eom.Month,1))\n",
    "        |\u003e Seq.max\n",
    "    // The end of sample is the last month when we have returns.\n",
    "    // So the last month when we can form portfolios is one month\n",
    "    // before that.\n",
    "    lastMonthWithData.AddMonths(-1) \n",
    "\n",
    "let sampleMonths = getSampleMonths (startSample, endSample)\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Strategy function\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let formStrategy ym =\n",
    "    ym\n",
    "    |\u003e getInvestmentUniverse\n",
    "    |\u003e getMySignals\n",
    "    |\u003e assignSignalSort strategyName 3\n",
    "    |\u003e Seq.toList\n",
    "    |\u003e List.map (giveValueWeights getMarketCap)\n",
    "    |\u003e List.map (getPortfolioReturn getSecurityReturn)  \n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Your strategy portfolios\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let doParallel = true\n",
    "let portfolios =\n",
    "    if doParallel then\n",
    "        sampleMonths\n",
    "        |\u003e List.toArray\n",
    "        |\u003e Array.Parallel.map formStrategy\n",
    "        |\u003e Array.toList\n",
    "        |\u003e List.collect id\n",
    "    else\n",
    "        sampleMonths\n",
    "        |\u003e List.collect formStrategy\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "A few of the portfolio return observations.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "portfolios[..2]\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "These portfolios were value-weighted. Can you do a version that is equal-weighted?.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let giveEqualWeights (port: AssignedPortfolio): Portfolio =\n",
    "    let makePosition securityId weight : Position =\n",
    "        { SecurityId = securityId; Weight = weight }\n",
    "    \n",
    "    { FormationMonth = failwith \"unimplemented\"\n",
    "      Name = failwith \"unimplemented\"\n",
    "      Index = failwith \"unimplemented\"\n",
    "      Positions = failwith \"unimplemented\" }\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Now make the equal-weight strategy.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let formEqualWeightStrategy ym =\n",
    "    ym\n",
    "    |\u003e getInvestmentUniverse\n",
    "    |\u003e getMySignals\n",
    "    |\u003e assignSignalSort strategyName 3\n",
    "    |\u003e Seq.toList\n",
    "    |\u003e List.map giveEqualWeights\n",
    "    |\u003e List.map (getPortfolioReturn getSecurityReturn)  \n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "If you have defined `giveEqualWeights` above then\n",
    "you can calculate equal weight portfolios with\n",
    "\n",
    "```fsharp\n",
    "let portfoliosEW = sampleMonths |\u003e List.collect formEqualWeightStrategy\n",
    "```\n",
    "## Plotting returns\n",
    "\n",
    "Common.fsx has some easy to use code to get Fama-French factors.\n",
    "We\u0027re going to use the French data to get monthly risk-free rates.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "open NovaSBE.Finance.French\n",
    "\n",
    "let ff3 = getFF3 Frequency.Monthly\n",
    "let monthlyRiskFreeRate =\n",
    "    [ for obs in ff3 do \n",
    "        let key = DateTime(obs.Date.Year,obs.Date.Month,1)\n",
    "        key, obs.Rf ]\n",
    "    |\u003e Map\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Now I\u0027ll convert my portfolios into excess returns.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let portfolioExcessReturns =\n",
    "    portfolios\n",
    "    |\u003e List.map(fun x -\u003e \n",
    "        match Map.tryFind x.Month monthlyRiskFreeRate with \n",
    "        | None -\u003e failwith $\"Can\u0027t find risk-free rate for {x.Month}\"\n",
    "        | Some rf -\u003e { x with Return = x.Return - rf })\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "### Single portfolio plot\n",
    "\n",
    "Let\u0027s plot the top portfolio, calling it long.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let long = \n",
    "    portfolioExcessReturns \n",
    "    |\u003e List.filter(fun x -\u003e x.Index = 3)\n",
    "\n",
    "let cumulateSimpleReturn (xs: PortfolioReturn list) =\n",
    "    let xs = xs |\u003e List.sortBy (fun x -\u003e x.Month)\n",
    "    let mutable cr = 1.0\n",
    "    [ for x in xs do \n",
    "        cr \u003c- cr * (1.0 + x.Return)\n",
    "        { x with Return = cr - 1.0 } ]\n",
    "\n",
    "let longCumulative = long |\u003e cumulateSimpleReturn\n",
    "\n",
    "let longCumulativeChart =\n",
    "    longCumulative\n",
    "    |\u003e List.map(fun x -\u003e x.Month, x.Return)\n",
    "    |\u003e Chart.Line \n",
    "    |\u003e Chart.withTitle \"Growth of 1 Euro\"\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [
          {
           "data": {
            "text/plain": ["Could not find reference \u0027cell37\u0027"]
        },
           "execution_count": null,
           "metadata": {},
           "output_type": "execute_result"
          }],
   "source": [
    "longCumulativeChart\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "And function to do the plot\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let portfolioReturnPlot (xs:PortfolioReturn list) =\n",
    "    xs\n",
    "    |\u003e List.map(fun x -\u003e x.Month, x.Return)\n",
    "    |\u003e Chart.Line \n",
    "    |\u003e Chart.withTitle \"Growth of 1 Euro\"\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Using the function:\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let longWithFunctionsPlot =\n",
    "    long\n",
    "    |\u003e cumulateSimpleReturn\n",
    "    |\u003e portfolioReturnPlot\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [
          {
           "data": {
            "text/plain": ["Could not find reference \u0027cell41\u0027"]
        },
           "execution_count": null,
           "metadata": {},
           "output_type": "execute_result"
          }],
   "source": [
    "longWithFunctionsPlot\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "### Multiple portfolio plot\n",
    "\n",
    "Now let\u0027s use the functions to do a bunch of portfolios at once.\n",
    "\n",
    "First, let\u0027s add a version of value-weighted market portfolio that\n",
    "has the same time range and same F# type as our portfolios.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let vwMktRf =\n",
    "    let portfolioMonths = \n",
    "        portfolioExcessReturns \n",
    "        |\u003e List.map(fun x -\u003e x.Month)\n",
    "    let minYm = portfolioMonths |\u003e List.min\n",
    "    let maxYm = portfolioMonths |\u003e List.max\n",
    "    \n",
    "    [ for x in ff3 do\n",
    "        if x.Date \u003e= minYm \u0026\u0026 x.Date \u003c= maxYm then\n",
    "            { Name = \"Mkt-Rf\"\n",
    "              Index = 1\n",
    "              Month = x.Date\n",
    "              Return = x.MktRf } ]\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "Let\u0027s also create a long-short portfolio.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let short = \n",
    "    portfolioExcessReturns \n",
    "    |\u003e List.filter(fun x -\u003e \n",
    "        x.Name = strategyName \u0026\u0026 x.Index = 1)\n",
    "\n",
    "let longShort = \n",
    "    // We\u0027ll loop through the long portfolio observations,\n",
    "    // looking for the short portfolio observation for that month.\n",
    "    // For efficiently looking up the short portfolio by month,\n",
    "    // put it in a Map collection indexed by month.\n",
    "    let shortByYearMonthMap = \n",
    "        short \n",
    "        |\u003e List.map(fun row -\u003e row.Month, row) \n",
    "        |\u003e Map\n",
    "    \n",
    "    [ for longObs in long do\n",
    "        match Map.tryFind longObs.Month shortByYearMonthMap with\n",
    "        | None -\u003e failwith \"probably your date variables are not aligned for a weird reason\"\n",
    "        | Some shortObs -\u003e\n",
    "            { Name = \"Long-Short\"\n",
    "              Index = 1\n",
    "              Month = longObs.Month\n",
    "              Return = longObs.Return - shortObs.Return } ] \n",
    "  \n",
    "\n",
    "let combinedChart =\n",
    "    List.concat [long; longShort; vwMktRf]\n",
    "    |\u003e List.groupBy(fun x -\u003e x.Name, x.Index)\n",
    "    |\u003e List.map(fun ((name, index), xs) -\u003e\n",
    "        xs\n",
    "        |\u003e cumulateSimpleReturn\n",
    "        |\u003e portfolioReturnPlot\n",
    "        |\u003e Chart.withTraceInfo (Name=($\"{name}: {index}\")))\n",
    "    |\u003e Chart.combine\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [
          {
           "data": {
            "text/plain": ["Could not find reference \u0027cell45\u0027"]
        },
           "execution_count": null,
           "metadata": {},
           "output_type": "execute_result"
          }],
   "source": [
    "combinedChart\n"
   ]
  }
,
  {
   "cell_type": "markdown",
   "metadata": {},
   
   "source": [
    "You might also want to save your results to a csv file.\n",
    "\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "let [\u003cLiteral\u003e] OutputSchema =\n",
    "    \"Name(string),Index(int),Month(date),Ret(float)\"\n"
   ]
  }
,
  {
   "cell_type": "code",
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "polyglot_notebook": {
     "kernelName": "fsharp"
    }
   },
   "execution_count": null, "outputs": [],
   "source": [
    "type PortfolioReturnCsv = CsvProvider\u003cOutputSchema\u003e\n",
    "\n",
    "let makePortfolioReturnCsvRow (row:PortfolioReturn) =\n",
    "    PortfolioReturnCsv\n",
    "        .Row(name=row.Name,\n",
    "             index = row.Index,\n",
    "             month=row.Month,\n",
    "             ret = row.Return)\n",
    "\n",
    "let csvRows =\n",
    "    portfolioExcessReturns\n",
    "    |\u003e List.map makePortfolioReturnCsvRow\n",
    "\n",
    "let csv = new PortfolioReturnCsv(csvRows)\n",
    "csv.Save(\"myExcessReturnPortfolios.csv\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (F#)",
   "language": "F#",
   "name": ".net-fsharp"
  },
  "language_info": {
   "file_extension": ".fs",
   "mimetype": "text/x-fsharp",
   "name": "polyglot-notebook",
   "pygments_lexer": "fsharp"
  },
  "polyglot_notebook": {
   "kernelInfo": {
    "defaultKernelName": "fsharp",
    "items": [
     {
      "aliases": [],
      "languageName": "fsharp",
      "name": "fsharp"
     }
    ]
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}