{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the main things that we want to do in scientific computing is get data into and out of our programs.  In addition to plain text files, there are modules that can read lots of different data formats we might encounter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've already been using print quite a bit, but now we'll look at how to control how information is printed.  Note that there is an older and newer way to format print statements -- we'll focus only on the newer way (it's nicer).\n",
    "\n",
    "This is compatible with both python 2 and 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "y = 0.0000354\n",
    "z = 3.0\n",
    "s = \"my string\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We write a string with `{}` embedded to indicate where variables are to be inserted.  Note that `{}` can take arguments.  We use the `format()` method on the string to match the variables to the `{}`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = 1, y = 3.54e-05, z = 3.0, s = my string\n"
     ]
    }
   ],
   "source": [
    "print(\"x = {}, y = {}, z = {}, s = {}\".format(x, y, z, s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before a semi-colon, we can give an optional index/position/descriptor of the value we want to print.\n",
    "\n",
    "After the semi-colon we give a format specifier.  It has a number field and a type, like `f` and `g` to describe how floating point numbers appear and how much precision to show.   Other bits are possible as well (like justification). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = 1, y =   3.54e-05, z = 3.000, s = my string\n"
     ]
    }
   ],
   "source": [
    "print(\"x = {0}, y = {1:10.5g}, z = {2:.3f}, s = {3}\".format(x, y, z, s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "there are other formatting things, like justification, etc.  See the tutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                centered string                                 \n"
     ]
    }
   ],
   "source": [
    "print(\"{:^80}\".format(\"centered string\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# File I/O"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "as expected, a file is an object.  Here we'll use the `try`, `except` block to capture exceptions (like if the file cannot be opened). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<_io.TextIOWrapper name='./sample.txt' mode='w' encoding='UTF-8'>\n"
     ]
    }
   ],
   "source": [
    "try: f = open(\"./sample.txt\", \"w\")   # open for writing -- any file of the same name will be overwritten\n",
    "except: \n",
    "    print(\"cannot open the file\")\n",
    "\n",
    "print(f)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "f.write(\"this is my first write\\n\")\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can easily loop over the lines in a file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Lorem', 'ipsum', 'dolor', 'sit', 'amet,', 'consectetur', 'adipisicing', 'elit,', 'sed', 'do']\n",
      "['eiusmod', 'tempor', 'incididunt', 'ut', 'labore', 'et', 'dolore', 'magna', 'aliqua.', 'Ut', 'enim', 'ad']\n",
      "['minim', 'veniam,', 'quis', 'nostrud', 'exercitation', 'ullamco', 'laboris', 'nisi', 'ut']\n",
      "['aliquip', 'ex', 'ea', 'commodo', 'consequat.', 'Duis', 'aute', 'irure', 'dolor', 'in']\n",
      "['reprehenderit', 'in', 'voluptate', 'velit', 'esse', 'cillum', 'dolore', 'eu', 'fugiat', 'nulla']\n",
      "['pariatur.', 'Excepteur', 'sint', 'occaecat', 'cupidatat', 'non', 'proident,', 'sunt', 'in']\n",
      "['culpa', 'qui', 'officia', 'deserunt', 'mollit', 'anim', 'id', 'est', 'laborum.']\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "try: \n",
    "    f = open(\"./test.txt\", \"r\")\n",
    "except:\n",
    "    print(\"error: cannot open the file\")\n",
    "    \n",
    "for line in f:\n",
    "    print(line.split())\n",
    "    \n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "split is a very neat function that acts on a string, what it does is split or breakup a string and add the data to a string array using a defined separator. In this case we used spaces and we have created arrays of words for each line in the file. If no separator is defined when you call upon the function, whitespace will be used by default.\n",
    "\n",
    "As mentioned earlier, there are lots of string functions.  Above we used `strip()` to remove the trailing whitespace and returns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSV Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "comma-separated values are an easy way to exchange data -- you can generate these from a spreadsheet program.  In the example below, we are assuming that the first line of the spreadsheet/csv file gives the headings that identify the columns.  \n",
    "\n",
    "Note that there is an amazing amount of variation in terms of what can be in a CSV file and what the format is -- the csv module does a good job sorting this all out for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "item: apples, quantity: 2, unit price: 0.33, total: 0.66\n",
      "item: bananas, quantity: 5, unit price: 0.1, total: 0.5\n",
      "item: milk, quantity: 1, unit price: 2.5, total: 2.5\n",
      "item: soda, quantity: 3, unit price: 1, total: 3\n",
      "item: rolls, quantity: 12, unit price: 0.33, total: 3.96\n",
      "item: eggs, quantity: 1, unit price: 2.5, total: 2.5\n"
     ]
    }
   ],
   "source": [
    "class Item(object):\n",
    "    def __init__(self):\n",
    "        self.name = \"\"\n",
    "        self.quantity = 0\n",
    "        self.unitprice = 0.0\n",
    "        self.total = 0.0\n",
    "    \n",
    "import csv\n",
    "\n",
    "reader = csv.reader(open(\"shopping.csv\", \"r\"))\n",
    "\n",
    "headings = None\n",
    "\n",
    "shopping_list = []\n",
    "\n",
    "for row in reader:\n",
    "    if headings == None:\n",
    "        # first row\n",
    "        headings = row\n",
    "    else:\n",
    "        my_item = Item()\n",
    "        my_item.name = row[headings.index(\"item\")]\n",
    "        my_item.quantity = row[headings.index(\"quantity\")]\n",
    "        my_item.unitprice = row[headings.index(\"unit price\")]\n",
    "        my_item.total = row[headings.index(\"total\")]\n",
    "        shopping_list.append(my_item)\n",
    "    \n",
    "\n",
    "for i in shopping_list:\n",
    "    print (\"item: {}, quantity: {}, unit price: {}, total: {}\".format(i.name, i.quantity, i.unitprice, i.total))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# INI Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "INI or Config files are a common way of specifying options to a program.  They can take the form (from the `ConfigParser` page):\n",
    "\n",
    "```\n",
    "[My Section]\n",
    "foodir: %(dir)s/whatever\n",
    "dir=frob\n",
    "long: this value continues\n",
    "   in the next line\n",
    "```\n",
    "\n",
    "Here we look at how to read in options and store them in a dictionary of the \n",
    "form `dict[\"sec.option\"] = value`\n",
    "\n",
    "We'll use a sample .ini file from a regression test suite (`VARDEN-tests.ini`)\n",
    "\n",
    "(Note: the name of the module is `ConfigParser` in python 2 but `configparser` in python 3 -- the latter confirms to the python style guidelines.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bubble-2d.compileTest           : 0\n",
      "bubble-3d.numprocs              : 3\n",
      "bubble-3d.useOMP                : 1\n",
      "bubble-2d.useMPI                : 1\n",
      "bubble-2d.buildDir              : varden/test\n",
      "bubble-restart.dim              : 3\n",
      "bubble-restart.doVis            : 0\n",
      "main.reportActiveTestsOnly      : 1\n",
      "bubble-2d.useOMP                : 0\n",
      "bubble-2d.inputFile             : inputs_2d-regt\n",
      "main.MPIcommand                 : /usr/local/bin/mpiexec -n @nprocs@ @command@\n",
      "bubble-3d.compileTest           : 0\n",
      "main.testTopDir                 : /home/regtester/RegTesting/rt-VARDEN/\n",
      "bubble-2d.restartTest           : 0\n",
      "bubble-3d.inputFile             : inputs_3d-regt\n",
      "bubble-restart.restartTest      : 1\n",
      "bubble-3d.restartTest           : 0\n",
      "bubble-3d.buildDir              : varden/test\n",
      "bubble-restart.buildDir         : varden/test\n",
      "bubble-3d.doVis                 : 0\n",
      "main.numMakeJobs                : 8\n",
      "bubble-restart.useOMP           : 1\n",
      "bubble-restart.restartFileNum   : 4\n",
      "main.webTopDir                  : /home/regtester/RegTesting/rt-VARDEN/web\n",
      "main.sourceDir                  : /home/regtester/RegTesting/VARDEN/\n",
      "bubble-restart.numthreads       : 2\n",
      "main.suiteName                  : VARDEN\n",
      "bubble-3d.useMPI                : 1\n",
      "bubble-2d.dim                   : 2\n",
      "main.FCOMP                      : gfortran\n",
      "main.sourceTree                 : F_Src\n",
      "main.boxLibDir                  : /home/regtester/RegTesting/BoxLib/\n",
      "main.compareToolDir             : /home/regtester/RegTesting/AmrPostprocessing/F_Src\n",
      "bubble-restart.compileTest      : 0\n",
      "bubble-2d.numprocs              : 2\n",
      "bubble-2d.doVis                 : 0\n",
      "main.goUpLink                   : 1\n",
      "main.MPIhost                    : \n",
      "main.COMP                       : g++\n",
      "bubble-restart.useMPI           : 1\n",
      "bubble-restart.inputFile        : inputs-restart-regt\n",
      "bubble-restart.numprocs         : 3\n",
      "main.MAKE                       : make\n",
      "bubble-2d.numthreads            : 2\n",
      "bubble-3d.dim                   : 3\n",
      "bubble-3d.numthreads            : 2\n"
     ]
    }
   ],
   "source": [
    "import configparser\n",
    "\n",
    "options = {}\n",
    "\n",
    "\n",
    "cp = configparser.ConfigParser()\n",
    "cp.optionxform = str    # this makes options case-sensitive\n",
    "cp.read(\"VARDEN-tests.ini\")\n",
    "\n",
    "for sec in cp.sections():\n",
    "    for opt in cp.options(sec):\n",
    "        key = str(sec) + \".\" + str(opt)\n",
    "        value = cp.get(sec,opt)\n",
    "        options[key] = value\n",
    "    \n",
    "for k, v in options.items():\n",
    "    print(\"{:32s}: {}\".format(k, v))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}