{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# WhiteLibrary Support"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RobotKernel can include dedicated support for selected libraries. One of these libraries is [WhiteLibrary](https://github.com/Omenia/robotframework-whitelibrary) – the current library of choice for testing and automating Windows applications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Settings ***\n",
    "\n",
    "Library  WhiteLibrary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## State management"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RobotKernel remembers the current WhiteLibrary context between executions. That's why it is possible to launch and attach application once and then move on to work on keywors testing or automating that application. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Keywords ***\n",
    "\n",
    "Attach Notepad window\n",
    "    Launch application  notepad.exe\n",
    "    Attach application by name  notepad\n",
    "    Attach window  title:Nimetön – Muistio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Element picker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RobotKernel has also an interactive built-in element picker to help selecting elements for WhiteLibrary keywords.\n",
    "\n",
    "This element picker is activated by typing the special pseudo selector prefix ``ae:`` (*automation element*) and pressing `<TAB>` (similarly to triggering autocompletion suggestions in Jupyter Notebook and JupyterLab).\n",
    "\n",
    "Once picker is activated, it changes mouse pointer into crosshair. After the desired element is clicked with that crosshair pointer, RobotKernel will suggest supported selector completions (like *text:*, *id:* or *class_name:*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Keywords ***\n",
    "\n",
    "Input into Notepad\n",
    "    [Arguments]  ${text}=Here be dragons\n",
    "    Input text to textbox  class_name=Edit  ${text}\\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because not all elements, like menu items for example, can be selected with a simple click, there is one more thing: **long press selection**.\n",
    "\n",
    "Once mouse button is kept pressed for a few seconds, single click is  emulated to the underlying element. This should be enough to let the menu be opened. Then it is possible to keep the mouse button pressed, move its pointer above the desired menu element, wait a second to see the menu item focused, and finally release the mouse button to get supported selectors for the element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Keywords ***\n",
    "\n",
    "Clear notepad\n",
    "    Click menu button  text=Muokkaa\n",
    "    Click menu button  text=Valitse kaikki\n",
    "    Click menu button  text=Muokkaa\n",
    "    Click menu button  text=Poista"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Keywords ***\n",
    "\n",
    "Save file\n",
    "    [Arguments]  ${filename}=hello-world.txt\n",
    "    Click menu button  text=Tiedosto\n",
    "    Click menu button  text=Tallenna nimellä...\n",
    "    Input text to textbox  text=Tiedostonimi:  ${filename}\n",
    "    Select listbox value  text=Koodaus:  Unicode\n",
    "    Click button  text=Tallenna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenCV template grabber and matcher"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, RobotKernel ships with an experimental image recognition based Windows automation support based on [OpenCV computer vision library](https://opencv.org/), if that's available on the current Python environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Settings ***\n",
    "\n",
    "Library  WhiteLibrary\n",
    "Library  robotkernel.WhiteLibraryCompanion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The special keyword library **robotkernel.WhiteLibraryCompanion** provides a single keyword: **Click template** with arguments **template** and **similarity** (with default value 0.95 to require exact match).\n",
    "\n",
    "The magic is hidden within the keyword argument **template** as follows:\n",
    "\n",
    "1. type ``Click template  template=`` and\n",
    "\n",
    "2. press ``<TAB>``.\n",
    "\n",
    "Once again the mouse cursor is changed into crosshair, but this time it expects a rectangular area to be painted with dragging the default mouse button pressed. Once the mouse button is released:\n",
    "\n",
    "1. a screenshot is captured from the painted area\n",
    "\n",
    "2. it is saved in to the current working directory of Jupyter process and\n",
    "\n",
    "3. ``template=`` part of the keyword is replaced with ``template=${EXECDIR}\\\\[TIMESTAMP].png``\n",
    "\n",
    "and now the keyword is ready to be called. Call of the keyword captures screenshot of the current default screen, looks for captured area using OpenCV matchTemplate and click the center of the found match using WhiteLibrary mouse automation functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "*** Keywords ***\n",
    "\n",
    "Open calc\n",
    "    Launch application  calc.exe\n",
    "    Attach window  Laskin\n",
    "    \n",
    "1\n",
    "    Click template  template=${EXECDIR}\\\\1554037865.png\n",
    "    \n",
    "+\n",
    "    Click template  template=${EXECDIR}\\\\1554037877.png\n",
    "    \n",
    "2\n",
    "    Click template  template=${EXECDIR}\\\\1554037888.png"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Robot Framework",
   "language": "robotframework",
   "name": "robotkernel"
  },
  "language_info": {
   "codemirror_mode": "robotframework",
   "file_extension": ".robot",
   "mimetype": "text/plain",
   "name": "robotframework",
   "pygments_lexer": "robotframework"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}