{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "### CS/ECE/ISyE 524 — Introduction to Optimization — Spring 2018 ###\n", "\n", "# CS 506 Group Assignment Optimization\n", "\n", "#### Zijie (Jay) Wang (zwang688@wisc.edu)\n", "\n", "\n", "*****\n", "\n", "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 1. Introduction\n", "\n", "In this project, we will use optimization techniques to find the best group assignment for CS 506, which maximizes the student happiness as well as assignment fairness, based on student's survey data.\n", "\n", "CS 506 (Software Engineering) is one of the most popular undergraduate computer science courses in UW-Madison. In this course, students are expected to complete a large-scale programing project in groups through the whole semester. Thus, wisely assigning groups is critical for instructors. Our goal in this project is to design a robust and flexible optimization model to improve the group assignment process. It will make teaching more effective and the learning experience more enjoyable. We have interviewed Professor Ben Liblit, the instructor for CS 506 in Spring 2018, regarding to the current assignment procedure. This model is tailored to fit his requirements and the data he is using.\n", "\n", "**CS 506 Group Assignment Process**\n", "1. All students are required to submit one project proposal in the first two weeks.\n", "2. The instructor and TA's choose about one third of the proposals for bidding.\n", "3. In the bidding phase, students read all selected proposals and complete a survey regarding to their preferences, and previous programming experiences.\n", "4. Instructor assigns students to groups based on the survey results.\n", "\n", "In the survey, Liblit asks students to rank $8$ most interested projects, $3$ most desired teammates, and $2$ students who they do not want to work with. He also collects data about student's previous project complexity and life time programming experience, measured by lines of code. For the confidentiality reason, Liblit did not share those data with us. We [generated fake data](#6.1-Demo-data-generation) having a similar format in this project. Within each step, Liblit has specific requirements (i.e. group size limit). These requirement will be discussed in detail in the [constraints](#2.3.-Constraints) section.\n", "\n", "

\n", " \n", "

One question from the Canvas survey, provided by Professor Liblit
\n", "

\n", "\n", "\n", "Liblit would like to assign students to their interested projects or groups having their friends. On the other hand, he expects each groups to have a similar programming experience level. In this project, we consider two objectives, student happiness and group experience equality. We propose there is a trade-off relationship between two objectives. We will design our model in a way that instructors can easily change the weights of objectives and each single factors.\n", "\n", "Currently, Liblit is using an optimization model, written in GAMS by some graduate students years ago, with fixed parameters. He didn't tell us any details about that method. Our goal is to make a more flexible model which can solve the same problem. Finally, we will analyze the model and its solution to gain more insights into the nature of CS 506 group assignment.\n", "\n", "**Source:** *Professor Ben Liblit, personal communication, April 18, 2018.*" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 2. Mathematical Model ##\n", "\n", "In this section, we will discuss how to transfer the constraints and objectives into mathematical form. For the sake of convenience, we define $\\mathcal{I}$ as the set containing labels of all student enrolled in CS506, and $\\mathcal{J}$ as the set of all projects which are selected for bidding. " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 2.1. Modeling Assumptions\n", "\n", "- In this problem, we are assigning students into groups, so we expect integer solutions.\n", "- We assume the given data are cleaned. Our model can handle `NA` data, but we assume there are no errors in the data (like one proposal is submitted by two students)." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 2.2. Decision Variables\n", "\n", "The decision variable is a matrix $A \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{J} \\Vert}\\left(\\{0,1\\}\\right)$.\n", "\n", "$$\n", "A_{i,j}=\\begin{cases}\n", " 1 &\\text{If student $i$ is assigned to project $j$}\\\\\n", " 0 &\\text{Otherwise}\n", " \\end{cases}\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 2.3. Constraints\n", "\n", "#### 2.3.1. Group Size Constraint\n", "\n", "Each group can only have $l$ to $u$ ($l, u \\in \\mathbb{N}, u > l$) members, where $i, j$ are decided by the instructor. However, some projects will be discarded after this assignment (lose the bidding). For example, there are $90$ students enrolled in CS506 in 2018 Spring, and TA's have chosen $29$ candidate project proposals. Each project requires $5 \\sim 7$ students. It implies that there are at most $18$ final projects, so at least $11$ project will be discarded. Therefore, each project should have variable size, either $0$ or $l$ to $u$.\n", " \n", "In other words, $$\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} = 0 \\quad \\text{or} \\quad l \\leq \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq u$$\n", "\n", "To model the variable lower bound, we introduce a new binary vector decision variable $z_1 \\in M_{\\Vert \\mathcal{J} \\Vert \\times 1}\\left(\\{0,1\\}\\right)$.\n", " \n", "Then, this constraint becomes,\n", "$$\\forall j \\in \\mathcal{J}, \\quad l z_{1_j} \\leq \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq u z_{1_j}$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "\n", "#### 2.3.2. Student Choice Constraint\n", "\n", "Each student will be assigned to exactly one group,\n", " \n", "$$\\forall i \\in \\mathcal{I}, \\quad \\sum_{j=1}^{\\Vert \\mathcal{J} \\Vert}A_{i,j} = 1$$\n", " " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### 2.3.3. Proposal Writer Constraint\n", "\n", "The instructor of CS506 made a rule that the person who submits the proposal must be working on his/her submitted project, if it won the bidding. \n", " \n", "We define a discrete injective function $F_w: \\mathcal{J} \\mapsto \\mathcal{I}$ to map each project to its proposer. Then, we can model this constraint as a logical constraint,\n", " \n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\geq 1 \\longrightarrow A_{F_w\\left(j\\right), j} = 1\n", "$$\n", " \n", "We need to introduce another binary vector variable $z_2 \\in M_{\\Vert \\mathcal{J} \\Vert \\times 1}\\left(\\{0,1\\}\\right)$ as an intermediate state variable.\n", " \n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\geq 1 \\longleftrightarrow z_{2_{j}} = 1\\\\\n", "$$\n", " \n", "One upper bound of $\\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j}$ is $u$, and one lower bound of $\\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} - 1$ is $-1$. Then, using the Big-M method we can model the above logical relationship as,\n", " \n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad \n", "\\begin{cases}\n", " \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} &\\leq z_{2_{j}} u \\\\\n", " \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} -1 &\\geq -1 \\left(1 - z_{2_{j}}\\right) \\quad \\text{or} \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} &\\geq z_{2_{j}}\n", "\\end{cases}\n", "$$\n", " \n", "Then, the original constraint becomes,\n", " \n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad z_{2_{j}} = 1 \\longrightarrow A_{F_w\\left(j\\right), j} = 1\n", "$$\n", " \n", "It can be modeled as,\n", " \n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad z_{2_{j}} \\leq A_{F_w\\left(j\\right), j}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.4. Objective Function\n", "\n", "At the beginning of the bidding phase, all students in CS506 are asked to complete a background survey. We can use the information to define the following five discrete surjective functions.\n", "\n", "1. $F_1: \\mathcal{I} \\mapsto \\{1, 2, 3, 4, 5\\}$, mapping each student to the scale of the most complicated project he/she has done.\n", "2. $F_2: \\mathcal{I} \\mapsto \\{1, 2, 3, 4, 5\\}$, mapping each student to the level of his/her lifetime programming experience.\n", "3. $F_3: \\mathcal{I} \\mapsto \\mathcal{J}^8$, mapping each student to an ordered 8-tuple of project numbers that he/she is interested.\n", "4. $F_4: \\mathcal{I} \\mapsto \\mathcal{I}^3$, mapping each student to an ordered 3-tuple of students who he/she wants to work with.\n", "5. $F_5: \\mathcal{I} \\mapsto \\mathcal{I}^2$, mapping each student to an ordered 2-tuple of students who he/she does not want to work with." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also introduce a weight for each of the function above to indicate the importance level for each measure. In this model, we assume all weights are fixed and given by the instructor.\n", "\n", "1. $\\beta_1 \\in \\mathbb{R}:$ scalar weight of $F_1$.\n", "2. $\\beta_2 \\in \\mathbb{R}:$ scalar weight of $F_2$.\n", "3. $\\beta_3 \\in \\mathbb{R}^8:$ an ordered 8-tuple weight of $F_3$.\n", "4. $\\beta_4 \\in \\mathbb{R}^3:$ an ordered 3-tuple weight of $F_4$.\n", "5. $\\beta_5 \\in \\mathbb{R}^2:$ an ordered 2-tuple weight of $F_5$.\n", "\n", "All weights $\\beta_1, \\dots, \\beta_5$ can be positive or negative, but usually we only want $\\beta_5$ to be negative (penalty)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Taking advantage of the fact that all decision variables are binary and $\\mathcal{I}$ being determinative and finite, we can vectorize these mappings to make the transformations of $A$ become easy linear algebra. To make the notations simple, we use the same notation $\\left(F_1,\\dots, F_5\\right)$ for the vectorized functions. The specific vectorization is discussed in the section below.\n", "\n", "For our model, we consider our objective $\\left(O\\right)$ as a linear combination of group experience equality score $\\left(D\\right)$ and student happiness score $\\left(H\\right)$. We assume only $F_1$ and $F_2$ contribute to the programing experience equality of each group, and only $F_3, F_4, F_5$ contribute to the student happiness of their assignments." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2.4.1. Experience Equality\n", "\n", "Experience Equality means we want each group to have a similar average programing experience. We can first vectorize functions $F_1, F_2$ into two column vectors with length $\\Vert \\mathcal{I} \\Vert$, where $F_1^{\\left( i \\right)}, F_2^{\\left( i \\right)}$ corresponds to the two experience measurements of student $i$. Then, we can write all $\\Vert \\mathcal{J} \\Vert$ groups' average experience level $\\left(E\\right)$ with regarding to the weights $\\beta_1, \\beta_2$ as below,\n", "\n", "$$\n", "E = \\text{diag}\\left( A^T \\times \\vec{1} \\right)^{-1} \\left(A^T\\left(\\beta_1 F_1\\right) + A^T\\left(\\beta_2 F_2\\right)\\right)\n", "$$\n", "\n", "It's worth mentioning that $\\vec{1}$ is a length $\\Vert \\mathcal{I} \\Vert$ column vector, where all values are $1$, so $A^T \\times \\vec{1}$ becomes a length $\\Vert \\mathcal{J} \\Vert$ column vector of all group sizes. Then, $\\text{diag}\\left( A^T \\times \\vec{1} \\right)^{-1}$ is a diagonal matrix whose diagonal is exactly the inverse of the group sizes. Finally, $E$ is a length $\\Vert \\mathcal{J} \\Vert$ vector of the experience average of all groups.\n", " \n", "The programming experience equality $\\left(D\\right)$ then can be modeled as the variance of those averages. Denote $\\overline{E}$ as the population mean of the group experience averages,\n", "\n", "$$\n", "\\overline{E} = \\frac{\\sum_{k=1}^{\\Vert \\mathcal{J} \\Vert} E_k}{\\Vert \\mathcal{J} \\Vert}\n", "$$\n", "\n", "Then, we can write the variance as (note here we are using the population variance formula),\n", " \n", "$$\n", "\\begin{align*}\n", "D &= \\text{Var}\\left(E \\right)\\\\\n", " &= \\frac{1}{\\Vert \\mathcal{J} \\Vert}\\sum_{k=1}^{\\Vert \\mathcal{J} \\Vert}\\left(E_k - \\overline{E}\\right)^2\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "#### 2.4.2. Experience Equality (Linear Relaxed)\n", " \n", "In the set up above, it requires a very complicated (large) quadratic expression and solvers will take a very long time solely to set up the expression $D$. In this section, we propose a linear relaxation to encode the group experience equality objective into a linear constraint.\n", "\n", "Instead of minimizing the variance of experience means, we can constrain the experience average of each group into a small range. The idea is similar to the robust LP we have discussed in the lecture.\n", "\n", "We first compute the overall experience mean regarding to the weights $\\beta_1, \\beta_2$,\n", "\n", "$$\n", "\\overline{E_1} = \\frac{\\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert} \\left(\\beta_1 F_{1_k} + \\beta_2 F_{2_k} \\right)}{\\Vert \\mathcal{I} \\Vert}\n", "$$\n", "\n", "Also compute the standard deviation $\\left(\\sigma\\right)$ of $\\left(\\beta_1 F_{1_i} + \\beta_2 F_{2_i}\\right)$,\n", "\n", "$$\n", "\\sigma = \\sqrt{\\frac{\\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert} \\left(\\beta_1 F_{1_k} + \\beta_2 F_{2_k} - \\overline{E_1}\\right)^2}{\\Vert \\mathcal{I} \\Vert}}\n", "$$\n", "\n", "Note $\\overline{E_1}$ and $\\sigma$ are computed before solving the optimization problem, so they are our model parameters. Then, we add the following constraint.\n", "\n", "$$\n", "\\forall j \\in \\mathcal{J}, \\quad \\left(\\overline{E_1} - \\delta_1 \\sigma \\right) \\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j} \\leq \\left(A^T\\left(\\beta_1 F_1\\right) + A^T\\left(\\beta_2 F_2\\right)\\right)_j \\leq \\quad \\left(\\overline{E_1} + \\delta_1 \\sigma \\right) \\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j}\n", "$$\n", "\n", "In the above inequality, $\\left(A^T\\left(\\beta_1 F_1\\right) + A^T\\left(\\beta_2 F_2\\right)\\right)_j $ is the weighted total experience level of group $j$. $\\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j}$ is the group size of group $j$. $\\left(\\overline{E_1} - \\delta_1 \\sigma \\right)$ is a lower bound regarding to the experience mean and standard deviation, where $\\delta_1$ determines the tolerant level. If $\\delta_1$ is too large, the constraint becomes meaningless, and if it is too small, the model can become infeasible. It requires the instructor to tune this parameter. Finally, $\\left(\\overline{E_1} + \\delta_1 \\sigma \\right)$ is the corresponding upper bound.\n", "\n", "The reason why we times the size $\\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j}$ on both sides is to avoid division which requires to use `@NLexpression`, and `JuMP.jl` doesn't support vector devisions. We also don't want to divide the total experience level by $0$. In the implementation, we will further factorize this constraint, and for the sake of notation clarity, we didn't write that form here.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "#### 2.4.3. Student Happiness\n", "\n", "Student happiness is defined as the number of matches of $F_3\\left(\\mathcal{I}\\right), F_4\\left(\\mathcal{I}\\right), F_5\\left(\\mathcal{I}\\right)$ to student's real assignments, weighted by $\\beta_3, \\beta_4, \\beta_5$. \n", "\n", "For example, student $\\mathcal{I}_p$ has an ordered list of projects $F_3\\left(\\mathcal{I}_p\\right)=\\begin{bmatrix}a_1\\\\a_2\\\\a_3\\\\q\\\\a_5\\\\a_6\\\\a_7\\\\a_8\\end{bmatrix}$ he wants to work on, and sorted lists $F_4\\left(\\mathcal{I}_p\\right) = \\begin{bmatrix}b_1\\\\b_2\\\\b_3\\end{bmatrix}$ and $F_5\\left(\\mathcal{I}_p\\right) = \\begin{bmatrix}p_4\\\\c_2\\end{bmatrix}$ of students he would like/ not like to be grouped with. He is assigned to group $q$ with other students $p_1, \\dots, p_5$. Suppose he did rank $q$ as his fourth favorite project, $F_3^{\\left(4\\right)}\\left(\\mathcal{I}_p\\right) = q$, then there is a happiness reward $\\beta_3^{\\left(4\\right)}$ for him. All the students he wants to work with are not assigned to his group, and the student $p_3$, who he doesn't like, is in his group. Then, there is no happiness bonus from $F_4$ but a penalty $\\beta_5^{\\left(1\\right)}$ from $F_5$.\n", "\n", "To make the algebraic expression simple, we vectorize $F_3$ and $\\beta_3$ (note they are all given) into a matrix $F_3 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{J} \\Vert}\\left(\\beta_3\\right)$.\n", "\n", "$$\n", "F_{3_{i,j}}=\\begin{cases}\n", " \\beta_3^{\\left(p\\right)} &\\text{If student $i$ has listed project $j$ in his/her preference list with rank $p$}\\\\\n", " 0 &\\text{If project $j$ is not on the preference list of student $i$}\n", " \\end{cases}\\\\\n", "$$\n", "\n", "The happiness reward from $F_3$ simply becomes (note $\\circ$ is Hadamard multiplication, a.k.a the element-wise matrix multiplication),\n", "\n", "$$\n", "H_{F_3} = \\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j}\n", "$$\n", "\n", "We then vectorize $F_4$ and $\\beta_4$ into a matrix $F_4 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{I} \\Vert}\\left(\\beta_4\\right)$.\n", "\n", "$$\n", "F_{4_{k,l}}=\\begin{cases}\n", " \\beta_4^{\\left(p\\right)} &\\text{If student $k$ has listed student $l$ in his/her preference list with rank $p$}\\\\\n", " 0 &\\text{If student $l$ is not on the preference list of student $k$}\n", " \\end{cases}\\\\\n", "$$\n", "\n", "By [lemma 1](#lemma1), the happiness score from $F_4$ can be written as,\n", "\n", "$$\n", "H_{F_4} = \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k}\n", "$$\n", "\n", "Using the same way of $F_4$ to vectorize $F_5 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{I} \\Vert}\\left(\\beta_5\\right)$, the happiness penalty from $F_5$ becomes,\n", "\n", "$$\n", "H_{F_5} = \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\n", "$$\n", "\n", "Finally, the total happiness score is (note weights $\\beta_3, \\beta_4, \\beta_5$ have been encoded in the matrices $F_3, F_4, F_5$),\n", "\n", "$$\n", "H = H_{F_3} + H_{F_4} + H_{F_5} = \\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2.4.4. Combination\n", "\n", "Intuitively we think there is a trade-off relationship between group experience equality and student happiness. We can give them two trade-off parameters $\\delta_1, \\delta_2 \\in \\mathbb{R}$. We actually only need one trade-off parameter, but having two gives more flexibility especially when the units of weights $\\beta_1,\\dots, \\beta_5$ can be quite different.\n", "\n", "We are trying to minimize the variance of experience levels, and maximize the student happiness. Therefore, we negate the first part of the objective and maximize the combination.\n", "\n", "Then the objective function $\\left(O\\right)$ is,\n", "\n", "$$\n", "\\begin{align*}\n", "O &= -\\delta_1 D + \\delta_2 H \\\\\n", " &= -\\delta_1 \\frac{1}{\\Vert \\mathcal{J} \\Vert}\\sum_{k=1}^{\\Vert \\mathcal{J} \\Vert}\\left(E_k - \\overline{E}\\right)^2 + \\delta_2 \\left(\\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\\right)\n", "\\end{align*}\n", "$$\n", "\n", "If we use the linear relaxed version, the objective of equal experience level $\\left(D\\right)$ is transformed into a linear constraint with $\\delta_1$ embedded. Then, the only objective function $\\left(O\\right)$ is,\n", "\n", "$$\n", "\\begin{align*}\n", "O &= \\delta_2 H = \\delta_2 \\left(\\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\\right)\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "**Lemma 1:** The matrix $AA^T$ maps students to their group members.\n", "\n", "**Proof:** The $k$th row of $AA^T$ is $\\big[\\langle A_{\\left(k,*\\right)}, A_{\\left(1,*\\right)}\\rangle, \\langle A_{\\left(k,*\\right)}, A_{\\left(2,*\\right)}\\rangle, \\dots, \\langle A_{\\left(k,*\\right)}, A_{\\left(i,*\\right)}\\rangle\\big]$.\n", "\n", "The dot product $\\langle A_{\\left(k,*\\right)}, A_{\\left(i,*\\right)}\\rangle = \\sum_{p=1}^i A_{k,p}A_{i,p}$.\n", "\n", "1. From the [Student Choice Constraint](#constraints), we know there is only one $1$ in each row of $A$. Also, $1\\times 0 = 0, 1 \\times 1 = 1$. Therefore, each dot product can only be $1$ or $0$. In other words, $AA^T$ is binary.\n", "\n", "2. We know $\\sum_{p=1}^i A_{k,p}A_{i,p} = 1$ if and only there is one $p$ such that $A_{k,p} = A_{i,p} = 1$. It implies the dot product becomes $1$ if and only there is a group $p$ such that both student $k$ and student $i$ are assigned to.\n", "\n", "Therefore, \n", " $$\n", "\\left(AA^T\\right)_{\\left(k,l\\right)}=\\begin{cases}\n", " 1 &\\text{If student $k$ and $l$ are in the same group}\\\\\n", " 0 &\\text{Otherwise}\n", " \\end{cases}\\\\\n", " $$\n", "\n", "**Example:**\n", "\n", " Suppose $A = \\begin{bmatrix}1&0\\\\0&1\\\\0&1\\\\1&0 \\end{bmatrix},\\quad AA^T = \\begin{bmatrix}1&0\\\\0&1\\\\0&1\\\\1&0 \\end{bmatrix}\\begin{bmatrix}1&0&0&1\\\\0&1&1&0 \\end{bmatrix} = \\begin{bmatrix}1&0&0&1\\\\0&1&1&0\\\\0&1&1&0\\\\ 1&0&0&1\\end{bmatrix}$\n", " \n", " From $A$, we know students $1$ and $4$ are assigned into group $1$. This relationship is also shown in the first or the fourth row of $AA^T$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.5. Standard Forms\n", "\n", "#### 2.5.1 Original Model\n", "\n", "The following model can be categorized to Mixed Integer Quadratic Programming (MIQP).\n", "\n", "$l, u, E, \\overline{E}, F_3, F_4, F_5, F_w, \\delta_1, \\delta_2$ are all model parameters, which will be computed before solving the problem. You can read the [Constraint](#2.3.-Constraints), [Objectives](#2.4.-Objective-Function), and [Solution](#3.-Solution) sections to see their definitions and implementations.\n", "\n", "$$\n", "\\begin{align*}\n", "\\underset{A, z_1, z_2}{\\text{maximize}} \\quad & -\\delta_1 \\frac{1}{\\Vert \\mathcal{J} \\Vert}\\sum_{k=1}^{\\Vert \\mathcal{J} \\Vert}\\left(E_k - \\overline{E}\\right)^2 + \\delta_2 \\left(\\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\\right) \\\\\n", "\\text{subject to:} \\quad &\\forall j \\in \\mathcal{J}, \\quad l z_{1_j}\\leq \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq u z_{1_j}\\\\\n", " &\\forall i \\in \\mathcal{I}, \\quad \\sum_{j=1}^{\\Vert \\mathcal{J} \\Vert}A_{i,j} = 1\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq z_{2_{j}} u\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\geq z_{2_{j}}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{2_{j}} \\leq A_{F_w\\left(j\\right), j}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{1_j} \\in \\{0, 1 \\}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{2_j} \\in \\{0, 1 \\}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\forall i \\in \\mathcal{I}, \\quad A_{i,j} \\in \\{0, 1\\}\\\\\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "#### 2.5.2 Experience Equality Linear Relaxed Model\n", "\n", "The relaxed version is still a Mixed Integer Quadratic Programming (MIQP). The purpose of relaxation is to avoid the memory issue of computing variance.\n", "\n", "$l, u, \\overline{E_1}, \\sigma, F_3, F_4, F_5, F_w, \\delta_1, \\delta_2$ are all model parameters, which will be computed before solving the problem. You can read the [Constraint](#2.3.-Constraints), [Objectives](#2.4.-Objective-Function), and [Solution](#3.-Solution) sections to see their definitions and implementations.\n", "\n", "Also note $\\delta_1, \\delta_2$ are still trade-off parameters, even though they look different from what we have seen in the lectures, more information [here](#relaxation).\n", "\n", "$$\n", "\\begin{align*}\n", "\\underset{A, z_1, z_2}{\\text{maximize}} \\quad & \\delta_2 \\left(\\sum_{i\\in\\mathcal{I}, j\\in\\mathcal{J}}\\left(A \\circ F_3\\right)_{i, j} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_4\\right)_{l,k} + \\sum_{l\\in\\mathcal{I}, k\\in\\mathcal{I}}\\left(AA^T \\circ F_5\\right)_{l,k}\\right) \\\\\n", "\\text{subject to:} \\quad &\\forall j \\in \\mathcal{J}, \\quad l z_{1_j}\\leq \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq u z_{1_j}\\\\\n", " &\\forall i \\in \\mathcal{I}, \\quad \\sum_{j=1}^{\\Vert \\mathcal{J} \\Vert}A_{i,j} = 1\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\leq z_{2_{j}} u\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\sum_{i=1}^{\\Vert \\mathcal{I} \\Vert}A_{i,j} \\geq z_{2_{j}}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{2_{j}} \\leq A_{F_w\\left(j\\right), j}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\left(\\overline{E_1} - \\delta_1 \\sigma \\right) \\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j} \\leq \\left(A^T\\left(\\beta_1 F_1\\right) + A^T\\left(\\beta_2 F_2\\right)\\right)_j \\leq \\quad \\left(\\overline{E_1} + \\delta_1 \\sigma \\right) \\sum_{k=1}^{\\Vert \\mathcal{I} \\Vert}A_{k,j}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{1_j} \\in \\{0, 1 \\}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad z_{2_j} \\in \\{0, 1 \\}\\\\\n", " &\\forall j \\in \\mathcal{J}, \\quad \\forall i \\in \\mathcal{I}, \\quad A_{i,j} \\in \\{0, 1\\}\\\\\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 3. Solution ##\n", "\n", "In this section, we will begin with building \"partial\" models (i.e. having only one objective), so we can know whether the mathematical assumptions discussed in the above section can be implemented in `JuMP.jl` and work as expected. Then we will add the final complete model.\n", "\n", "One important trick we are using in this project is vectorization. Below we will define three vectorization functions for the mapping $F_3, F_4, F_5$. To learn more about how the vectorization works on given data format, you can read the [appendix](#6.1-Demo-data-generation)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "vectorize_f5 (generic function with 1 method)" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Convert preference from STUDENT_NUM*8 integer matrix to a STUDENT_NUM*PROJECT_NUM\n", "# binary matrix weighted by β3\n", "# β3 is a length 8 vector\n", "function vectorize_f3(preference, β3)\n", " project_rank = convert(Array{Int64,2}, zeros(STUDENT_NUM,PROJECT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:8\n", " # If the student did't fill the list full\n", " if preference[r,c] == 0\n", " break\n", " end\n", " project_rank[r,preference[r,c]] = β3[c]\n", " end\n", " end\n", " return project_rank\n", "end\n", "\n", "# Convert preference from STUDENT_NUM*3 integer matrix to a STUDENT_NUM*STUDENT_NUM\n", "# binary matrix weighted by β4\n", "# β4 is a length 3 vector\n", "function vectorize_f4(likes, β4)\n", " student_likes = convert(Array{Int64,2}, zeros(STUDENT_NUM,STUDENT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:3\n", " # If the student did't fill the list full\n", " if likes[r,c] == 0\n", " break\n", " end\n", " student_likes[r, likes[r,c]] = β4[c]\n", " end\n", " end\n", " return student_likes\n", "end\n", "\n", "# Convert preference from STUDENT_NUM*2 integer matrix to a STUDENT_NUM*STUDENT_NUM\n", "# binary matrix weighted by β5\n", "# β5 is a length 2 vector\n", "function vectorize_f5(dislikes, β5)\n", " student_dislikes = convert(Array{Int64,2}, zeros(STUDENT_NUM,STUDENT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:2\n", " # If the student did't fill the list full\n", " if dislikes[r,c] == 0\n", " break\n", " end\n", " student_dislikes[r, dislikes[r,c]] = β5[c]\n", " end\n", " end\n", " return student_dislikes\n", "end" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.1. Model with only considering the happiness of students\n", "\n", "We use the same size limits as Professor Liblit uses in Spring 2018. Each final group should have from $5$ to $7$ members inclusively.\n", "\n", "$$l = 5 \\quad u = 7$$\n", "\n", "Then, we choose the weights of different happiness factors. There is no \"correct\" way to determine the weights, and the weights can be different in different terms.\n", "\n", "For choosing following weights, our reasons are:\n", "1. Working on the project the student likes gives more happiness than working with his/her friends.\n", "2. Working with people the student doesn't like can ruin the course experience. Therefore, $\\beta_5$ has a large penalty.\n", "\n", "$$\n", "\\beta_3 = \n", "\\begin{bmatrix}\n", "10 \\\\ 8 \\\\ 8 \\\\ 6 \\\\ 6 \\\\ 6 \\\\ 4 \\\\ 3\n", "\\end{bmatrix} \\quad\n", "\\beta_4 = \n", "\\begin{bmatrix}\n", "6 \\\\ 5 \\\\ 4\n", "\\end{bmatrix} \\quad\n", "\\beta_5 = \n", "\\begin{bmatrix}\n", "-500 \\\\ -500\n", "\\end{bmatrix}\n", "$$" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Dict{String,Any} with 6 entries:\n", " \"exp_1\" => [4, 2, 3, 1, 3, 4, 3, 3, 4, 2 … 4, 4, 2, 4, 4, 1, 4, 4…\n", " \"project_writers\" => [69, 87, 13, 36, 29, 12, 37, 62, 60, 74 … 20, 41, 55, …\n", " \"exp_2\" => [4, 2, 5, 2, 4, 3, 2, 2, 2, 4 … 4, 5, 4, 3, 3, 4, 4, 1…\n", " \"preference\" => [14 19 … 12 29; 28 25 … 26 12; … ; 20 6 … 0 0; 16 17 … 1…\n", " \"likes\" => [81 32 42; 67 18 0; … ; 0 0 0; 13 0 0]\n", " \"dislikes\" => [0 0; 0 0; … ; 0 0; 0 0]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using JuMP, Gurobi, JLD\n", "\n", "STUDENT_NUM = 90\n", "PROJECT_NUM = 29\n", "\n", "# Track the results of all the models below\n", "model_results = Dict()\n", "\n", "# Set up parameters\n", "beta_3 = [10, 8, 8, 6, 6, 6, 4, 3]\n", "beta_4 = [6, 5, 4]\n", "beta_5 = [-500, -500]\n", "\n", "# Load the fake data\n", "data = load(\"./data/data.jld\")" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "F3 dimension: (90, 29), F4 dimension: (90, 90), F5 dimension: (90, 90).\n" ] } ], "source": [ "# Vectorize the mappings F3, F4 and F5\n", "f3 = vectorize_f3(data[\"preference\"], beta_3)\n", "f4 = vectorize_f4(data[\"likes\"], beta_4)\n", "f5 = vectorize_f5(data[\"dislikes\"], beta_5)\n", "println(\"F3 dimension: $(size(f3)), F4 dimension: $(size(f4)), F5 dimension: $(size(f5)).\")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Academic license - for non-commercial use only\n", " 53.577204 seconds (1.50 M allocations: 93.602 MiB, 0.27% gc time)\n" ] }, { "data": { "text/plain": [ ":Optimal" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Start building the model\n", "m = Model(solver=GurobiSolver(OutputFlag=0))\n", "\n", "# ----------------------- Variables --------------------------\n", "@variable(m, A[1:STUDENT_NUM, 1:PROJECT_NUM], Bin)\n", "@variable(m, z1[1:PROJECT_NUM], Bin)\n", "@variable(m, z2[1:PROJECT_NUM], Bin)\n", "\n", "# ----------------------- Constraints --------------------------\n", "# Project size constraint (variable lower bound)\n", "@constraint(m, [j in 1:PROJECT_NUM], 5 * z1[j] <= sum(A[:,j]))\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z1[j])\n", "\n", "# Student choice constraint\n", "@constraint(m, [i in 1:STUDENT_NUM], sum(A[i,:]) == 1)\n", "\n", "# Proposal Writer Constraint (logical constriang) \n", "# Intermediate steps\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z2[j])\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) >= z2[j])\n", "# Original constraint\n", "@constraint(m, [j in 1:PROJECT_NUM], z2[j] <= A[data[\"project_writers\"][j], j])\n", "\n", "# ----------------------- Objectives --------------------------\n", "# F3 factor: project bonus\n", "@expression(m, Hf3, sum(A .* f3))\n", "\n", "# F4 factor: bonus for being in the group with friends\n", "@expression(m, Hf4, sum((A * A') .* f4))\n", "\n", "# F5 factor: penalty for being in the group with disliked people\n", "@expression(m, Hf5, sum((A * A') .* f5))\n", "\n", "# For this model, objective is just to maximize the happiness of students\n", "@objective(m, Max, Hf3 + Hf4 + Hf5)\n", "\n", "@time solve(m)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Dict{String,Any} with 9 entries:\n", " \"beta_1\" => nothing\n", " \"delta_1\" => nothing\n", " \"delta_2\" => nothing\n", " \"beta_5\" => [-500, -500]\n", " \"objective\" => 1136.0\n", " \"beta_4\" => [6, 5, 4]\n", " \"beta_3\" => [10, 8, 8, 6, 6, 6, 4, 3]\n", " \"best_A\" => [0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; …\n", " \"beta_2\" => nothing" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Record the result of this model\n", "result = Dict(\n", " \"beta_1\" => nothing,\n", " \"beta_2\" => nothing,\n", " \"beta_3\" => beta_3,\n", " \"beta_4\" => beta_4,\n", " \"beta_5\" => beta_5,\n", " \"delta_1\" => nothing,\n", " \"delta_2\" => nothing,\n", " \"best_A\" => getvalue(A),\n", " \"objective\" => getobjectivevalue(m)\n", ")\n", "model_results[\"happy\"] = result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.2. Model with only considering the equality of student experience\n", "\n", "We use the same lower and upper bounds as in [model 3.1](#3.1.-Model-with-only-considering-the-happiness-of-students).\n", "$$l = 5, \\quad u = 7$$\n", "\n", "For this model, we need to choose the parameters $\\beta_1, \\beta_2$ for two types of programming experience: single project complexity and lifetime programing experience. They are totally subjective to the Instructor. We just assume they have the same weights here.\n", "\n", "$$\\beta_1 = 1, \\quad \\beta_2 = 1$$" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dimension of f1 is (90,), of f2 is (90,).\n" ] } ], "source": [ "f1 = data[\"exp_1\"]\n", "f2 = data[\"exp_2\"]\n", "beta_1 = beta_2 = 1\n", "println(\"Dimension of f1 is $(Base.size(f1)), of f2 is $(Base.size(f2)).\")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# 🔥🔥🔥🔥🔥🔥🔥 Don't run this cell, it will burn your machine! 🔥🔥🔥🔥🔥🔥🔥 \n", "\n", "# Start building the model\n", "#m = Model(solver=GurobiSolver(OutputFlag=0))\n", "\n", "# ----------------------- Variables --------------------------\n", "#@variable(m, A[1:STUDENT_NUM, 1:PROJECT_NUM], Bin)\n", "#@variable(m, z1[1:PROJECT_NUM], Bin)\n", "#@variable(m, z2[1:PROJECT_NUM], Bin)\n", "\n", "# ----------------------- Constraints --------------------------\n", "# Project size constraint (variable lower bound)\n", "#@constraint(m, [j in 1:PROJECT_NUM], 5 * z1[j] <= sum(A[:,j]))\n", "#@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z1[j])\n", "\n", "# Student choice constraint\n", "#@constraint(m, [i in 1:STUDENT_NUM], sum(A[i,:]) == 1)\n", "\n", "# Proposal Writer Constraint (logical constriang) \n", "# Intermediate steps\n", "#@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z2[j])\n", "#@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) >= z2[j])\n", "# Original constraint\n", "#@constraint(m, [j in 1:PROJECT_NUM], z2[j] <= A[data[\"project_writers\"][j], j])\n", "\n", "# ----------------------- Objectives --------------------------\n", "# Get the combined experience of each group\n", "#@expression(m, experience, (beta_1 .* (A' * f1)) + (beta_2 .* (A' * f2)))\n", "\n", "# NLexpression doesnt support vectorized division, so we use the variane of total\n", "# experience level of groups, instead of variance of average experience level of\n", "# groups. The result will be the same.\n", "#@expression(m, pop_mean, sum(experience) / PROJECT_NUM)\n", "\n", "# Minimize the variance across all groups\n", "#@objective(m, Min, sum((experience .- pop_mean).^2))\n", "\n", "#solve(m)\n", "\n", "# 🔥🔥🔥🔥🔥🔥🔥 Don't run this cell, it will burn your machine! 🔥🔥🔥🔥🔥🔥🔥 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The code above is runnable, but it requires much more than 20GB memory to compute `(experience .- pop_mean).^2`. After running for 40 minutes on my laptop, it triggered `OutOfMemoryError()`. Therefore, the original model is not practical, and we might want to use the linear relaxed version." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.3. Model with only considering the equality of student experience (relaxed)\n", "\n", "In this model, we will use the linear relaxed constraint (described [here](#relaxation)) to make sure the experience levels for each group are fair.\n", "\n", "We use the same $u, l, \\beta_1, \\beta_2$ as above.\n", "\n", "$$u = 5, \\quad l = 7, \\quad \\beta_1 = \\beta_2 = 1$$\n", "\n", "We also need to compute some other parameters $\\left(\\overline{E_1}, \\sigma \\right)$ for this model. We set the tolerate level $\\delta_1$ as $0.5$, which tolerates fluctuation in one standard deviation in both direction for one student in the group." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The mean of experience level across all student is 6.3, with standard deviation 1.705575786661192.\n" ] } ], "source": [ "# Compute the parameteres\n", "weighted_exp = beta_1 * f1 + beta_2 * f2\n", "E1 = mean(weighted_exp)\n", "σ = sqrt(var(weighted_exp))\n", "\n", "# One sd level\n", "δ1 = 0.5\n", "\n", "println(\"The mean of experience level across all student is $(E1), with standard deviation $(σ).\")" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Academic license - for non-commercial use only\n", " 0.007609 seconds (153 allocations: 2.398 MiB)\n" ] }, { "data": { "text/plain": [ ":Optimal" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Start building the model\n", "m = Model(solver=GurobiSolver(OutputFlag=0))\n", "\n", "# ----------------------- Variables ----------------------------\n", "@variable(m, A[1:STUDENT_NUM, 1:PROJECT_NUM], Bin)\n", "@variable(m, z1[1:PROJECT_NUM], Bin)\n", "@variable(m, z2[1:PROJECT_NUM], Bin)\n", "\n", "# ----------------------- Constraints --------------------------\n", "# Project size constraint (variable lower bound)\n", "@constraint(m, [j in 1:PROJECT_NUM], 5 * z1[j] <= sum(A[:,j]))\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z1[j])\n", "\n", "# Student choice constraint\n", "@constraint(m, [i in 1:STUDENT_NUM], sum(A[i,:]) == 1)\n", "\n", "# Proposal Writer Constraint (logical constriang) \n", "# Intermediate steps\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z2[j])\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) >= z2[j])\n", "# Original constraint\n", "@constraint(m, [j in 1:PROJECT_NUM], z2[j] <= A[data[\"project_writers\"][j], j])\n", "\n", "# -------------------- Extra Constraint for Exp Equality ---------------------\n", "# Get the size of each project\n", "@expression(m, group_size[j in 1:PROJECT_NUM], sum(A[:,j]))\n", "\n", "# Combined exp level for each group\n", "@expression(m, experience, (beta_1 .* (A' * f1)) + (beta_2 .* (A' * f2)))\n", "\n", "# Relaxed constraint\n", "@constraint(m, group_size .* (E1 - δ1 * σ) .<= experience)\n", "@constraint(m, experience .<= group_size .* (E1 + δ1 * σ))\n", "\n", "@time solve(m)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Dict{String,Any} with 9 entries:\n", " \"beta_1\" => 1\n", " \"delta_1\" => 0.5\n", " \"delta_2\" => nothing\n", " \"beta_5\" => nothing\n", " \"objective\" => 0.0\n", " \"beta_4\" => nothing\n", " \"beta_3\" => nothing\n", " \"best_A\" => [0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; …\n", " \"beta_2\" => 1" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Store the result\n", "result = Dict(\n", " \"beta_1\" => beta_1,\n", " \"beta_2\" => beta_2,\n", " \"beta_3\" => nothing,\n", " \"beta_4\" => nothing,\n", " \"beta_5\" => nothing,\n", " \"delta_1\" => 0.5,\n", " \"delta_2\" => nothing,\n", " \"best_A\" => getvalue(A),\n", " \"objective\" => getobjectivevalue(m)\n", ")\n", "model_results[\"equality\"] = result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.4. Combination of student happiness and experience fairness (relaxed)\n", "\n", "Now let's see if we can combine two objectives together into a trade-off model. The standard form of this model is described [here](#relaxed_model). This trade-off is different from the models we have seen in the lecture. One objective (group fairness) is relaxed into a constraint, so its trade-off parameter $\\delta_1$ is embedded in the constraint. The other parameter $\\delta_2$ as usual is a scaler of the objective function (happiness).\n", "\n", "We use the same model parameters from [3.1](#3.1.-Model-with-only-considering-the-happiness-of-students) and 3.3.\n", "\n", "\n", "$$\n", "l = 5, \\quad u = 7, \\quad\n", "\\beta_1 = \\beta_2 = 1, \\quad\n", "\\beta_3 = \n", "\\begin{bmatrix}\n", "10 \\\\ 8 \\\\ 8 \\\\ 6 \\\\ 6 \\\\ 6 \\\\ 4 \\\\ 3\n", "\\end{bmatrix}, \\quad\n", "\\beta_4 = \n", "\\begin{bmatrix}\n", "6 \\\\ 5 \\\\ 4\n", "\\end{bmatrix}, \\quad\n", "\\beta_5 = \n", "\\begin{bmatrix}\n", "-500 \\\\ -500\n", "\\end{bmatrix}\n", "$$\n", "\n", "We will do more experiments on the trade-off parameter in the [analysis section](#4.3-Trade-off-Analysis), here we just use,\n", "\n", "$$\n", "\\delta_1 = 0.5, \\quad \\delta_2 = 5\n", "$$" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "δ1 = 0.5\n", "δ2 = 5;" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Academic license - for non-commercial use only\n", "189.471085 seconds (168 allocations: 16.736 MiB, 0.00% gc time)\n" ] }, { "data": { "text/plain": [ ":Optimal" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Running this cell will take about 190 seconds.\n", "\n", "# Start building the model\n", "m = Model(solver=GurobiSolver(OutputFlag=0))\n", "\n", "# ----------------------- Variables --------------------------\n", "@variable(m, A[1:STUDENT_NUM, 1:PROJECT_NUM], Bin)\n", "@variable(m, z1[1:PROJECT_NUM], Bin)\n", "@variable(m, z2[1:PROJECT_NUM], Bin)\n", "\n", "# ----------------------- Constraints --------------------------\n", "# Project size constraint (variable lower bound)\n", "@constraint(m, [j in 1:PROJECT_NUM], 5 * z1[j] <= sum(A[:,j]))\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z1[j])\n", "\n", "# Student choice constraint\n", "@constraint(m, [i in 1:STUDENT_NUM], sum(A[i,:]) == 1)\n", "\n", "# Proposal Writer Constraint (logical constriang) \n", "# Intermediate steps\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) <= 7 * z2[j])\n", "@constraint(m, [j in 1:PROJECT_NUM], sum(A[:,j]) >= z2[j])\n", "# Original constraint\n", "@constraint(m, [j in 1:PROJECT_NUM], z2[j] <= A[data[\"project_writers\"][j], j])\n", "\n", "# -------------------- Extra Constraint for Exp Equality ---------------------\n", "# Get the size of each project\n", "@expression(m, group_size[j in 1:PROJECT_NUM], sum(A[:,j]))\n", "\n", "# Combined exp level for each group\n", "@expression(m, experience, (beta_1 .* (A' * f1)) + (beta_2 .* (A' * f2)))\n", "\n", "# Relaxed constraint\n", "@constraint(m, group_size .* (E1 - δ1 * σ) .<= experience)\n", "@constraint(m, experience .<= group_size .* (E1 + δ1 * σ))\n", "\n", "# ----------------------- Objectives for Happiness --------------------------\n", "# F3 factor: project bonus\n", "@expression(m, Hf3, sum(A .* f3))\n", "\n", "# F4 factor: bonus for being in the group with friends\n", "@expression(m, Hf4, sum((A * A') .* f4))\n", "\n", "# F5 factor: penalty for being in the group with disliked people\n", "@expression(m, Hf5, sum((A * A') .* f5))\n", "\n", "# For this model, objective is just to maximize the happiness of students\n", "@objective(m, Max, δ2 * (Hf3 + Hf4 + Hf5))\n", "\n", "@time solve(m)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Dict{String,Any} with 9 entries:\n", " \"beta_1\" => 1\n", " \"delta_1\" => 0.5\n", " \"delta_2\" => 5\n", " \"beta_5\" => [-500, -500]\n", " \"objective\" => 5635.0\n", " \"beta_4\" => [6, 5, 4]\n", " \"beta_3\" => [10, 8, 8, 6, 6, 6, 4, 3]\n", " \"best_A\" => [-0.0 -0.0 … -0.0 -0.0; 0.0 0.0 … -0.0 0.0; … ; -0.0 -0.0 … -0…\n", " \"beta_2\" => 1" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Store the result\n", "result = Dict(\n", " \"beta_1\" => beta_1,\n", " \"beta_2\" => beta_2,\n", " \"beta_3\" => beta_3,\n", " \"beta_4\" => beta_4,\n", " \"beta_5\" => beta_5,\n", " \"delta_1\" => 0.5,\n", " \"delta_2\" => 5,\n", " \"best_A\" => getvalue(A),\n", " \"objective\" => getobjectivevalue(m)\n", ")\n", "model_results[\"combined_5_05\"] = result\n", "\n", "# Write the results to disk, so we don't need to run all the optimization for interpretation\n", "# save(\"./data/model_results.jld\", \"results\", model_results)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 4. Results and discussion ##\n", "\n", "After getting the results of our model, we can analyze the model and solutions from different perspectives to gain more domain insights." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.1 Solution Visualization\n", "\n", "In this section, we will try different ways to visualize the solution of our model to help instructors specify and understand the optimal group assignment.\n", "\n", "#### 4.1.1. Table Visualization\n", "\n", "We can try to use a table view to tell instructors how to assign groups." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# We keep redefining these two global variables and load data, to \n", "# make each section of codes is self-contained. So reader can jump\n", "# back and force to play around our implementations.\n", "\n", "using JLD, DataFrames\n", "\n", "STUDENT_NUM = 90\n", "PROJECT_NUM = 29;\n", "\n", "results = load(\"./data/model_results.jld\")[\"results\"]\n", "solution = results[\"combined_5_05\"][\"best_A\"];" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Project IndexMembers
12[4, 5, 11, 43, 45, 72, 87]
24[9, 10, 16, 36, 46, 58, 71]
36[12, 18, 33, 60, 63, 74, 89]
412[13, 31, 57, 81, 90]
513[21, 29, 35, 39, 50, 73, 80]
614[23, 38, 40, 59, 62, 69, 76]
715[24, 34, 44, 70, 85]
817[6, 7, 25, 28, 75, 79, 86]
918[20, 22, 27, 47, 65]
1021[3, 8, 41, 77, 88]
1122[17, 26, 55, 68, 84]
1223[19, 53, 56, 64, 66, 83]
1325[1, 2, 42, 54, 67]
1427[14, 15, 37, 48, 78]
1528[30, 32, 49, 51, 52, 61, 82]
" ], "text/plain": [ "15×2 DataFrames.DataFrame\n", "│ Row │ Project Index │ Members │\n", "├─────┼───────────────┼──────────────────────────────┤\n", "│ 1 │ 2 │ [4, 5, 11, 43, 45, 72, 87] │\n", "│ 2 │ 4 │ [9, 10, 16, 36, 46, 58, 71] │\n", "│ 3 │ 6 │ [12, 18, 33, 60, 63, 74, 89] │\n", "│ 4 │ 12 │ [13, 31, 57, 81, 90] │\n", "│ 5 │ 13 │ [21, 29, 35, 39, 50, 73, 80] │\n", "│ 6 │ 14 │ [23, 38, 40, 59, 62, 69, 76] │\n", "│ 7 │ 15 │ [24, 34, 44, 70, 85] │\n", "│ 8 │ 17 │ [6, 7, 25, 28, 75, 79, 86] │\n", "│ 9 │ 18 │ [20, 22, 27, 47, 65] │\n", "│ 10 │ 21 │ [3, 8, 41, 77, 88] │\n", "│ 11 │ 22 │ [17, 26, 55, 68, 84] │\n", "│ 12 │ 23 │ [19, 53, 56, 64, 66, 83] │\n", "│ 13 │ 25 │ [1, 2, 42, 54, 67] │\n", "│ 14 │ 27 │ [14, 15, 37, 48, 78] │\n", "│ 15 │ 28 │ [30, 32, 49, 51, 52, 61, 82] │" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "groups = []\n", "assignments = []\n", "\n", "# Find winning projects and their members\n", "for j in 1:PROJECT_NUM\n", " students = round.(Int64, solution[:,j])\n", " members = [i for i in 1:STUDENT_NUM if students[i] == 1]\n", " if !isempty(members)\n", " push!(groups, j)\n", " push!(assignments, members)\n", " end\n", "end\n", "\n", "# Use dataframe to display a table\n", "df = DataFrame(project_index=groups, member=assignments)\n", "names!(df, [Symbol(\"Project Index\"), Symbol(\"Members\")])\n", "df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the table above, instructors can easily specify which projects win the bidding phase and their associated members.\n", "\n", "#### 4.1.2. Graph Visualization\n", "\n", "We also can try to design a graph visualization to express the results of group assignment. We didn't find any Julia graphic packages which can be used for this task, so we decide to use a popular interactive visualization library `D3.js`. \n", "\n", "We have tried to use `interact.jl` and `D3Magic.jl` to embed the Javascript code into Jupyter cells, but currently `interact.jl` [doesn't work](https://github.com/JuliaGizmos/Interact.jl/issues/193) with pure Javascript visualization. Therefore, we just put the non-interactive version image inside Jupyter.\n", "\n", "To see the interactive version, please click here, which links to a local `html` file submitted with this notebook. Inside each blue box, positions of nodes are random, so you can refresh the `html` to have different views.\n", "\n", "This is our first time to use `D3.js`, so we are sorry if the plot looks rather fuzzy.\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this visualization, each node (circle) is a student, and it is colored according to the student's project preference. If one student is assigned to a project he/she likes, we color it green. The lightness of color is associated to the rank of assigned project in each student's project preference. For example, student $67$ ranked project $25$ as his/her first choice, so the color of node $67$ is light green.\n", "\n", "Each blue rectangle is a winning project with index labeled on the top left corner. Student nodes are contained in the project boxes they are assigned to. **Instructors can move cursor over each node to see the student labels.**\n", "\n", "Gray edges link students to their desired teammates, and red edges connects students with the ones they don't want to work with. If one gray edge looks darker, then it means two nodes mutually desires each other.\n", "\n", "From this visualization, we can see:\n", "\n", "1. Each winning project indeed only has $5$ to $7$ members.\n", "2. No students are paired with the people they don't want to work with.\n", "3. Most of the students are assigned to one of their interested projects (green). For the few students who are not assigned to their top $8$ desired projects (black), they have their friends in the same group, so presumably they are happy too.\n", "\n", "We found it is very interesting to observe this plot. Project $2$ has $4$ students who don't like project $2$ at all. However, all members of project $2$ are mutually friends! If you look it carefully, you can see those nodes almost form a closed cycle. They might be one of the friend circles we intentionally generated (read more [here](#friendcircle)). Therefore, it is up to instructors to choose the weights between desired teammates and desired projects ($\\beta_3$ and $\\beta_4$)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.2. Experience level visualization\n", "\n", "It can be very helpful to see how the group experience level distributes. We know it is controlled by the ranged we defined, but we do not have a good sense of whether the pre-defined range is \"good\" enough.\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Dict{String,Any} with 6 entries:\n", " \"exp_1\" => [4, 2, 3, 1, 3, 4, 3, 3, 4, 2 … 4, 4, 2, 4, 4, 1, 4, 4…\n", " \"project_writers\" => [69, 87, 13, 36, 29, 12, 37, 62, 60, 74 … 20, 41, 55, …\n", " \"exp_2\" => [4, 2, 5, 2, 4, 3, 2, 2, 2, 4 … 4, 5, 4, 3, 3, 4, 4, 1…\n", " \"preference\" => [14 19 … 12 29; 28 25 … 26 12; … ; 20 6 … 0 0; 16 17 … 1…\n", " \"likes\" => [81 32 42; 67 18 0; … ; 0 0 0; 13 0 0]\n", " \"dislikes\" => [0 0; 0 0; … ; 0 0; 0 0]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using Gadfly, DataFrames, JLD\n", "\n", "STUDENT_NUM = 90\n", "PROJECT_NUM = 29\n", "\n", "# Load the saved model results\n", "results = load(\"./data/model_results.jld\")[\"results\"]\n", "data = load(\"./data/data.jld\")" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "get_mean_exp (generic function with 1 method)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Construct a dataframe using A to encode mean exp level by groups\n", "function get_mean_exp(A, beta_1, beta_2, exp_1, exp_2)\n", " # Make A integer\n", " A = round.(Int, A)\n", " means = []\n", " for j in 1:PROJECT_NUM\n", " combined_exp = beta_1 * (A[:,j] .* exp_1) + beta_2 * (A[:,j] .* exp_2)\n", " \n", " # The size is number of members in group j\n", " g_size = sum(A[:,j])\n", " \n", " if g_size == 0\n", " push!(means, 0)\n", " else\n", " push!(means, sum(combined_exp) / g_size)\n", " end\n", " end\n", " return DataFrame(group_num = 1:PROJECT_NUM, mean = convert.(Float64, means))\n", "end" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "best_A = results[\"combined_5_05\"][\"best_A\"]\n", "beta_1, beta_2 = results[\"combined_5_05\"][\"beta_1\"], results[\"combined_5_05\"][\"beta_2\"]\n", "exp_1, exp_2 = data[\"exp_1\"], data[\"exp_2\"]\n", "\n", "df = get_mean_exp(best_A, beta_1, beta_2, exp_1, exp_2);" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " Project Index\n", " \n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Individual Max\n", " Individual Min\n", " Individual Mean\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0.0\n", " 2.5\n", " 5.0\n", " 7.5\n", " 10.0\n", " \n", " \n", " Group Experience Mean\n", " \n", " \n", " Group Experience Distribution\n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " Project Index\n", " \n", " \n", " -50\n", " -40\n", " -30\n", " -20\n", " -10\n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " 50\n", " 60\n", " 70\n", " 80\n", " 90\n", " -34\n", " -32\n", " -30\n", " -28\n", " -26\n", " -24\n", " -22\n", " -20\n", " -18\n", " -16\n", " -14\n", " -12\n", " -10\n", " -8\n", " -6\n", " -4\n", " -2\n", " 0\n", " 2\n", " 4\n", " 6\n", " 8\n", " 10\n", " 12\n", " 14\n", " 16\n", " 18\n", " 20\n", " 22\n", " 24\n", " 26\n", " 28\n", " 30\n", " 32\n", " 34\n", " 36\n", " 38\n", " 40\n", " 42\n", " 44\n", " 46\n", " 48\n", " 50\n", " 52\n", " 54\n", " 56\n", " 58\n", " 60\n", " 62\n", " 64\n", " 66\n", " 68\n", " -50\n", " 0\n", " 50\n", " 100\n", " -35\n", " -30\n", " -25\n", " -20\n", " -15\n", " -10\n", " -5\n", " 0\n", " 5\n", " 10\n", " 15\n", " 20\n", " 25\n", " 30\n", " 35\n", " 40\n", " 45\n", " 50\n", " 55\n", " 60\n", " 65\n", " 70\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Individual Max\n", " Individual Min\n", " Individual Mean\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " -12.5\n", " -10.0\n", " -7.5\n", " -5.0\n", " -2.5\n", " 0.0\n", " 2.5\n", " 5.0\n", " 7.5\n", " 10.0\n", " 12.5\n", " 15.0\n", " 17.5\n", " 20.0\n", " 22.5\n", " -10.0\n", " -9.5\n", " -9.0\n", " -8.5\n", " -8.0\n", " -7.5\n", " -7.0\n", " -6.5\n", " -6.0\n", " -5.5\n", " -5.0\n", " -4.5\n", " -4.0\n", " -3.5\n", " -3.0\n", " -2.5\n", " -2.0\n", " -1.5\n", " -1.0\n", " -0.5\n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " 3.0\n", " 3.5\n", " 4.0\n", " 4.5\n", " 5.0\n", " 5.5\n", " 6.0\n", " 6.5\n", " 7.0\n", " 7.5\n", " 8.0\n", " 8.5\n", " 9.0\n", " 9.5\n", " 10.0\n", " 10.5\n", " 11.0\n", " 11.5\n", " 12.0\n", " 12.5\n", " 13.0\n", " 13.5\n", " 14.0\n", " 14.5\n", " 15.0\n", " 15.5\n", " 16.0\n", " 16.5\n", " 17.0\n", " 17.5\n", " 18.0\n", " 18.5\n", " 19.0\n", " 19.5\n", " 20.0\n", " -10\n", " 0\n", " 10\n", " 20\n", " -10\n", " -9\n", " -8\n", " -7\n", " -6\n", " -5\n", " -4\n", " -3\n", " -2\n", " -1\n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " 7\n", " 8\n", " 9\n", " 10\n", " 11\n", " 12\n", " 13\n", " 14\n", " 15\n", " 16\n", " 17\n", " 18\n", " 19\n", " 20\n", " \n", " \n", " Group Experience Mean\n", " \n", " \n", " Group Experience Distribution\n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Nullable{Compose.UnitBox}(), Nullable{Compose.Rotation}(), Nullable{Compose.Mirror}(), List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}},Tuple{Measures.Add{Measures.Add{Measures.Length{:w,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Add{Measures.Add{Measures.Length{:h,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}}}}((5.0mm, 5.0mm), (1.0w + -5.0mm + -5.0mm, 1.0h + -5.0mm + -5.0mm)), Nullable{Compose.UnitBox}(), Nullable{Compose.Rotation}(), Nullable{Compose.Mirror}(), List([Compose.Table(Array{Compose.Context,1}[Compose.Context[] Compose.Context[] Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([]), List([Compose.Form{Compose.TextPrimitive{Tuple{Measures.Length{:w,Float64},Measures.Add{Measures.Add{Measures.Length{:h,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}}(Compose.TextPrimitive{Tuple{Measures.Length{:w,Float64},Measures.Add{Measures.Add{Measures.Length{:h,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}[Compose.TextPrimitive{Tuple{Measures.Length{:w,Float64},Measures.Add{Measures.Add{Measures.Length{:h,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}((0.5w, 1.0h + -3.61167mm + -2.0mm), \"Group Experience Distribution\", Compose.HCenter(), Compose.VTop(), Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(0.0, (0.5w, 0.5h)))], Symbol(\"\"))]), List([Compose.Property{Compose.FontSizePrimitive}(Compose.FontSizePrimitive[Compose.FontSizePrimitive(3.88056mm)]), Compose.Property{Compose.FontPrimitive}(Compose.FontPrimitive[Compose.FontPrimitive(\"'PT Sans','Helvetica Neue','Helvetica',sans-serif\")]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.337255,0.290196,0.333333,1.0))]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))])]), 0, false, false, false, false, 49.9767, 5.61167, 0.0, Symbol(\"\"))]; Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 0.0mm, 0.0mm, 2.0mm, 2.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#75, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 43.435, 7.61167, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 0.0mm, 0.0mm, 2.0mm, 2.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#76, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 7.61167, 43.435, 0.0, Symbol(\"\"))] Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 0.0mm, 0.0mm, 2.0mm, 2.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#55, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 7.69333, 13.3667, 3.0, Symbol(\"\"))] Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 2.0mm, 2.0mm), #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 2.0mm, 2.0mm), #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}}(Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}[Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((0.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((1.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 6.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((2.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((3.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 7.14286cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((4.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((5.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 5.57143cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((6.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((7.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((8.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((9.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy) … Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((19.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((20.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 5.6cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((21.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 7.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((22.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 5.5cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((23.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((24.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 5.8cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((25.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((26.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 6.4cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((27.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 7.0cy), Compose.RectanglePrimitive{Tuple{Measures.Measure,Measures.Measure},Measures.Measure,Measures.Measure}((28.5cx + -0.025mm, 0.0cy), 1.0cx + 0.05mm, 0.0cy)], Symbol(\"\"))]), List([Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.0,0.747369,1.0,1.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([Compose.Property{Compose.SVGAttributePrimitive}(Compose.SVGAttributePrimitive[Compose.SVGAttributePrimitive(\"shape-rendering\", \"crispEdges\")]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.3mm)])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.TextPrimitive{Tuple{Measures.Add{Measures.Length{:cx,Int64},Measures.Length{:mm,Float64}},Measures.Add{Measures.Length{:cy,Float64},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}}(Compose.TextPrimitive{Tuple{Measures.Add{Measures.Length{:cx,Int64},Measures.Length{:mm,Float64}},Measures.Add{Measures.Length{:cy,Float64},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}[Compose.TextPrimitive{Tuple{Measures.Add{Measures.Length{:cx,Int64},Measures.Length{:mm,Float64}},Measures.Add{Measures.Length{:cy,Float64},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}((29cx + 2.0mm, 9.0cy + 0.0mm), \"Individual Max\", Compose.HLeft(), Compose.VCenter(), Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(0.0, (0.5w, 0.5h))), Compose.TextPrimitive{Tuple{Measures.Add{Measures.Length{:cx,Int64},Measures.Length{:mm,Float64}},Measures.Add{Measures.Length{:cy,Float64},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}((29cx + 2.0mm, 2.0cy + 0.0mm), \"Individual Min\", Compose.HLeft(), Compose.VCenter(), Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(0.0, (0.5w, 0.5h))), Compose.TextPrimitive{Tuple{Measures.Add{Measures.Length{:cx,Int64},Measures.Length{:mm,Float64}},Measures.Add{Measures.Length{:cy,Float64},Measures.Length{:mm,Float64}}},Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}}((29cx + 2.0mm, 6.3cy + 0.0mm), \"Individual Mean\", Compose.HLeft(), Compose.VCenter(), Compose.Rotation{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(0.0, (0.5w, 0.5h)))], Symbol(\"\"))]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.298039,0.25098,0.294118,1.0))]), Compose.Property{Compose.FontSizePrimitive}(Compose.FontSizePrimitive[Compose.FontSizePrimitive(2.82222mm)]), Compose.Property{Compose.FontPrimitive}(Compose.FontPrimitive[Compose.FontPrimitive(\"'PT Sans Caption','Helvetica Neue','Helvetica',sans-serif\")])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0cx, 6.3cy), (29cx, 6.3cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.529101mm)]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[4.0mm, 1.0mm])]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(1.0,0.752941,0.796079,1.0))])]), 2, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0cx, 2cy), (29cx, 2cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.529101mm)]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[4.0mm, 1.0mm])]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(1.0,0.752941,0.796079,1.0))])]), 2, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0cx, 9cy), (29cx, 9cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.529101mm)]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"geometry\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[4.0mm, 1.0mm])]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(1.0,0.752941,0.796079,1.0))])]), 2, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"plotpanel\")])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Float64,Float64,Float64,Float64}(0.0, 0.0, 1.0, 1.0, 0.0mm, 0.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Add{Measures.Add{Measures.Length{:w,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}((1.0w + -3.0mm + -4.0mm, 3.0mm), (4.0mm, 4.0mm)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}}(Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}[Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.2cx, 0.4cy), (0.4cx, 0.4cy), (0.4cx, 0.2cy), (0.6cx, 0.2cy), (0.6cx, 0.4cy), (0.8cx, 0.4cy), (0.8cx, 0.6cy), (0.6cx, 0.6cy), (0.6cx, 0.8cy), (0.4cx, 0.8cy), (0.4cx, 0.6cy), (0.2cx, 0.6cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"button_logo\")]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.415686,0.415686,0.415686,1.0))])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}[Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}((0.0w, 0.0h), 1.0w, 1.0h)], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseover_color\\\", \\\"#CD5C5C\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseout_color\\\", \\\"#6A6A6A\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"click(Gadfly.zoomslider_zoomin_click)\\n.mouseenter(Gadfly.zoomslider_button_mouseover)\\n.mouseleave(Gadfly.zoomslider_button_mouseout)\\n\", Measures.Measure[])]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.917647,0.917647,0.917647,1.0))]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.3mm)]), Compose.Property{Compose.StrokeOpacityPrimitive}(Compose.StrokeOpacityPrimitive[Compose.StrokeOpacityPrimitive(0.0)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.415686,0.415686,0.415686,1.0))])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Add{Measures.Add{Measures.Add{Measures.Add{Measures.Length{:w,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}((1.0w + -3.0mm + -4.0mm + -20.0mm + 0.5mm, 3.0mm), (19.0mm, 4.0mm)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Add{Measures.Length{:cx,Float64},Measures.Length{:mm,Float64}},Measures.Length{:cy,Float64}},Measures.Length{:mm,Float64},Measures.Length{:h,Float64}}}(Compose.RectanglePrimitive{Tuple{Measures.Add{Measures.Length{:cx,Float64},Measures.Length{:mm,Float64}},Measures.Length{:cy,Float64}},Measures.Length{:mm,Float64},Measures.Length{:h,Float64}}[Compose.RectanglePrimitive{Tuple{Measures.Add{Measures.Length{:cx,Float64},Measures.Length{:mm,Float64}},Measures.Length{:cy,Float64}},Measures.Length{:mm,Float64},Measures.Length{:h,Float64}}((0.5cx + -1.0mm, 0.0cy), 2.0mm, 1.0h)], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"max_pos\\\", %x)\\n\", Measures.Measure[1.0w + -3.0mm + -4.0mm + -20.0mm + 0.5mm + 20.0mm + -1.0mm + -1.0mm])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"min_pos\\\", %x)\\n\", Measures.Measure[1.0w + -3.0mm + -4.0mm + -20.0mm + 0.5mm + 1.0mm])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseover_color\\\", \\\"#CD5C5C\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseout_color\\\", \\\"#6A6A6A\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"drag(Gadfly.zoomslider_thumb_dragmove,\\n Gadfly.zoomslider_thumb_dragstart,\\n Gadfly.zoomslider_thumb_dragend)\\n\", Measures.Measure[])]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"zoomslider_thumb\")]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.415686,0.415686,0.415686,1.0))])]), 1, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}[Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}((0.0w, 0.0h), 1.0w, 1.0h)], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"max_pos\\\", %x)\\n\", Measures.Measure[1.0w + -3.0mm + -4.0mm + -20.0mm + 0.5mm + 20.0mm + -1.0mm + -1.0mm])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"min_pos\\\", %x)\\n\", Measures.Measure[1.0w + -3.0mm + -4.0mm + -20.0mm + 0.5mm + 1.0mm])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"click(Gadfly.zoomslider_track_click)\", Measures.Measure[])]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.917647,0.917647,0.917647,1.0))])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Add{Measures.Add{Measures.Add{Measures.Length{:w,Float64},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Measures.Length{:mm,Float64}},Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}((1.0w + -3.0mm + -8.0mm + -20.0mm, 3.0mm), (4.0mm, 4.0mm)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}}(Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}[Compose.SimplePolygonPrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.2cx, 0.4cy), (0.8cx, 0.4cy), (0.8cx, 0.6cy), (0.2cx, 0.6cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"button_logo\")]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.415686,0.415686,0.415686,1.0))])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}[Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}((0.0w, 0.0h), 1.0w, 1.0h)], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseover_color\\\", \\\"#CD5C5C\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"data(\\\"mouseout_color\\\", \\\"#6A6A6A\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"click(Gadfly.zoomslider_zoomout_click)\\n.mouseenter(Gadfly.zoomslider_button_mouseover)\\n.mouseleave(Gadfly.zoomslider_button_mouseout)\\n\", Measures.Measure[])]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.917647,0.917647,0.917647,1.0))]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.3mm)]), Compose.Property{Compose.StrokeOpacityPrimitive}(Compose.StrokeOpacityPrimitive[Compose.StrokeOpacityPrimitive(0.0)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.415686,0.415686,0.415686,1.0))])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([Compose.Property{Compose.FillOpacityPrimitive}(Compose.FillOpacityPrimitive[Compose.FillOpacityPrimitive(0.0)]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide zoomslider\")]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))])]), 0, false, true, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 1000, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, -12.5cy), (1.0w, -12.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, -10.0cy), (1.0w, -10.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, -7.5cy), (1.0w, -7.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, -5.0cy), (1.0w, -5.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, -2.5cy), (1.0w, -2.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 0.0cy), (1.0w, 0.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 2.5cy), (1.0w, 2.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 5.0cy), (1.0w, 5.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 7.5cy), (1.0w, 7.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 10.0cy), (1.0w, 10.0cy)]) … Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 11.0cy), (1.0w, 11.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 12.0cy), (1.0w, 12.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 13.0cy), (1.0w, 13.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 14.0cy), (1.0w, 14.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 15.0cy), (1.0w, 15.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 16.0cy), (1.0w, 16.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 17.0cy), (1.0w, 17.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 18.0cy), (1.0w, 18.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 19.0cy), (1.0w, 19.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 20.0cy), (1.0w, 20.0cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"plotroot().data(\\\"unfocused_ygrid_color\\\", \\\"#D0D0E0\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"plotroot().data(\\\"focused_ygrid_color\\\", \\\"#A0A0A0\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.SVGAttributePrimitive}(Compose.SVGAttributePrimitive[Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\") … Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\")]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide ygridlines xfixed\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[0.5mm, 0.5mm])]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.2mm)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.815686,0.815686,0.878431,1.0))]), Compose.Property{Compose.VisiblePrimitive}(Compose.VisiblePrimitive[Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true) … Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false)])]), 0, false, true, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 0.0cy), (1.0w, 0.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 2.5cy), (1.0w, 2.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 5.0cy), (1.0w, 5.0cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 7.5cy), (1.0w, 7.5cy)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0w, 10.0cy), (1.0w, 10.0cy)])], Symbol(\"\"))]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide ygridlines xfixed\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[0.5mm, 0.5mm])]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.2mm)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.815686,0.815686,0.878431,1.0))])]), 0, false, false, true, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(-50.0cx, 0.0h), (-50.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(-40.0cx, 0.0h), (-40.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(-30.0cx, 0.0h), (-30.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(-20.0cx, 0.0h), (-20.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(-10.0cx, 0.0h), (-10.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0cx, 0.0h), (0.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(10.0cx, 0.0h), (10.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(20.0cx, 0.0h), (20.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(30.0cx, 0.0h), (30.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(40.0cx, 0.0h), (40.0cx, 1.0h)]) … Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(25.0cx, 0.0h), (25.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(30.0cx, 0.0h), (30.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(35.0cx, 0.0h), (35.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(40.0cx, 0.0h), (40.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(45.0cx, 0.0h), (45.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(50.0cx, 0.0h), (50.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(55.0cx, 0.0h), (55.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(60.0cx, 0.0h), (60.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(65.0cx, 0.0h), (65.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(70.0cx, 0.0h), (70.0cx, 1.0h)])], Symbol(\"\"))]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"plotroot().data(\\\"unfocused_xgrid_color\\\", \\\"#D0D0E0\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"plotroot().data(\\\"focused_xgrid_color\\\", \\\"#A0A0A0\\\")\\n\", Measures.Measure[])]), Compose.Property{Compose.SVGAttributePrimitive}(Compose.SVGAttributePrimitive[Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"1.0\") … Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\"), Compose.SVGAttributePrimitive(\"gadfly:scale\", \"5.0\")]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide xgridlines yfixed\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[0.5mm, 0.5mm])]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.2mm)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.815686,0.815686,0.878431,1.0))]), Compose.Property{Compose.VisiblePrimitive}(Compose.VisiblePrimitive[Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(true), Compose.VisiblePrimitive(false) … Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false), Compose.VisiblePrimitive(false)])]), 0, false, true, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.LinePrimitive}(Compose.LinePrimitive[Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(0.0cx, 0.0h), (0.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(10.0cx, 0.0h), (10.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(20.0cx, 0.0h), (20.0cx, 1.0h)]), Compose.LinePrimitive{Tuple{Measures.Measure,Measures.Measure}}(Tuple{Measures.Measure,Measures.Measure}[(30.0cx, 0.0h), (30.0cx, 1.0h)])], Symbol(\"\"))]), List([Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide xgridlines yfixed\")]), Compose.Property{Compose.StrokeDashPrimitive}(Compose.StrokeDashPrimitive[Compose.StrokeDashPrimitive(Measures.Measure[0.5mm, 0.5mm])]), Compose.Property{Compose.LineWidthPrimitive}(Compose.LineWidthPrimitive[Compose.LineWidthPrimitive(0.2mm)]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.815686,0.815686,0.878431,1.0))])]), 0, false, false, true, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), #NULL, #NULL, #NULL, List([]), List([Compose.Form{Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}(Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}[Compose.RectanglePrimitive{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Measures.Length{:w,Float64},Measures.Length{:h,Float64}}((0.0w, 0.0h), 1.0w, 1.0h)], Symbol(\"\"))]), List([Compose.Property{Compose.SVGAttributePrimitive}(Compose.SVGAttributePrimitive[Compose.SVGAttributePrimitive(\"pointer-events\", \"visible\")]), Compose.Property{Compose.FillOpacityPrimitive}(Compose.FillOpacityPrimitive[Compose.FillOpacityPrimitive(1.0)]), Compose.Property{Compose.FillPrimitive}(Compose.FillPrimitive[Compose.FillPrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.StrokePrimitive}(Compose.StrokePrimitive[Compose.StrokePrimitive(RGBA{Float64}(0.0,0.0,0.0,0.0))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"guide background\")])]), -1, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), -1, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([Compose.Property{Compose.JSCallPrimitive}(Compose.JSCallPrimitive[Compose.JSCallPrimitive(\"init_gadfly()\", Measures.Measure[])])]), 0, true, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]; Compose.Context[] Compose.Context[] Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#34, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 11.7133, 4.67333, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#35, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 10.6933, 5.34667, 3.0, Symbol(\"\"))]; Compose.Context[] Compose.Context[] Compose.Context[Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#70, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 27.9908, 9.61167, 0.0, Symbol(\"\")), Compose.Context(Measures.BoundingBox{Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}},Tuple{Measures.Length{:w,Float64},Measures.Length{:h,Float64}}}((0.0w, 0.0h), (1.0w, 1.0h)), Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 0.0mm, 0.0mm), #NULL, #NULL, List([Compose.AdhocContainerPromise(Gadfly.Guide.#71, 0, false, false, nothing, nothing)]), List([]), List([]), 0, false, false, false, false, 9.61167, 27.9908, 3.0, Symbol(\"\"))]], 3:3, 2:2, nothing, nothing, nothing, Any[], Nullable{Compose.UnitBox}(Compose.UnitBox{Int64,Float64,Int64,Float64}(0, 10.0, 34, -10.0, 2.0mm, 2.0mm, 2.0mm, 2.0mm)), 0, false, false)]), List([]), List([Compose.Property{Compose.JSIncludePrimitive}(Compose.JSIncludePrimitive[Compose.JSIncludePrimitive(\"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\", (\"Gadfly\", \"Gadfly\"))]), Compose.Property{Compose.SVGClassPrimitive}(Compose.SVGClassPrimitive[Compose.SVGClassPrimitive(\"plotroot xscalable yscalable\")])]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))]), List([]), List([]), 0, false, false, false, false, nothing, nothing, 0.0, Symbol(\"\"))" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p1 = plot(layer(x=[0, PROJECT_NUM], y=[maximum(exp_1 * beta_1 + exp_2 * beta_2),\n", " maximum(exp_1 * beta_1 + exp_2 * beta_2)],\n", " Geom.line(), Theme(default_color=\"pink\", line_style=:dash, line_width=2px)),\n", " layer(x=[0, PROJECT_NUM], y=[minimum(exp_1 * beta_1 + exp_2 * beta_2),\n", " minimum(exp_1 * beta_1 + exp_2 * beta_2)],\n", " Geom.line(), Theme(default_color=\"pink\", line_style=:dash, line_width=2px)),\n", " layer(x=[0, PROJECT_NUM], y=[mean(exp_1 * beta_1 + exp_2 * beta_2),\n", " mean(exp_1 * beta_1 + exp_2 * beta_2)],\n", " Geom.line(), Theme(default_color=\"pink\", line_style=:dash, line_width=2px)),\n", " layer(x=[PROJECT_NUM, PROJECT_NUM, PROJECT_NUM],\n", " y=[maximum(exp_1 * beta_1 + exp_2 * beta_2),\n", " minimum(exp_1 * beta_1 + exp_2 * beta_2),\n", " mean(exp_1 * beta_1 + exp_2 * beta_2),],\n", " label=[\"Individual Max\", \"Individual Min\", \"Individual Mean\"],\n", " Geom.label(position=:right)),\n", " layer(df, x=\"group_num\", y=\"mean\", Geom.bar),\n", " Coord.cartesian(xmin=0, xmax=PROJECT_NUM+5),\n", " Guide.xlabel(\"Project Index\"), Guide.ylabel(\"Group Experience Mean\"),\n", " Guide.title(\"Group Experience Distribution\")\n", " )\n", "\n", "set_default_plot_size(25cm, 10cm)\n", "render(p1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "From the plot above, we can see the relaxed approach to control experience equality is working. Compared to the overall range of individual combined experience level, the mean experience of groups is constrained in a reasonable region. One could further decrease $\\delta_1$ to make the distribution more even. We also see only few means are closed to the individual mean. It could imply this constraint is very sensitive (tight constraint)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.3 Trade-off Analysis\n", "\n", "We have assumed that there is a trade-off relationship between happiness score and group experience equality. To test our hypothesis, we have run this optimizations with 190 different $\\delta_1$ values from $2$ to $0.11$. The original plan was to try $200$ values from $2$ to $0.01$, but we stopped at $0.11$ because $\\delta_1 = 0.10$ took too long ($81$ minutes) to solve. Also note we didn't change the scaler $\\delta_2$, which were set to $5$ in our script.\n", "\n", "In this section, we will visualize and analyze the results from those 190 optimization solutions." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
delta_1timeobjectiveneg_objectiverangelabel
11.86271356783919632.9946819485680.0-5680.03.725427135678392δ1=1.86
21.18692307692307739.6409757255679.999969356109-5679.9999693561092.373846153846154δ1=1.19
30.898461538461538434.8720555195680.0-5680.01.7969230769230768δ1=0.90
41.221538461538461539.7978556075680.0-5680.02.443076923076923δ1=1.22
50.621228070175438672.490767585645.0-5645.01.2424561403508771δ1=0.62
60.771538461538461549.4806242795680.0-5680.01.543076923076923δ1=0.77
" ], "text/plain": [ "6×6 DataFrames.DataFrame\n", "│ Row │ delta_1 │ time │ objective │ neg_objective │ range │ label │\n", "├─────┼──────────┼─────────┼───────────┼───────────────┼─────────┼───────────┤\n", "│ 1 │ 1.86271 │ 32.9947 │ 5680.0 │ -5680.0 │ 3.72543 │ \"δ1=1.86\" │\n", "│ 2 │ 1.18692 │ 39.641 │ 5680.0 │ -5680.0 │ 2.37385 │ \"δ1=1.19\" │\n", "│ 3 │ 0.898462 │ 34.8721 │ 5680.0 │ -5680.0 │ 1.79692 │ \"δ1=0.90\" │\n", "│ 4 │ 1.22154 │ 39.7979 │ 5680.0 │ -5680.0 │ 2.44308 │ \"δ1=1.22\" │\n", "│ 5 │ 0.621228 │ 72.4908 │ 5645.0 │ -5645.0 │ 1.24246 │ \"δ1=0.62\" │\n", "│ 6 │ 0.771538 │ 49.4806 │ 5680.0 │ -5680.0 │ 1.54308 │ \"δ1=0.77\" │" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load the results for running optimization results over different delta1 value\n", "results = load(\"./data/trade_off_results.jld\")\n", "\n", "# Construct a dataframe to visualize\n", "delta_1s = []\n", "times = []\n", "objectives = []\n", "labels = []\n", "\n", "for k in collect(keys(results))\n", " cur_dict = results[k]\n", " push!(delta_1s, cur_dict[\"δ1\"])\n", " push!(times, cur_dict[\"time\"])\n", " push!(objectives, cur_dict[\"obj\"])\n", " push!(labels, \"δ1=$(k)\")\n", "end\n", "\n", "df = DataFrame(delta_1=delta_1s, time=times,\n", " objective=objectives, neg_objective=-objectives,\n", " range=2.*delta_1s, label=labels)\n", "head(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We first see how happiness is related to the group fairness." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " δ1 (Tolerance Level)\n", " \n", " \n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " δ1=0.76\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 5450\n", " 5500\n", " 5550\n", " 5600\n", " 5650\n", " 5700\n", " \n", " \n", " Happiness Score\n", " \n", " \n", " Happiness v.s. Experience Fairness\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " δ1 (Tolerance Level)\n", " \n", " \n", " -3.0\n", " -2.5\n", " -2.0\n", " -1.5\n", " -1.0\n", " -0.5\n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " 3.0\n", " 3.5\n", " 4.0\n", " 4.5\n", " 5.0\n", " 5.5\n", " -2.5\n", " -2.4\n", " -2.3\n", " -2.2\n", " -2.1\n", " -2.0\n", " -1.9\n", " -1.8\n", " -1.7\n", " -1.6\n", " -1.5\n", " -1.4\n", " -1.3\n", " -1.2\n", " -1.1\n", " -1.0\n", " -0.9\n", " -0.8\n", " -0.7\n", " -0.6\n", " -0.5\n", " -0.4\n", " -0.3\n", " -0.2\n", " -0.1\n", " 0.0\n", " 0.1\n", " 0.2\n", " 0.3\n", " 0.4\n", " 0.5\n", " 0.6\n", " 0.7\n", " 0.8\n", " 0.9\n", " 1.0\n", " 1.1\n", " 1.2\n", " 1.3\n", " 1.4\n", " 1.5\n", " 1.6\n", " 1.7\n", " 1.8\n", " 1.9\n", " 2.0\n", " 2.1\n", " 2.2\n", " 2.3\n", " 2.4\n", " 2.5\n", " 2.6\n", " 2.7\n", " 2.8\n", " 2.9\n", " 3.0\n", " 3.1\n", " 3.2\n", " 3.3\n", " 3.4\n", " 3.5\n", " 3.6\n", " 3.7\n", " 3.8\n", " 3.9\n", " 4.0\n", " 4.1\n", " 4.2\n", " 4.3\n", " 4.4\n", " 4.5\n", " 4.6\n", " 4.7\n", " 4.8\n", " 4.9\n", " 5.0\n", " -2.5\n", " 0.0\n", " 2.5\n", " 5.0\n", " -2.6\n", " -2.4\n", " -2.2\n", " -2.0\n", " -1.8\n", " -1.6\n", " -1.4\n", " -1.2\n", " -1.0\n", " -0.8\n", " -0.6\n", " -0.4\n", " -0.2\n", " 0.0\n", " 0.2\n", " 0.4\n", " 0.6\n", " 0.8\n", " 1.0\n", " 1.2\n", " 1.4\n", " 1.6\n", " 1.8\n", " 2.0\n", " 2.2\n", " 2.4\n", " 2.6\n", " 2.8\n", " 3.0\n", " 3.2\n", " 3.4\n", " 3.6\n", " 3.8\n", " 4.0\n", " 4.2\n", " 4.4\n", " 4.6\n", " 4.8\n", " 5.0\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " δ1=0.76\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 5150\n", " 5200\n", " 5250\n", " 5300\n", " 5350\n", " 5400\n", " 5450\n", " 5500\n", " 5550\n", " 5600\n", " 5650\n", " 5700\n", " 5750\n", " 5800\n", " 5850\n", " 5900\n", " 5950\n", " 6000\n", " 5200\n", " 5210\n", " 5220\n", " 5230\n", " 5240\n", " 5250\n", " 5260\n", " 5270\n", " 5280\n", " 5290\n", " 5300\n", " 5310\n", " 5320\n", " 5330\n", " 5340\n", " 5350\n", " 5360\n", " 5370\n", " 5380\n", " 5390\n", " 5400\n", " 5410\n", " 5420\n", " 5430\n", " 5440\n", " 5450\n", " 5460\n", " 5470\n", " 5480\n", " 5490\n", " 5500\n", " 5510\n", " 5520\n", " 5530\n", " 5540\n", " 5550\n", " 5560\n", " 5570\n", " 5580\n", " 5590\n", " 5600\n", " 5610\n", " 5620\n", " 5630\n", " 5640\n", " 5650\n", " 5660\n", " 5670\n", " 5680\n", " 5690\n", " 5700\n", " 5710\n", " 5720\n", " 5730\n", " 5740\n", " 5750\n", " 5760\n", " 5770\n", " 5780\n", " 5790\n", " 5800\n", " 5810\n", " 5820\n", " 5830\n", " 5840\n", " 5850\n", " 5860\n", " 5870\n", " 5880\n", " 5890\n", " 5900\n", " 5910\n", " 5920\n", " 5930\n", " 5940\n", " 5950\n", " 5200\n", " 5400\n", " 5600\n", " 5800\n", " 6000\n", " 5200\n", " 5220\n", " 5240\n", " 5260\n", " 5280\n", " 5300\n", " 5320\n", " 5340\n", " 5360\n", " 5380\n", " 5400\n", " 5420\n", " 5440\n", " 5460\n", " 5480\n", " 5500\n", " 5520\n", " 5540\n", " 5560\n", " 5580\n", " 5600\n", " 5620\n", " 5640\n", " 5660\n", " 5680\n", " 5700\n", " 5720\n", " 5740\n", " 5760\n", " 5780\n", " 5800\n", " 5820\n", " 5840\n", " 5860\n", " 5880\n", " 5900\n", " 5920\n", " 5940\n", " 5960\n", " \n", " \n", " Happiness Score\n", " \n", " \n", " Happiness v.s. Experience Fairness\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n" ], "text/plain": [ "Plot(...)" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p1 = plot(layer(df, x=\"delta_1\", y=\"objective\", Geom.point()),\n", " layer(df, xintercept=[0.76], x=[0.76], y=[5550],\n", " label=[\"δ1=0.76\"],\n", " Geom.vline, Geom.label(position=:left),\n", " Theme(default_color=\"pink\", line_style=:dot)),\n", " Coord.cartesian(xflip=true),\n", " Guide.title(\"Happiness v.s. Experience Fairness\"),\n", " Guide.xlabel(\"δ1 (Tolerance Level)\"),\n", " Guide.ylabel(\"Happiness Score\"))\n", "\n", "# Save plots\n", "# draw(SVG(\"./plots/happy_group.svg\", 25cm, 10cm), p1)\n", "\n", "set_default_plot_size(25cm, 10cm)\n", "p1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the plot above, we can see:\n", "\n", "1. When $\\delta_1 \\geq 0.76$, the happiness score is not changing regarding to a decreasing tolerance level. In other words, our model is not sensitive to the change of $\\delta_1$ when $\\delta_1 \\geq 0.76$. \n", "2. The relationship between happiness and experience fairness is like a stepwise function. Within each step, the model is not sensitive to the change of $\\delta_1$.\n", "3. When $\\delta_1$ decreases after $0.76$, the happiness score is significantly negatively affected by the tolerance level. From this simple plot, we guess the relationship is exponential. Therefore, there is indeed a trade-off between happiness score and group experience equality.\n", "\n", "We can try to understand the \"stepping\" effect using the following image:\n", "\n", "\n", "Suppose pink dots are available inter solutions, the blue line is a changing linear constraint, and the orange circle is a quadratic objective function with different values. Then within the change of the linear constraint, we do see there are small gaps between different optimal solutions. Within those ranges, the optimal solution and objective values are not sensitive to the small change of constraints. That's why we see flat objective values in the plot.\n", "\n", "If there is a trade-off relationship, let's see if we can analyze the Pareto curve. Note that our Pareto curve is a little bit different from the standard Pareto curve that we discussed in the lecture, since our trade-off parameter $\\delta_1$ is directly related to one of the \"objective\". Read more [here](#relaxation).\n", "\n", "Also note the Pareto we have learned is plotted based on two minimizing objectives, so we use the range (\"variance\") and negative happiness score as two objective functions. " ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "scrolled": false }, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " Negative Happiness Score\n", " \n", " \n", " -5600\n", " -5500\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " δ1=1.86\n", " δ1=1.19\n", " δ1=0.90\n", " δ1=1.22\n", " δ1=0.62\n", " δ1=0.77\n", " δ1=1.63\n", " δ1=2.15\n", " δ1=0.67\n", " δ1=0.86\n", " δ1=1.49\n", " δ1=0.59\n", " δ1=2.25\n", " δ1=1.75\n", " δ1=0.16\n", " δ1=2.14\n", " δ1=1.74\n", " δ1=1.82\n", " δ1=1.04\n", " δ1=0.20\n", " δ1=1.28\n", " δ1=0.25\n", " δ1=0.12\n", " δ1=0.63\n", " δ1=1.01\n", " δ1=2.13\n", " δ1=0.82\n", " δ1=0.48\n", " δ1=1.27\n", " δ1=1.15\n", " δ1=1.26\n", " δ1=0.34\n", " δ1=1.85\n", " δ1=1.87\n", " δ1=0.41\n", " δ1=2.01\n", " δ1=2.27\n", " δ1=2.17\n", " δ1=1.44\n", " δ1=1.64\n", " δ1=1.29\n", " δ1=0.93\n", " δ1=1.13\n", " δ1=1.00\n", " δ1=2.20\n", " δ1=0.27\n", " δ1=2.30\n", " δ1=1.70\n", " δ1=1.52\n", " δ1=1.90\n", " δ1=0.29\n", " δ1=1.33\n", " δ1=0.28\n", " δ1=1.84\n", " δ1=0.56\n", " δ1=2.18\n", " δ1=0.43\n", " δ1=0.69\n", " δ1=0.13\n", " δ1=1.45\n", " δ1=0.47\n", " δ1=0.85\n", " δ1=0.52\n", " δ1=1.24\n", " δ1=1.30\n", " δ1=1.38\n", " δ1=1.39\n", " δ1=0.71\n", " δ1=0.78\n", " δ1=0.11\n", " δ1=1.83\n", " δ1=0.97\n", " δ1=2.10\n", " δ1=1.93\n", " δ1=0.60\n", " δ1=1.95\n", " δ1=1.91\n", " δ1=0.33\n", " δ1=1.12\n", " δ1=0.23\n", " δ1=0.32\n", " δ1=0.44\n", " δ1=2.23\n", " δ1=0.70\n", " δ1=1.55\n", " δ1=1.99\n", " δ1=1.34\n", " δ1=0.99\n", " δ1=1.41\n", " δ1=2.12\n", " δ1=2.02\n", " δ1=0.92\n", " δ1=2.16\n", " δ1=1.62\n", " δ1=2.00\n", " δ1=1.79\n", " δ1=1.94\n", " δ1=2.06\n", " δ1=0.81\n", " δ1=0.94\n", " δ1=0.49\n", " δ1=1.11\n", " δ1=1.58\n", " δ1=1.36\n", " δ1=1.77\n", " δ1=1.60\n", " δ1=1.16\n", " δ1=2.24\n", " δ1=1.54\n", " δ1=0.15\n", " δ1=2.21\n", " δ1=0.21\n", " δ1=2.29\n", " δ1=1.76\n", " δ1=0.75\n", " δ1=1.09\n", " δ1=1.23\n", " δ1=2.08\n", " δ1=0.55\n", " δ1=0.76\n", " δ1=1.72\n", " δ1=1.56\n", " δ1=1.31\n", " δ1=0.30\n", " δ1=0.61\n", " δ1=0.89\n", " δ1=0.66\n", " δ1=1.46\n", " δ1=1.98\n", " δ1=0.53\n", " δ1=0.68\n", " δ1=0.37\n", " δ1=1.53\n", " δ1=0.35\n", " δ1=0.50\n", " δ1=0.19\n", " δ1=1.50\n", " δ1=1.43\n", " δ1=0.39\n", " δ1=0.46\n", " δ1=0.22\n", " δ1=0.57\n", " δ1=0.40\n", " δ1=0.42\n", " δ1=0.54\n", " δ1=2.22\n", " δ1=0.26\n", " δ1=1.59\n", " δ1=1.68\n", " δ1=1.81\n", " δ1=1.89\n", " δ1=0.96\n", " δ1=1.20\n", " δ1=1.37\n", " δ1=1.66\n", " δ1=1.78\n", " δ1=2.05\n", " δ1=1.42\n", " δ1=2.09\n", " δ1=0.64\n", " δ1=2.07\n", " δ1=1.07\n", " δ1=0.88\n", " δ1=1.18\n", " δ1=2.28\n", " δ1=1.08\n", " δ1=1.06\n", " δ1=1.03\n", " δ1=1.67\n", " δ1=1.48\n", " δ1=0.79\n", " δ1=1.97\n", " δ1=0.98\n", " δ1=0.83\n", " δ1=1.05\n", " δ1=1.71\n", " δ1=1.14\n", " δ1=1.51\n", " δ1=1.21\n", " δ1=0.17\n", " δ1=0.91\n", " δ1=1.35\n", " δ1=0.84\n", " δ1=1.61\n", " δ1=1.92\n", " δ1=2.04\n", " δ1=0.36\n", " δ1=0.74\n", " δ1=1.69\n", " δ1=0.73\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " \n", " \n", " Experience Range\n", " \n", " \n", " Pareto Curve\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " Negative Happiness Score\n", " \n", " \n", " -6100\n", " -6000\n", " -5900\n", " -5800\n", " -5700\n", " -5600\n", " -5500\n", " -5400\n", " -5300\n", " -5200\n", " -5100\n", " -5000\n", " -5920\n", " -5910\n", " -5900\n", " -5890\n", " -5880\n", " -5870\n", " -5860\n", " -5850\n", " -5840\n", " -5830\n", " -5820\n", " -5810\n", " -5800\n", " -5790\n", " -5780\n", " -5770\n", " -5760\n", " -5750\n", " -5740\n", " -5730\n", " -5720\n", " -5710\n", " -5700\n", " -5690\n", " -5680\n", " -5670\n", " -5660\n", " -5650\n", " -5640\n", " -5630\n", " -5620\n", " -5610\n", " -5600\n", " -5590\n", " -5580\n", " -5570\n", " -5560\n", " -5550\n", " -5540\n", " -5530\n", " -5520\n", " -5510\n", " -5500\n", " -5490\n", " -5480\n", " -5470\n", " -5460\n", " -5450\n", " -5440\n", " -5430\n", " -5420\n", " -5410\n", " -5400\n", " -5390\n", " -5380\n", " -5370\n", " -5360\n", " -5350\n", " -5340\n", " -5330\n", " -5320\n", " -5310\n", " -5300\n", " -5290\n", " -5280\n", " -5270\n", " -5260\n", " -5250\n", " -5240\n", " -5230\n", " -5220\n", " -5210\n", " -5200\n", " -6000\n", " -5800\n", " -5600\n", " -5400\n", " -5200\n", " -5920\n", " -5900\n", " -5880\n", " -5860\n", " -5840\n", " -5820\n", " -5800\n", " -5780\n", " -5760\n", " -5740\n", " -5720\n", " -5700\n", " -5680\n", " -5660\n", " -5640\n", " -5620\n", " -5600\n", " -5580\n", " -5560\n", " -5540\n", " -5520\n", " -5500\n", " -5480\n", " -5460\n", " -5440\n", " -5420\n", " -5400\n", " -5380\n", " -5360\n", " -5340\n", " -5320\n", " -5300\n", " -5280\n", " -5260\n", " -5240\n", " -5220\n", " -5200\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " δ1=1.86\n", " δ1=1.19\n", " δ1=0.90\n", " δ1=1.22\n", " δ1=0.62\n", " δ1=0.77\n", " δ1=1.63\n", " δ1=2.15\n", " δ1=0.67\n", " δ1=0.86\n", " δ1=1.49\n", " δ1=0.59\n", " δ1=2.25\n", " δ1=1.75\n", " δ1=0.16\n", " δ1=2.14\n", " δ1=1.74\n", " δ1=1.82\n", " δ1=1.04\n", " δ1=0.20\n", " δ1=1.28\n", " δ1=0.25\n", " δ1=0.12\n", " δ1=0.63\n", " δ1=1.01\n", " δ1=2.13\n", " δ1=0.82\n", " δ1=0.48\n", " δ1=1.27\n", " δ1=1.15\n", " δ1=1.26\n", " δ1=0.34\n", " δ1=1.85\n", " δ1=1.87\n", " δ1=0.41\n", " δ1=2.01\n", " δ1=2.27\n", " δ1=2.17\n", " δ1=1.44\n", " δ1=1.64\n", " δ1=1.29\n", " δ1=0.93\n", " δ1=1.13\n", " δ1=1.00\n", " δ1=2.20\n", " δ1=0.27\n", " δ1=2.30\n", " δ1=1.70\n", " δ1=1.52\n", " δ1=1.90\n", " δ1=0.29\n", " δ1=1.33\n", " δ1=0.28\n", " δ1=1.84\n", " δ1=0.56\n", " δ1=2.18\n", " δ1=0.43\n", " δ1=0.69\n", " δ1=0.13\n", " δ1=1.45\n", " δ1=0.47\n", " δ1=0.85\n", " δ1=0.52\n", " δ1=1.24\n", " δ1=1.30\n", " δ1=1.38\n", " δ1=1.39\n", " δ1=0.71\n", " δ1=0.78\n", " δ1=0.11\n", " δ1=1.83\n", " δ1=0.97\n", " δ1=2.10\n", " δ1=1.93\n", " δ1=0.60\n", " δ1=1.95\n", " δ1=1.91\n", " δ1=0.33\n", " δ1=1.12\n", " δ1=0.23\n", " δ1=0.32\n", " δ1=0.44\n", " δ1=2.23\n", " δ1=0.70\n", " δ1=1.55\n", " δ1=1.99\n", " δ1=1.34\n", " δ1=0.99\n", " δ1=1.41\n", " δ1=2.12\n", " δ1=2.02\n", " δ1=0.92\n", " δ1=2.16\n", " δ1=1.62\n", " δ1=2.00\n", " δ1=1.79\n", " δ1=1.94\n", " δ1=2.06\n", " δ1=0.81\n", " δ1=0.94\n", " δ1=0.49\n", " δ1=1.11\n", " δ1=1.58\n", " δ1=1.36\n", " δ1=1.77\n", " δ1=1.60\n", " δ1=1.16\n", " δ1=2.24\n", " δ1=1.54\n", " δ1=0.15\n", " δ1=2.21\n", " δ1=0.21\n", " δ1=2.29\n", " δ1=1.76\n", " δ1=0.75\n", " δ1=1.09\n", " δ1=1.23\n", " δ1=2.08\n", " δ1=0.55\n", " δ1=0.76\n", " δ1=1.72\n", " δ1=1.56\n", " δ1=1.31\n", " δ1=0.30\n", " δ1=0.61\n", " δ1=0.89\n", " δ1=0.66\n", " δ1=1.46\n", " δ1=1.98\n", " δ1=0.53\n", " δ1=0.68\n", " δ1=0.37\n", " δ1=1.53\n", " δ1=0.35\n", " δ1=0.50\n", " δ1=0.19\n", " δ1=1.50\n", " δ1=1.43\n", " δ1=0.39\n", " δ1=0.46\n", " δ1=0.22\n", " δ1=0.57\n", " δ1=0.40\n", " δ1=0.42\n", " δ1=0.54\n", " δ1=2.22\n", " δ1=0.26\n", " δ1=1.59\n", " δ1=1.68\n", " δ1=1.81\n", " δ1=1.89\n", " δ1=0.96\n", " δ1=1.20\n", " δ1=1.37\n", " δ1=1.66\n", " δ1=1.78\n", " δ1=2.05\n", " δ1=1.42\n", " δ1=2.09\n", " δ1=0.64\n", " δ1=2.07\n", " δ1=1.07\n", " δ1=0.88\n", " δ1=1.18\n", " δ1=2.28\n", " δ1=1.08\n", " δ1=1.06\n", " δ1=1.03\n", " δ1=1.67\n", " δ1=1.48\n", " δ1=0.79\n", " δ1=1.97\n", " δ1=0.98\n", " δ1=0.83\n", " δ1=1.05\n", " δ1=1.71\n", " δ1=1.14\n", " δ1=1.51\n", " δ1=1.21\n", " δ1=0.17\n", " δ1=0.91\n", " δ1=1.35\n", " δ1=0.84\n", " δ1=1.61\n", " δ1=1.92\n", " δ1=2.04\n", " δ1=0.36\n", " δ1=0.74\n", " δ1=1.69\n", " δ1=0.73\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " -6\n", " -5\n", " -4\n", " -3\n", " -2\n", " -1\n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " 7\n", " 8\n", " 9\n", " 10\n", " 11\n", " -5.0\n", " -4.8\n", " -4.6\n", " -4.4\n", " -4.2\n", " -4.0\n", " -3.8\n", " -3.6\n", " -3.4\n", " -3.2\n", " -3.0\n", " -2.8\n", " -2.6\n", " -2.4\n", " -2.2\n", " -2.0\n", " -1.8\n", " -1.6\n", " -1.4\n", " -1.2\n", " -1.0\n", " -0.8\n", " -0.6\n", " -0.4\n", " -0.2\n", " 0.0\n", " 0.2\n", " 0.4\n", " 0.6\n", " 0.8\n", " 1.0\n", " 1.2\n", " 1.4\n", " 1.6\n", " 1.8\n", " 2.0\n", " 2.2\n", " 2.4\n", " 2.6\n", " 2.8\n", " 3.0\n", " 3.2\n", " 3.4\n", " 3.6\n", " 3.8\n", " 4.0\n", " 4.2\n", " 4.4\n", " 4.6\n", " 4.8\n", " 5.0\n", " 5.2\n", " 5.4\n", " 5.6\n", " 5.8\n", " 6.0\n", " 6.2\n", " 6.4\n", " 6.6\n", " 6.8\n", " 7.0\n", " 7.2\n", " 7.4\n", " 7.6\n", " 7.8\n", " 8.0\n", " 8.2\n", " 8.4\n", " 8.6\n", " 8.8\n", " 9.0\n", " 9.2\n", " 9.4\n", " 9.6\n", " 9.8\n", " 10.0\n", " -5\n", " 0\n", " 5\n", " 10\n", " -5.0\n", " -4.5\n", " -4.0\n", " -3.5\n", " -3.0\n", " -2.5\n", " -2.0\n", " -1.5\n", " -1.0\n", " -0.5\n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " 3.0\n", " 3.5\n", " 4.0\n", " 4.5\n", " 5.0\n", " 5.5\n", " 6.0\n", " 6.5\n", " 7.0\n", " 7.5\n", " 8.0\n", " 8.5\n", " 9.0\n", " 9.5\n", " 10.0\n", " \n", " \n", " Experience Range\n", " \n", " \n", " Pareto Curve\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n" ], "text/plain": [ "Plot(...)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p1 = plot(layer(df, x=\"neg_objective\", y=\"range\", label=\"label\",\n", " Geom.smooth(method=:loess, smoothing=0.9),\n", " Geom.label(position=:dynamic, hide_overlaps=true)),\n", " layer(df, x=\"neg_objective\", y=\"range\", Geom.point,\n", " Theme(default_color=\"rgba(253,192,203,0.4)\", point_size=2px)),\n", " Guide.title(\"Pareto Curve\"), Guide.xlabel(\"Negative Happiness Score\"),\n", " Guide.ylabel(\"Experience Range\"),\n", " Coord.cartesian(xmax=maximum(df[:neg_objective])+20))\n", "\n", "# Store the svg plot\n", "# draw(SVG(\"./plots/pareto.svg\", 25cm, 15cm), p1)\n", "\n", "set_default_plot_size(25cm, 15cm)\n", "p1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the Pareto curve above, we notice:\n", "1. In our case, the scatter plot is more straightforward, because Pareto curve does not catch the \"stepwise function\" relationship.\n", "2. We can see the infeasible region at first is mostly controlled by the happiness score, but later experience range also significantly influences it.\n", "3. Suppose we fix all the parameters ($\\beta_1, \\dots, \\beta_5,$ etc.), one good $\\delta_1$ value is around $0.52$. It doesn't scarifies much of the student happiness, but gives a way better group experience range (comparing to use $\\delta_1=2$). Thus, instructors can use Pareto curve to help tuning the parameters.\n", "\n", "We have also recorded the computing time of each $\\delta_1$ value." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " δ1 (Tolerance Level)\n", " \n", " \n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 500\n", " 1000\n", " 1500\n", " 2000\n", " 2500\n", " \n", " \n", " Computing Time (Seconds)\n", " \n", " \n", " Computing Time v.s. Group Fairness\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " δ1 (Tolerance Level)\n", " \n", " \n", " -3.0\n", " -2.5\n", " -2.0\n", " -1.5\n", " -1.0\n", " -0.5\n", " 0.0\n", " 0.5\n", " 1.0\n", " 1.5\n", " 2.0\n", " 2.5\n", " 3.0\n", " 3.5\n", " 4.0\n", " 4.5\n", " 5.0\n", " 5.5\n", " -2.5\n", " -2.4\n", " -2.3\n", " -2.2\n", " -2.1\n", " -2.0\n", " -1.9\n", " -1.8\n", " -1.7\n", " -1.6\n", " -1.5\n", " -1.4\n", " -1.3\n", " -1.2\n", " -1.1\n", " -1.0\n", " -0.9\n", " -0.8\n", " -0.7\n", " -0.6\n", " -0.5\n", " -0.4\n", " -0.3\n", " -0.2\n", " -0.1\n", " 0.0\n", " 0.1\n", " 0.2\n", " 0.3\n", " 0.4\n", " 0.5\n", " 0.6\n", " 0.7\n", " 0.8\n", " 0.9\n", " 1.0\n", " 1.1\n", " 1.2\n", " 1.3\n", " 1.4\n", " 1.5\n", " 1.6\n", " 1.7\n", " 1.8\n", " 1.9\n", " 2.0\n", " 2.1\n", " 2.2\n", " 2.3\n", " 2.4\n", " 2.5\n", " 2.6\n", " 2.7\n", " 2.8\n", " 2.9\n", " 3.0\n", " 3.1\n", " 3.2\n", " 3.3\n", " 3.4\n", " 3.5\n", " 3.6\n", " 3.7\n", " 3.8\n", " 3.9\n", " 4.0\n", " 4.1\n", " 4.2\n", " 4.3\n", " 4.4\n", " 4.5\n", " 4.6\n", " 4.7\n", " 4.8\n", " 4.9\n", " 5.0\n", " -2.5\n", " 0.0\n", " 2.5\n", " 5.0\n", " -2.6\n", " -2.4\n", " -2.2\n", " -2.0\n", " -1.8\n", " -1.6\n", " -1.4\n", " -1.2\n", " -1.0\n", " -0.8\n", " -0.6\n", " -0.4\n", " -0.2\n", " 0.0\n", " 0.2\n", " 0.4\n", " 0.6\n", " 0.8\n", " 1.0\n", " 1.2\n", " 1.4\n", " 1.6\n", " 1.8\n", " 2.0\n", " 2.2\n", " 2.4\n", " 2.6\n", " 2.8\n", " 3.0\n", " 3.2\n", " 3.4\n", " 3.6\n", " 3.8\n", " 4.0\n", " 4.2\n", " 4.4\n", " 4.6\n", " 4.8\n", " 5.0\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " -3000\n", " -2500\n", " -2000\n", " -1500\n", " -1000\n", " -500\n", " 0\n", " 500\n", " 1000\n", " 1500\n", " 2000\n", " 2500\n", " 3000\n", " 3500\n", " 4000\n", " 4500\n", " 5000\n", " 5500\n", " -2500\n", " -2400\n", " -2300\n", " -2200\n", " -2100\n", " -2000\n", " -1900\n", " -1800\n", " -1700\n", " -1600\n", " -1500\n", " -1400\n", " -1300\n", " -1200\n", " -1100\n", " -1000\n", " -900\n", " -800\n", " -700\n", " -600\n", " -500\n", " -400\n", " -300\n", " -200\n", " -100\n", " 0\n", " 100\n", " 200\n", " 300\n", " 400\n", " 500\n", " 600\n", " 700\n", " 800\n", " 900\n", " 1000\n", " 1100\n", " 1200\n", " 1300\n", " 1400\n", " 1500\n", " 1600\n", " 1700\n", " 1800\n", " 1900\n", " 2000\n", " 2100\n", " 2200\n", " 2300\n", " 2400\n", " 2500\n", " 2600\n", " 2700\n", " 2800\n", " 2900\n", " 3000\n", " 3100\n", " 3200\n", " 3300\n", " 3400\n", " 3500\n", " 3600\n", " 3700\n", " 3800\n", " 3900\n", " 4000\n", " 4100\n", " 4200\n", " 4300\n", " 4400\n", " 4500\n", " 4600\n", " 4700\n", " 4800\n", " 4900\n", " 5000\n", " -2500\n", " 0\n", " 2500\n", " 5000\n", " -2600\n", " -2400\n", " -2200\n", " -2000\n", " -1800\n", " -1600\n", " -1400\n", " -1200\n", " -1000\n", " -800\n", " -600\n", " -400\n", " -200\n", " 0\n", " 200\n", " 400\n", " 600\n", " 800\n", " 1000\n", " 1200\n", " 1400\n", " 1600\n", " 1800\n", " 2000\n", " 2200\n", " 2400\n", " 2600\n", " 2800\n", " 3000\n", " 3200\n", " 3400\n", " 3600\n", " 3800\n", " 4000\n", " 4200\n", " 4400\n", " 4600\n", " 4800\n", " 5000\n", " \n", " \n", " Computing Time (Seconds)\n", " \n", " \n", " Computing Time v.s. Group Fairness\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n" ], "text/plain": [ "Plot(...)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p1 = plot(df, x=\"delta_1\", y=\"time\", Geom.point,\n", " Coord.cartesian(xflip=true),\n", " Guide.title(\"Computing Time v.s. Group Fairness\"),\n", " Guide.xlabel(\"δ1 (Tolerance Level)\"),\n", " Guide.ylabel(\"Computing Time (Seconds)\"))\n", "\n", "# Save the svg\n", "# draw(SVG(\"./plots/time.svg\", 25cm, 10cm), p1)\n", "\n", "set_default_plot_size(25cm, 10cm)\n", "p1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We notice:\n", "1. It takes much shorter time to solve the model when $\\delta_1$ is large (when it is slack). The computing time explodes exponentially when $\\delta_1 > 0.5$. It implies the constraint gets tighter and tighter, which matches our sensitiveness analysis of the \"Happiness v.s. Group Fairness\" plot.\n", "2. There is another trade-off between running time and tolerance level. Instructors can get an optimal solution pretty quickly if their tolerance of group fairness is not extremely low. If instructors really value the group experience equality, this model will take a very long time (limitation of our model). However, as we have discussed [here](#plot1), the group experience looks fair enough even when $\\delta_1 = 0.5$." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 5. Conclusion\n", "\n", "\n", "In this project, we have demonstrated how to use an optimization model to solve a real-life group assignment problem. We also illustrated the model design process through variable determination, data manipulation and constraint/objective transformation. We found the \"tricks\" we have seen in CS 524 comes very handy, such as logical constraints, relaxation, trade-off and weighting. \n", "\n", "In the trade-off analysis, we have shown there is indeed a trade-off relationship between student happiness and group experience fairness. Through constraint and objectives design, we made this model flexible such that instructors can control all the weights for objectives and each single factors. Also, instructors can add new parameters easily, such as student GPA, gender and ethnicity, to further diversify the group assignment.\n", "\n", "There are two major limitations in this project:\n", "1. We didn't actually optimize the group fairness. In other words, the group experience equality is controlled by parameter $\\delta_1$, which itself can be an optimization variable. Even though it is good enough to limit the group experience variance in a small range in most cases. We still think that there might be a clever way to modify the variance-based objective, so that we can really find the minimal various solution. It can be a follow-up problem for this project.\n", "2. All the data we are using in this project are [fake](#6.1-Fake-data-generation). Even though we have tried our best to generate those data look like real ones, we cannot guarantee they are representative enough.\n", "\n", "Finally, we do realize the importance of model interpretation. Interpreting the optimal solution really helped us see why it is optimal, and understand how model behaves in a higher level. With good interpretations, one can further improve the model by tuning parameters. On the other hand, interpretation tells us more information about the domain problem, which provides a way to design a more problem-specific algorithm." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## 6. Appendix\n", "\n", "### 6.1 Fake data generation\n", "\n", "The real data of CS506 student are confidential, so we have to generate simulation data manually. According to Professor Ben Liblit, there are 90 students and 29 available projects during the bidding phase. We will simulate the survey, and construct $\\mathcal{I}, \\mathcal{J}, F_w, F_1, F_2, \\dots, F_5$ below. " ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: using StatsBase.df in module Main conflicts with an existing identifier.\n" ] } ], "source": [ "using StatsBase, Gadfly, DataFrames, JLD\n", "\n", "STUDENT_NUM = 90\n", "PROJECT_NUM = 29;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Generate labels for students $\\left(\\mathcal{I}\\right)$ and projects$\\left(\\mathcal{J}\\right)$. We just use integer index here. Also, we uniform randomly assign project writers to projects $\\left(\\mathcal{F_w}\\right)$." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "students = [i for i in 1:STUDENT_NUM]\n", "projects = [j for j in 1:PROJECT_NUM]\n", "project_writers = shuffle(students)[1:PROJECT_NUM];" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- There are five levels of experience level, noted as $1,2,3,4,5$. One can argue the programming experience of CS506 students are normally distributed with mean $3$, and rounded into those five discrete values. However, the rounded normal is actually not normal, and the variance is hard to determine without any prior information. Therefore, it is reasonable to self-define a probability mass function and sample from it. Assume the probability of getting $1,2,3,4,5$ is $0.1, 0.2, 0.35, 0.25, 0.1$ respectively for both experience levels." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "# Random sampling from the PMF\n", "exp_1 = wsample([1,2,3,4,5], [0.1, 0.2, 0.25, 0.35, 0.1], STUDENT_NUM)\n", "exp_2 = wsample([1,2,3,4,5], [0.1, 0.2, 0.25, 0.35, 0.1], STUDENT_NUM);" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " Experience\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 5\n", " 10\n", " 15\n", " 20\n", " 25\n", " \n", " \n", " Count\n", " \n", " \n", " Cumulative Project Experience Level\n", " \n", "\n", "\n", " \n", " Experience\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " \n", " \n", " Count\n", " \n", " \n", " Single Project Experience Level\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " Experience\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 5\n", " 10\n", " 15\n", " 20\n", " 25\n", " \n", " \n", " Count\n", " \n", " \n", " Cumulative Project Experience Level\n", " \n", "\n", "\n", " \n", " Experience\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " 5\n", " 6\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " \n", " \n", " Count\n", " \n", " \n", " Single Project Experience Level\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/plain": [ "Compose.SVG(200.0mm, 80.0mm, IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=8481, maxsize=Inf, ptr=8482, mark=-1), nothing, \"img-6497f0fa\", 0, Compose.SVGPropertyFrame[], Dict{Type,Union{Compose.Property, Void}}(), DataStructures.OrderedDict{Compose.ClipPrimitive,String}(Compose.ClipPrimitive{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}(Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}[(117.958mm, 10.6117mm), (195.0mm, 10.6117mm), (195.0mm, 60.715mm), (117.958mm, 60.715mm)])=>\"img-6497f0fa-4\",Compose.ClipPrimitive{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}(Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}[(17.9583mm, 10.6117mm), (95.0mm, 10.6117mm), (95.0mm, 60.715mm), (17.9583mm, 60.715mm)])=>\"img-6497f0fa-18\"), Tuple{Compose.FormPrimitive,String}[], Set{AbstractString}(), true, false, nothing, true, \"img-6497f0fa-28\", false, 28, AbstractString[\"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\", \"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\"], Tuple{AbstractString,AbstractString}[(\"Snap.svg\", \"Snap\"), (\"Gadfly\", \"Gadfly\"), (\"Gadfly\", \"Gadfly\")], AbstractString[\"fig.select(\\\"#img-6497f0fa-5\\\")\\n .init_gadfly();\", \"fig.select(\\\"#img-6497f0fa-19\\\")\\n .init_gadfly();\"], false, :none)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Visualize the sampling, the distribution is reasonable\n", "df = DataFrame(exp_1 = exp_1, exp_2 = exp_2);\n", "p1 = plot(df, x=\"exp_1\", Geom.histogram(bincount=5), Guide.title(\"Single Project Experience Level\"),\n", " Guide.xlabel(\"Experience\"), Guide.ylabel(\"Count\"))\n", "p2 = plot(df, x=\"exp_2\", Geom.histogram(bincount=5), Guide.title(\"Cumulative Project Experience Level\"),\n", " Guide.xlabel(\"Experience\"), Guide.ylabel(\"Count\"))\n", "draw(SVG(20cm, 8cm), hstack(p1, p2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Each student can list and sort at most $8$ unique projects he/she wants to work on. All non-listed projects are treated equally.\n", "\n", " It is hard to simulate student's preference of projects, but we should not leave it random (some projects are indeed more popular than others). One heuristic is to assume there are three types of students in CS 506: mobile application fan, web development fan, and others. We assume among each group of students, they share the same interest (probability) of different projects. Therefore, we can define three PMF's then partition students and sample their preference.

\n", " \n", " - Samples are weighted, so the more interested projects are more likely to be sampled at the front of list.\n", " - We add $0$ as a project, and we stop sampling when $0$ is sampled in the current student's list.\n", " - The PMF definition is based on our subjective classification of 29 projects of CS506 Spring 2018, provided by Professor Liblit.\n", " - Suppose each group has one third of the students\n", " - Suppose the proposal writer would list his/her own project in his/her top3 most interested projects." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "# The first ticket is for project 0 (stop listing)\n", "pmf_app = [1, 2, 10, 8, 1, 10, 9, 7, 7, 9, 9, 9, 3, 3, 7, 10, 5, 2, 10, 5, 3, 2, 8, 4, 1, 1, 0, 0, 8, 7]\n", "pmf_web = [1, 5, 2, 5, 8, 2, 5, 1, 1, 1, 2, 3, 9, 9, 5, 2, 2, 9, 1, 9, 7, 10, 2, 10, 8, 9, 8, 8, 5, 3]\n", "pmf_others = vcat([0], [5 for i in 1:29]);\n", "\n", "# Sampling\n", "preference = Array{Int64, 2}(STUDENT_NUM, 8);\n", "\n", "# Students who like apps\n", "for i in 1:floor(Int, 1/3*STUDENT_NUM)\n", " temp_sample = wsample([i for i in 0:29], pmf_app, 8, replace=false)\n", " \n", " # check if 0 is in the sampling, if so make all the following samples 0\n", " for j in 1:8\n", " if temp_sample[j] == 0\n", " temp_sample[j:end] = 0\n", " break\n", " end\n", " end\n", " \n", " preference[i,:] = temp_sample\n", "end\n", "\n", "# Students who like webs\n", "for i in floor(Int, 1/3*STUDENT_NUM) + 1 : floor(Int, 2/3*STUDENT_NUM)\n", " temp_sample = wsample([i for i in 0:29], pmf_web, 8, replace=false)\n", " \n", " # check if 0 is in the sampling, if so make all the following samples 0\n", " for j in 1:8\n", " if temp_sample[j] == 0\n", " temp_sample[j:end] = 0\n", " break\n", " end\n", " end\n", " \n", " preference[i,:] = temp_sample\n", "end\n", "\n", "# Other students\n", "for i in floor(Int, 2/3*STUDENT_NUM) + 1 : STUDENT_NUM\n", " temp_sample = wsample([i for i in 0:29], pmf_web, 8, replace=false)\n", " \n", " # check if 0 is in the sampling, if so make all the following samples 0\n", " for j in 1:8\n", " if temp_sample[j] == 0\n", " temp_sample[j:end] = 0\n", " break\n", " end\n", " end\n", " \n", " preference[i,:] = temp_sample\n", "end\n", "\n", "# The generation is ordered, so we shuffle the rows\n", "preference = preference[shuffle(1:end), :];" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "# Assume proposal writer would put his/her own proposal in top 3\n", "for i in 1:STUDENT_NUM\n", " if(i in project_writers)\n", " index = findin(project_writers, i)[1]\n", " # Randomly choose rank 1/2/3\n", " rank = rand([1,2,3])\n", " # Check if his/her project is already choosen by him/her\n", " # If so, just swap it with rank\n", " original_rank = findin(index, preference[i,:])\n", " if(original_rank != 0)\n", " preference[i, original_rank] = preference[i, rank]\n", " end\n", " preference[i, rank] = index\n", " end\n", "end" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
x1x2x3x4x5x6x7x8
1194111526172521
2131227232211710
322325272411723
4261620271323144
5611314109119
61825231011135
" ], "text/plain": [ "6×8 DataFrames.DataFrame\n", "│ Row │ x1 │ x2 │ x3 │ x4 │ x5 │ x6 │ x7 │ x8 │\n", "├─────┼────┼────┼────┼────┼────┼────┼────┼────┤\n", "│ 1 │ 19 │ 4 │ 11 │ 15 │ 26 │ 17 │ 25 │ 21 │\n", "│ 2 │ 13 │ 12 │ 27 │ 23 │ 22 │ 11 │ 7 │ 10 │\n", "│ 3 │ 22 │ 3 │ 25 │ 27 │ 24 │ 1 │ 17 │ 23 │\n", "│ 4 │ 26 │ 16 │ 20 │ 27 │ 13 │ 23 │ 14 │ 4 │\n", "│ 5 │ 6 │ 11 │ 3 │ 14 │ 10 │ 9 │ 1 │ 19 │\n", "│ 6 │ 18 │ 25 │ 2 │ 3 │ 10 │ 11 │ 13 │ 5 │" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "head(DataFrame(preference))" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " Project Number\n", " \n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " \n", " \n", " \n", " app\n", " web\n", " others\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Group\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " 50\n", " 60\n", " \n", " \n", " Counts\n", " \n", " \n", " Prefered Project by Each Group\n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " Project Number\n", " \n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " \n", " \n", " \n", " app\n", " web\n", " others\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Group\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " 50\n", " 60\n", " \n", " \n", " Counts\n", " \n", " \n", " Prefered Project by Each Group\n", " \n", "\n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/plain": [ "Compose.SVG(250.0mm, 100.0mm, IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=14251, maxsize=Inf, ptr=14252, mark=-1), nothing, \"img-1aaa2a0a\", 0, Compose.SVGPropertyFrame[], Dict{Type,Union{Compose.Property, Void}}(Pair{Type,Union{Compose.Property, Void}}(Compose.Property{Compose.FillPrimitive}, nothing),Pair{Type,Union{Compose.Property, Void}}(Compose.Property{Compose.JSCallPrimitive}, nothing),Pair{Type,Union{Compose.Property, Void}}(Compose.Property{Compose.SVGClassPrimitive}, nothing)), DataStructures.OrderedDict{Compose.ClipPrimitive,String}(Compose.ClipPrimitive{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}(Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}[(17.9583mm, 10.6117mm), (233.593mm, 10.6117mm), (233.593mm, 80.715mm), (17.9583mm, 80.715mm)])=>\"img-1aaa2a0a-15\"), Tuple{Compose.FormPrimitive,String}[], Set{AbstractString}(), true, false, nothing, true, \"img-1aaa2a0a-133\", false, 133, AbstractString[\"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\"], Tuple{AbstractString,AbstractString}[(\"Snap.svg\", \"Snap\"), (\"Gadfly\", \"Gadfly\")], AbstractString[\"fig.select(\\\"#img-1aaa2a0a-4\\\")\\n .drag(function() {}, function() {}, function() {});\", \"fig.select(\\\"#img-1aaa2a0a-6\\\")\\n .data(\\\"color_class\\\", \\\"color_app\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-7\\\")\\n .data(\\\"color_class\\\", \\\"color_web\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-8\\\")\\n .data(\\\"color_class\\\", \\\"color_others\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-10\\\")\\n .data(\\\"color_class\\\", \\\"color_app\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-11\\\")\\n .data(\\\"color_class\\\", \\\"color_web\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-12\\\")\\n .data(\\\"color_class\\\", \\\"color_others\\\")\\n.click(Gadfly.colorkey_swatch_click)\\n;\", \"fig.select(\\\"#img-1aaa2a0a-16\\\")\\n .init_gadfly();\"], false, :none)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Visualize the sampling\n", "cut_1 = floor(Int, 1/3*STUDENT_NUM)\n", "cut_2 = floor(Int, 2/3*STUDENT_NUM)\n", "df = DataFrame(app = vcat([preference[r,:] for r in 1:cut_1]...),\n", " web = vcat([preference[r,:] for r in cut_1+1:cut_2]...),\n", " others = vcat([preference[r,:] for r in cut_2+1:STUDENT_NUM]...));\n", "df_2 = stack(df, [:app, :web, :others])\n", "p1 = plot(df_2, x=\"value\", color=\"variable\", Geom.histogram, Guide.title(\"Prefered Project by Each Group\"),\n", " Guide.xlabel(\"Project Number\"), Guide.ylabel(\"Counts\"), Guide.colorkey(title=\"Group\"))\n", "draw(SVG(25cm, 10cm), p1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see the sampling is not totally random. There are some popular projects among three groups, and there are projects only one group really likes. Although this sampling is not perfect, we will use it for the optimization problem.\n", "\n", "The next step is to transfer the mapping to the discussed [format](#happiness) of $F_3 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{I} \\Vert}\\left(\\beta_3\\right)$. Since $\\beta_3$ is a model parameter, we just define a function and use it to convert in the solution section." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "vectorize_f3 (generic function with 1 method)" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Convert preference from STUDENT_NUM*8 integer matrix to a STUDENT_NUM*PROJECT_NUM\n", "# binary matrix weighted by β3\n", "# β3 is a length 8 vector\n", "function vectorize_f3(preference, β3)\n", " project_rank = convert(Array{Int64,2}, zeros(STUDENT_NUM,PROJECT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:8\n", " # If the student did't fill the list full\n", " if preference[r,c] == 0\n", " break\n", " end\n", " project_rank[r,preference[r,c]] = β3[c]\n", " end\n", " end\n", " return project_rank\n", "end" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The size of f_3 is (90, 29), the first 6 rows of f_3 is:\n" ] }, { "data": { "text/html": [ "
x1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17x18x19x20x21x22x23x24x25x26x27x28x29
100070000006000503080100024000
200000020013780000000045000600
330700000000000002000081460500
400010000000042070006003008500
520600800347005000010000000000
606501000043020000800000070000
" ], "text/plain": [ "6×29 DataFrames.DataFrame\n", "│ Row │ x1 │ x2 │ x3 │ x4 │ x5 │ x6 │ x7 │ x8 │ x9 │ x10 │ x11 │ x12 │ x13 │\n", "├─────┼────┼────┼────┼────┼────┼────┼────┼────┼────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 7 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 6 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 2 │ 0 │ 0 │ 1 │ 3 │ 7 │ 8 │\n", "│ 3 │ 3 │ 0 │ 7 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 4 │\n", "│ 5 │ 2 │ 0 │ 6 │ 0 │ 0 │ 8 │ 0 │ 0 │ 3 │ 4 │ 7 │ 0 │ 0 │\n", "│ 6 │ 0 │ 6 │ 5 │ 0 │ 1 │ 0 │ 0 │ 0 │ 0 │ 4 │ 3 │ 0 │ 2 │\n", "\n", "│ Row │ x14 │ x15 │ x16 │ x17 │ x18 │ x19 │ x20 │ x21 │ x22 │ x23 │ x24 │ x25 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 5 │ 0 │ 3 │ 0 │ 8 │ 0 │ 1 │ 0 │ 0 │ 0 │ 2 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 4 │ 5 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 2 │ 0 │ 0 │ 0 │ 0 │ 8 │ 1 │ 4 │ 6 │\n", "│ 4 │ 2 │ 0 │ 7 │ 0 │ 0 │ 0 │ 6 │ 0 │ 0 │ 3 │ 0 │ 0 │\n", "│ 5 │ 5 │ 0 │ 0 │ 0 │ 0 │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 8 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 7 │\n", "\n", "│ Row │ x26 │ x27 │ x28 │ x29 │\n", "├─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 4 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 6 │ 0 │ 0 │\n", "│ 3 │ 0 │ 5 │ 0 │ 0 │\n", "│ 4 │ 8 │ 5 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Test the conversion\n", "test_β3 = [8,7,6,5,4,3,2,1]\n", "println(\"The size of f_3 is $(size(vectorize_f3(preference, test_β3))), the first 6 rows of f_3 is:\")\n", "head(DataFrame(vectorize_f3(preference, test_β3)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "- Finally, we can generate student's preference to each other. It is a social network generation problem, and can be very hard without prior information.\n", " \n", " This heuristic is not perfect, but we think it is good enough for this optimization modeling:\n", " 1. We assume three fifth of the students enroll CS506 without knowing anyone, and the others enroll in the form of circles of friends. \n", " 2. We assume we have a probability of $0.1, 0.2, 0.7$ to see friend size $7, 5, 3,$ and each circle of friends are disconnected from other circles.\n", " 3. For a student enrolled alone, he/she has a low probability to \"like\" or \"dislike\" any other students.\n", " 4. For a student enrolled with his/her friends, he/she has a high probability to \"like\" people from his/her friends, and a low probability to \"like\" or \"dislike\" anyone else.\n", " 5. Alone student's choice is not independent. If he/she is \"liked\" by someone else, then he/she has a higher probability to \"like\" (not \"dislike\") that person back. We assume students with friends chooses independently.\n", " 6. We assume each student would not \"dislike\" the one he/she \"likes\" or any of his/her friends." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "get_group_size (generic function with 1 method)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# If see one zero, then just fill all the left as zero\n", "# (this student didn't fill all of the list)\n", "function clean_zeros(sampled)\n", " for i in 1:length(sampled)\n", " if sampled[i] == 0\n", " sampled[i:end] = 0\n", " break\n", " end\n", " end\n", " return sampled\n", "end\n", "\n", "# Choose group size 7,5,3 with probability 0.1, 0.2, 0.7\n", "function get_group_size()\n", " flip = rand()\n", " if flip < 0.1\n", " return 7\n", " elseif flip < 0.3\n", " return 5\n", " else\n", " return 3\n", " end\n", "end" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "likes = convert.(Int, zeros(STUDENT_NUM, 3))\n", "dislikes = convert.(Int, zeros(STUDENT_NUM, 2))\n", "\n", "# Partition students and assign probabilities\n", "pmfs = zeros(STUDENT_NUM, STUDENT_NUM + 1)\n", "shuffled_students = shuffle([1:STUDENT_NUM]...)\n", "options = [0:STUDENT_NUM...]\n", "\n", "# Alone student's init pmf is [90, 1, 1, 1, 1, ....] for example\n", "alone_student_init_pmf = vcat([STUDENT_NUM], [1 for in in 1:STUDENT_NUM])\n", "\n", "# Each student has 1/6 probability to have someone he/she deosnt want to work with\n", "dislike_pmf = vcat([5 * STUDENT_NUM], [1 for in in 1:STUDENT_NUM]);\n", "\n", "no_need_friends = Set(sample([1:STUDENT_NUM...], floor(Int, 3 * STUDENT_NUM/5), replace=false))\n", "need_friends = setdiff(Set(1:STUDENT_NUM), no_need_friends);" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "36 students need friends! → 36 → 33 → 30 → 27 → 24 → 21 → 18 → 18 → 18 → 11 → 8 → 8 → 8 → 8 → 5 → 5 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0 → 0" ] } ], "source": [ "print(\"$(length(collect(need_friends))) students need friends!\")\n", "for i in shuffled_students\n", " if i in no_need_friends\n", " # Update student's pmf\n", " pmfs[i,:] = pmfs[i,:] + alone_student_init_pmf\n", " pmfs[i,i] = 0\n", " \n", " # Sample his preference\n", " like = wsample(options, pmfs[i,:], 3, replace=false)\n", " like = clean_zeros(like)\n", " \n", " # He cannot dislike the one he likes\n", " cur_dislike_pmf = copy(dislike_pmf)\n", " cur_dislike_pmf[filter(x->x!=0, like)] = 0\n", " dislike = wsample(options, cur_dislike_pmf, 2, replace=false)\n", " dislike = clean_zeros(dislike)\n", " \n", " # Update the liked student's pmf\n", " for l in like\n", " if l == 0\n", " break\n", " end\n", " # Gaurantee at least 1/2 probability to like back\n", " pmfs[l, i+1] = pmfs[l, i+1] + STUDENT_NUM\n", " end\n", " \n", " # Update the global preference matrix\n", " likes[i,:] = like\n", " dislikes[i,:] = dislike\n", " \n", " else\n", " print(\" → $(length(collect(need_friends)))\")\n", " # The other half of the students\n", " # Check if he has already been asigned of friends, if not, assign to him\n", " if i in need_friends\n", " setdiff!(need_friends, [i])\n", " g_size = get_group_size()\n", " \n", " # Randomly choose friends for him\n", " friends = vcat([i], sample(collect(need_friends),\n", " minimum([g_size - 1, length(collect(need_friends))]), replace=false))\n", " setdiff!(need_friends, friends)\n", " \n", " # They share the same \"liking\" pmf\n", " friend_pmf = copy(alone_student_init_pmf)\n", " friend_pmf[1] = 1\n", " friend_pmf[vcat([i], friends)+1] = STUDENT_NUM\n", " for f in friends\n", " pmfs[f,:] = friend_pmf\n", " end\n", " end\n", " \n", " friends = [j-1 for j in 2:STUDENT_NUM+1 if pmfs[i,j] > 2]\n", " \n", " # Sample preference for him\n", " like = wsample(options, pmfs[i,:], 3, replace=false)\n", " like = clean_zeros(like)\n", " \n", " # He cannot dislike the one he likes or his friends\n", " cur_dislike_pmf = copy(dislike_pmf)\n", " cur_dislike_pmf[filter(x->x!=0, like)] = 0\n", " cur_dislike_pmf[friends] = 0\n", " dislike = wsample(options, cur_dislike_pmf, 2, replace=false)\n", " dislike = clean_zeros(dislike)\n", " \n", " # Update the preference matrix\n", " likes[i,:] = like\n", " dislikes[i,:] = dislike\n", " end\n", "end" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " Number of dislikes\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 20\n", " 40\n", " 60\n", " 80\n", " \n", " \n", " Count\n", " \n", " \n", " Distribution of number of dislikes (max=2)\n", " \n", "\n", "\n", " \n", " Number of likes\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " 50\n", " \n", " \n", " Count\n", " \n", " \n", " Distribution of number of likes (max=3)\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " Number of dislikes\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 20\n", " 40\n", " 60\n", " 80\n", " \n", " \n", " Count\n", " \n", " \n", " Distribution of number of dislikes (max=2)\n", " \n", "\n", "\n", " \n", " Number of likes\n", " \n", " \n", " 0\n", " 1\n", " 2\n", " 3\n", " 4\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " 0\n", " 10\n", " 20\n", " 30\n", " 40\n", " 50\n", " \n", " \n", " Count\n", " \n", " \n", " Distribution of number of likes (max=3)\n", " \n", "\n", "\n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", "\n" ], "text/plain": [ "Compose.SVG(220.0mm, 80.0mm, IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=7662, maxsize=Inf, ptr=7663, mark=-1), nothing, \"img-9388c072\", 0, Compose.SVGPropertyFrame[], Dict{Type,Union{Compose.Property, Void}}(), DataStructures.OrderedDict{Compose.ClipPrimitive,String}(Compose.ClipPrimitive{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}(Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}[(127.958mm, 10.6117mm), (215.0mm, 10.6117mm), (215.0mm, 60.715mm), (127.958mm, 60.715mm)])=>\"img-9388c072-4\",Compose.ClipPrimitive{Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}}(Tuple{Measures.Length{:mm,Float64},Measures.Length{:mm,Float64}}[(17.9583mm, 10.6117mm), (105.0mm, 10.6117mm), (105.0mm, 60.715mm), (17.9583mm, 60.715mm)])=>\"img-9388c072-18\"), Tuple{Compose.FormPrimitive,String}[], Set{AbstractString}(), true, false, nothing, true, \"img-9388c072-28\", false, 28, AbstractString[\"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\", \"/Users/JayWong/.julia/v0.6/Gadfly/src/gadfly.js\"], Tuple{AbstractString,AbstractString}[(\"Snap.svg\", \"Snap\"), (\"Gadfly\", \"Gadfly\"), (\"Gadfly\", \"Gadfly\")], AbstractString[\"fig.select(\\\"#img-9388c072-5\\\")\\n .init_gadfly();\", \"fig.select(\\\"#img-9388c072-19\\\")\\n .init_gadfly();\"], false, :none)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Visualize the friends count distribution\n", "my_count = x -> 0 in x ? (length(x) - countmap(x)[0]) : length(x)\n", "like_num = []\n", "dislike_num = []\n", "for i in 1:STUDENT_NUM\n", " push!(like_num, my_count(likes[i,:]))\n", " push!(dislike_num, my_count(dislikes[i,:]))\n", "end\n", "\n", "df = DataFrame(like_num=like_num, dislike_num=dislike_num)\n", "p1 = plot(df, x=\"like_num\", Geom.histogram(bincount=4), Guide.title(\"Distribution of number of likes (max=3)\"),\n", " Guide.xlabel(\"Number of likes\"), Guide.ylabel(\"Count\"))\n", "p2 = plot(df, x=\"dislike_num\", Geom.histogram(bincount=3), Guide.title(\"Distribution of number of dislikes (max=2)\"),\n", " Guide.xlabel(\"Number of dislikes\"), Guide.ylabel(\"Count\"))\n", "draw(SVG(22cm, 8cm), hstack(p1, p2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also used `D3.js` to visualize the network we created. The interactive version is at [here](./plots/network.html). Each black node is a student, and they are randomly plotted. The gray links between students indicate there is one student put the other into his/her preferred teammate list. If the link looks darker, it means they the relation is bidirectional. Red links, however, means one student doesn't want to work with the linked student. The number of dislike relationship is much less than the like relationships.\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The distribution looks reasonable. Then we need to transfer student's partner preference into the [format](#f4) of $F_4 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{I} \\Vert}\\left(\\beta_4\\right), F_5 \\in M_{\\Vert \\mathcal{I} \\Vert \\times \\Vert \\mathcal{I} \\Vert}\\left(\\beta_5\\right)$. Since parameter $\\beta_4, \\beta_5$ may change, we just define the convert functions here." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "vectorize_f5 (generic function with 1 method)" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Convert preference from STUDENT_NUM*3 integer matrix to a STUDENT_NUM*STUDENT_NUM\n", "# binary matrix weighted by β4\n", "# β4 is a length 3 vector\n", "function vectorize_f4(likes, β4)\n", " student_likes = convert(Array{Int64,2}, zeros(STUDENT_NUM,STUDENT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:3\n", " # If the student did't fill the list full\n", " if likes[r,c] == 0\n", " break\n", " end\n", " student_likes[r, likes[r,c]] = β4[c]\n", " end\n", " end\n", " return student_likes\n", "end\n", "\n", "# Convert preference from STUDENT_NUM*2 integer matrix to a STUDENT_NUM*STUDENT_NUM\n", "# binary matrix weighted by β5\n", "# β5 is a length 2 vector\n", "function vectorize_f5(dislikes, β5)\n", " student_dislikes = convert(Array{Int64,2}, zeros(STUDENT_NUM,STUDENT_NUM))\n", " for r in 1:STUDENT_NUM\n", " for c in 1:2\n", " # If the student did't fill the list full\n", " if dislikes[r,c] == 0\n", " break\n", " end\n", " student_dislikes[r, dislikes[r,c]] = β5[c]\n", " end\n", " end\n", " return student_dislikes\n", "end" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The size of f_4 is (90, 90), the first 6 rows of f_3 is:\n" ] }, { "data": { "text/html": [ "
x1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17x18x19x20x21x22x23x24x25x26x27x28x29x30x31x32x33x34x35x36x37x38x39x40x41x42x43x44x45x46x47x48x49x50x51x52x53x54x55x56x57x58x59x60x61x62x63x64x65x66x67x68x69x70x71x72x73x74x75x76x77x78x79x80x81x82x83x84x85x86x87x88x89x90
1000000000000003000000000000000000000010000000000000000000000000000000200000000000000000000
2000000000000000000000000000030000000000000000000000000000000000000000000002000000000010000
3000000000000000000032000100000000000000000000000000000000000000000000000000000000000000000
4030000000000000000000000000020000000000000000000000000000000000000000000001000000000000000
5000000020000000000000000000000000000000000000000000000000000000100000000000003000000000000
6000000000100000000000000000000000000000000000000000000000000020000000000000003000000000000
" ], "text/plain": [ "6×90 DataFrames.DataFrame\n", "│ Row │ x1 │ x2 │ x3 │ x4 │ x5 │ x6 │ x7 │ x8 │ x9 │ x10 │ x11 │ x12 │ x13 │\n", "├─────┼────┼────┼────┼────┼────┼────┼────┼────┼────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 1 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x14 │ x15 │ x16 │ x17 │ x18 │ x19 │ x20 │ x21 │ x22 │ x23 │ x24 │ x25 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 3 │ 2 │ 0 │ 0 │ 0 │ 1 │\n", "│ 4 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x26 │ x27 │ x28 │ x29 │ x30 │ x31 │ x32 │ x33 │ x34 │ x35 │ x36 │ x37 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x38 │ x39 │ x40 │ x41 │ x42 │ x43 │ x44 │ x45 │ x46 │ x47 │ x48 │ x49 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x50 │ x51 │ x52 │ x53 │ x54 │ x55 │ x56 │ x57 │ x58 │ x59 │ x60 │ x61 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x62 │ x63 │ x64 │ x65 │ x66 │ x67 │ x68 │ x69 │ x70 │ x71 │ x72 │ x73 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 2 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x74 │ x75 │ x76 │ x77 │ x78 │ x79 │ x80 │ x81 │ x82 │ x83 │ x84 │ x85 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 0 │ 2 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "\n", "│ Row │ x86 │ x87 │ x88 │ x89 │ x90 │\n", "├─────┼─────┼─────┼─────┼─────┼─────┤\n", "│ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 2 │ 1 │ 0 │ 0 │ 0 │ 0 │\n", "│ 3 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 4 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │\n", "│ 6 │ 0 │ 0 │ 0 │ 0 │ 0 │" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Test the conversion\n", "test_β4 = [3, 2, 1]\n", "println(\"The size of f_4 is $(size(vectorize_f4(likes, test_β4))), the first 6 rows of f_3 is:\")\n", "head(DataFrame(vectorize_f4(likes, test_β4)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we want to store the generated data to make the modeling consistent. If you are one of the few readers who actually read appendix, Congratulations!! 🎉🎉 You should have a better understanding of this plot now." ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "#save(\"data.jld\", \"project_writers\", project_writers,\n", "# \"exp_1\", exp_1, \"exp_2\", exp_2,\n", "# \"preference\", preference, \"likes\", likes,\n", "# \"dislikes\", dislikes)" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.6.2", "language": "julia", "name": "julia-0.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.6.2" }, "toc": { "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": true, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 1 }