{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "c8f589b4-3595-4eb7-9dcd-129f9faaaac0",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"%load_ext itikz"
]
},
{
"cell_type": "markdown",
"id": "0ad51939-2f0a-4fb1-be30-fd146cfa9089",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"source": [
"(privacy)=\n",
"# Security and privacy in speech technology\n",
"\n",
"\n",
"***DISCLAIMER:*** *This document is meant to be an introduction to\n",
"questions in security and privacy in speech technology for engineering\n",
"students, such that they would understand the main problematic. In\n",
"particular, this is not a legal document. In real-life application of\n",
"technology and data collection, you must consult legal experts to\n",
"determine whether you follow the law. You are responsible.*\n",
"\n",
"The [right to privacy](https://en.wikipedia.org/wiki/Right_to_privacy)\n",
"is a widely accepted concept though its definition varies. It is however\n",
"clear that people tend to think that some things are \"theirs\", that they\n",
"have ownership of *things*, including information about themselves. A\n",
"possible definition of privacy would then be \"the absence of attention\n",
"from others\" and correspondingly security could be defined as the\n",
"protection of that which one owns, including material and immaterial\n",
"things. It however must be emphasised that there are no widely shared\n",
"and accepted definitions and in particular, the legal community has a\n",
"wide range of definitions depending on the context and field of\n",
"application.\n",
"\n",
"From the perspective of speech technology, security and privacy has two\n",
"principal aspects;\n",
"- Security and privacy of data related to speech signals and\n",
"- Protection against attacks which use speech signals as a tool.\n",
"\n",
"The latter aspect is mainly related to speaker identity; fraudsters can\n",
"for example synthesise speech which mimics (spoofs) a target person to\n",
"gain access to restricted systems, such as access to the bank account of\n",
"the target person. Such use cases fall mainly under the discussions\n",
"under [speaker recognition and\n",
"verification](Recognition/Speaker_Recognition_and_Verification.md), and not discussed\n",
"further here.\n",
"\n",
"Observe that in the isolated category of telephony (classical telephone\n",
"connections) privacy and security already have well-established ethical\n",
"standards as well as legislation. In typical jurisdictions, telephone\n",
"calls are private in the sense that only the \"intended\" participants can\n",
"listen to them and sometimes even recording them is restricted. Covert\n",
"listening is usually allowed only for the police and even for them only\n",
"in specially regulated situations, such as with a permission granted by\n",
"a court or judge.\n",
"\n",
"{cite}`edps2019techdispatch,nautsch2019gdpr` "
]
},
{
"cell_type": "markdown",
"id": "7d2a95c3-6cf9-4ae6-bb67-e3ebe62c0242",
"metadata": {
"tags": []
},
"source": [
"## System models\n",
"### All-human interaction\n",
"Speech is a tool for communication such that it is generally sensible to always discuss interactions between two agents, say, Alice and Bob. The interaction between them is the desired function such that the information exchanged there is explicitly permitted. By choosing to talk with each other, they both reveal information to the extent speech contains such information.\n",
"\n",
"#### Primary interaction"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d1fd3b9e-6836-4f9d-b74c-16604d73a13b",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Bob};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "24f70a5f-d71b-4f5d-a6d1-ffd2c447dded",
"metadata": {
"tags": []
},
"source": [
"Though Alice and Bob knowingly and intentionally interact, they might reveal private things. This is the classic \"*slip of the tongue*\".\n",
"{cite:p}`petronio2002boundaries`\n",
"\n",
"#### Secondary interactions\n",
"\n",
"A second-order question are third parties, who are not part of the main speech interaction. The pertinent question is the degree to which the third party is allowed to partake in an interaction. As a practical example, suppose Alice and Bob have a romantic dinner at a restaurant. To which extent is the waitress Eve allowed to interact with the discussion of Alice and Bob? Clearly Eve has some necessary tasks such that interaction is unavoidable. Will Alice and Bob, for example, pause their discussion when Eve approaches?"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "704c4fc4-9cff-4260-a915-996029d3cfe9",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir \n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Bob};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=115pt, minimum height = 40pt] at (1,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (1,2.5) (eve) {Eve};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (private);\n",
" \\draw[color=purple,->, line width=1pt] (private) to [bend left] (eve);\n",
" \n",
" \\node at (-2,0) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \\node at (-2,2.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "f8a2c427-96a5-4bee-a415-8004f0ea0017",
"metadata": {
"tags": []
},
"source": [
"Observe that we have here labelled Eve as a \"*restricted*\" and not as an \"*unauthorized*\" interactor. If access is unauthorized, then it is clear that Eve should not have any access to the speech interaction, which is generally straightforward to handle. The word restricted, on the other hand, implies that unimpeded access should not be granted, but that some access can be allowed. It is thus not question of \"*if*\" access should be granted but \"*how much?*\".\n",
"\n",
"#### Ownership and personal privacy\n",
"\n",
"Privacy is closely connected to ownership of immaterial property, that is, information. Such ownership can also be translated to the question of *who has the control* over some information? In terms of *personal privacy*, it is clear that it relates to information to which a single person can claim ownership. \n",
"\n",
"Speech is more complicated. Speech is a form of communication and thus relates to an interaction between two parties. Dialogues can also commonly lead to co-creation of meaning, where new information is generated through the dialogue in a form which none of the involved parties could have alone produced {cite:p}`gasiorek2018message`. None of the users can thus claim sole ownership of the information, but the ownership is shared. Currently we do not have the tools for handling such shared ownership.\n",
"\n",
"### Interactions which involve devices or services\n",
"#### Telecommunication"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "dc52d487-cfae-4ded-aff6-184594f0a7a1",
"metadata": {
"jupyter": {
"source_hidden": true
},
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[draw,fill=lightgray,minimum width=20pt,minimum height=50pt] at (2,0) (tx) {\\parbox{1.35cm}{Telecom.\\\\ service}};\n",
" \\node[ellipse,draw,minimum width=40pt] at (4,0) (bob) {Bob};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
"\\end{tikzpicture}\n",
"\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "690f5b1c-f547-4a39-b23f-7f912a1bb074",
"metadata": {
"tags": []
},
"source": [
"Talking over the phone and video conference involve transmission of speech through a telecommunication service. Here we consider scenarios where the telecommunication device does not include any advanced functionalities or artificial intelligence. Most countries have clearly definied rules that specify the situations when such communication can be eavesdropped. In most jurisdictions, only the police is allowed to intercept such traffic and only in specific situations."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "77046b01-d9fc-46dd-81ef-f55ff6ba8b0f",
"metadata": {
"jupyter": {
"source_hidden": true
},
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[draw,fill=lightgray,minimum width=20pt,minimum height=50pt] at (2,0) (tx) {\\parbox{1.35cm}{Telecom.\\\\ service}};\n",
" \\node[ellipse,draw,minimum width=40pt] at (4,0) (bob) {Bob};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\node[draw=gray,very thick,ellipse,minimum width=170pt, minimum height = 60pt] at (2,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (2,2.5) (eve) {Police};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (private);\n",
" \\draw[color=purple,->, line width=1pt] (private) to [bend left] (eve);\n",
" \n",
" \\node at (-2,0) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \\node at (-2,2.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "aaee2352-ce4e-4d7b-b2d9-3f16ca681c97",
"metadata": {
"tags": []
},
"source": [
"Such interception of private communication is interesting primarily from ethical and legal perspectives, but does not contain technological challenges related to the communication itself. The main technological challenges are related to forensics; \n",
"- What information can the police extract (e.g. speaker identity, emotions and health)?\n",
"- How can speakers (and service providers) protect themselves from unauthorized interception by, for example, stripping away information such as speaker identity, from the transmitted signal?\n",
"\n",
"\n",
"\n",
"#### Discussion in the presence of speech interfaces\n",
"A commonly occuring scenario is one where two or more users engage in a discussion such that there is one or more speech operated devices nearby. For example, a user could have their mobile phone or there could be a smart speaker nearby. "
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f49affdf-6e99-4335-9830-7ba06d71457d",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Bob};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=115pt, minimum height = 40pt] at (1,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (1,2.5) (eve) {Agent};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (private);\n",
" \\draw[color=purple,->, line width=1pt] (private) to [bend left] (eve);\n",
" \n",
" \\node at (-2,0) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \\node at (-2,2.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "eb2ec1cb-1a01-4037-9712-08443ef645d9",
"metadata": {
"tags": []
},
"source": [
"Often, one of the users will be the primary user of said devices (e.g. it is *their* phone), so the question is how the devices should relate to the other users. For example, suppose Alice has a smart speaker at home and Bob comes to visit. What would be the appropriate approach then for both Alice and the agent? Should Alice or the agent notify Bob of the presence of an agent? Or should the agent automatically detect the presence of Bob and change its behaviour (e.g. go to sleep)? \n",
"\n",
"We seem to lack both the cultural habits which dictate how to handle such situations, the legal tools which regulates such situations as well as the technical tools to manage multi-user access.\n",
"\n",
"\n",
"\n",
"#### Interaction with a speech interface\n",
"An interaction with a speech interface or agent is surprisingly free of problems as long as the agent is not connected to any outside entity. We can think of the agent as a local device. If nobody else has access to that device, then all the information remains in the user's direct control. Even if the agent exist in a remote cloud-service, if information remains strictly within the desired service, there are very little problems to consider. "
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "b6edeb72-98f1-4510-ae9b-8c100424c59f",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Agent};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
"\\end{tikzpicture}\n",
"\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "421f6461-d0ad-45d3-a003-a8926398588e",
"metadata": {
"tags": []
},
"source": [
"An exception is analysis, which the agent can perform, which is not related to the desired service, which can take abusive forms. For example, suppose the agent analyses the user's voice for health problems and identifies that the user has [Alzheimer's disease](https://en.wikipedia.org/wiki/Alzheimer%27s_disease). What should the agent then do with that information? Not doing anthing seems unethical - getting early access to medical services could greatly improve life quality. Informing the user, on the other hand, involves risks. How will the user react to the information? Is the user sufficiently psychologically stable to handle it? What if the analysis is incorrect and the agent thus causes suffering? It is also easy to think of further problematic scenarios. {cite:p}`konig2015automatic`\n",
"\n",
"\n",
"\n",
"#### Multi-user interaction with a speech interface\n",
"\n",
"An agent can be involved in an interaction with multiple users at the same time. "
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "c6994ac8-f934-496b-ab4d-a4486a0c457a",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
"\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice\\strut };\n",
" \\node[ellipse,draw,minimum width=40pt] at (1,1.5) (eve) {Agent\\strut };\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Bob\\strut };\n",
" \\node at (-2.2,.9) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \n",
" \\draw[->, line width=1pt] (alice) to (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (eve);\n",
" \\draw[->, line width=1pt] (eve) to (alice);\n",
"\n",
" \\draw[->, line width=1pt] (eve) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to (eve);\n",
" \n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "b8a8a3d1-6368-436e-8e7b-a5fe1105587b",
"metadata": {
"tags": []
},
"source": [
"This scenario differs from the single-user case in particular in the way the device can store information. To which extent should different combinations of users be permitted to have access to information from prior interactions? Quite obviously Alice should not have default, unrestricted access to Bob's prior interactions without Bob's permission. Where Alice and Bob have engaged in a joint discussion, the question of access becomes more complicated. It would seem natural that both can have access to information about their prior discussions. However, if Bob is in a discussion with Eve, then access to prior discussion between Bob and Alice should again be restricted. The rules governing access will thus be complicated, often non-obvious and they will have many exceptions.\n",
"\n",
"\n",
"\n",
"#### Interaction with a speech interface in the presence of others\n",
"\n",
"In the early days of mobile phones, a common *faux pas* was to speak loudly on the phone in public places such as on a bus or subway. It seems that it causes an uncomfortable feeling to people when they overhear private discussions. It can also be hard to ignore speech when you hear it.\n",
"Obviously, the reverse is also true, participants of a private discussion often feel uncomfortable if they fear that outsiders can hear their discussion."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "6b67cbf2-7f40-4d83-9d76-75e067565b0c",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=60pt] at (0,0) (alice) {Alice\\strut };\n",
" \\node[ellipse,draw,minimum width=60pt] at (2.6,0) (bob) {Agent\\strut };\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\node[draw=gray,very thick,ellipse,minimum width=140pt, minimum height = 45pt] at (1.3,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (1.3,2.5) (eve) {Eve};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (private);\n",
" \\draw[color=purple,->, line width=1pt] (private) to [bend left] (eve);\n",
" \n",
" \\node at (-2.2,0) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \\node at (-2.2,2.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "ea688ddc-0796-4a04-abef-a45c1b972ee6",
"metadata": {
"tags": []
},
"source": [
"The same applies to speech operated devices. Such interactions can be private, but even when they are not, the fact that they can be overheard is often uncomfortable to all parties. \n",
"\n",
"This is a problem when designing user interfaces to services. Speech interaction is often a natural way to user a service or device, but it is not practical in locations where other people can overhear private information and where it the sound is annoying to other people present.\n",
"\n",
"\n",
"#### Interaction with a speech interface connected to other services\n",
"\n",
"When interacting with a speech interface, users typically do it with a specific objective in mind. For example, suppose Alice wants to turn off the lights in the bedroom and says \"*Computer, lights off*\". To which extent is it permissible that that information is relayed to a cloud-service? If the local device is unable to or uncapable of deciphering the command, it can transmit it to the cloud. The cloud-service then obtains information from a very private part of Alice's life. "
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "1ccf5a24-c750-43c8-adae-ee3881dc9e76",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=60pt] at (0,0) (alice) {Alice\\strut };\n",
" \\node[ellipse,draw,minimum width=60pt] at (2.6,0) (bob) {Agent 1\\strut };\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\node[draw=gray,very thick,ellipse,minimum width=140pt, minimum height = 45pt] at (1.3,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (1.3,2.5) (eve) {Agent 2};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (private);\n",
" \\draw[color=purple,->, line width=1pt] (private) to [bend left] (eve);\n",
" \n",
" \\node at (-2.2,0) {\\parbox{2cm}{\\centering Private speech interaction}};\n",
" \\node at (-2.2,2.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "07f43cb1-604f-463e-b096-763166ab706b",
"metadata": {
"tags": []
},
"source": [
"Information obtained this way can be very useful for example, to advertisers. By analyzing the habits of users, they can serve more meaningful advertisements. Arguably, by better targeting of users, advertisement can more effective, which could *potentially* reduce the need for advertisement. It is however questionable whether advertisers ever would have incentives to *reduce* the amount of advertisement. \n",
"Still, some people are creeped out by \"*overly fitting*\" advertisement. \n",
"\n",
"There are however plenty of other scenarios which are more potent sources of danger. What if insurance agencies analyze users life patterns and increase payments for at-risk users such as substance abusers? Some smart devices already today can call the emergency services if they recognize cries of help or other obvious signs of distress. What are the moral dilemmas of that?\n",
"\n",
"### Multi-user and multi-device scenarios\n",
"Things get even more complicated when multipled users and/or users co-exist in the same space. Consider, for example, an open office with two users simulatenously engaged in independent video conferences. "
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "192fbff8-9bcc-4a4b-9423-ba2a61d8239f",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
"\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,0) (alice) {Alice\\strut };\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,0) (bob) {Bob\\strut };\n",
" \\node at (-2.2,0) {\\parbox{2cm}{\\centering Private speech interaction 2}};\n",
" \n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\node[ellipse,draw,minimum width=40pt] at (0,2) (alice2) {Adam\\strut };\n",
" \\node[ellipse,draw,minimum width=40pt] at (2,2) (bob2) {Beata\\strut };\n",
" \\node at (-2.2,2) {\\parbox{2cm}{\\centering Private speech interaction 1}};\n",
" \n",
" \\draw[->, line width=1pt] (alice2) to [bend left] (bob2);\n",
" \\draw[->, line width=1pt] (bob2) to [bend left] (alice2);\n",
"\n",
" \\draw[color=purple,<->, line width=1pt] (alice) to (alice2);\n",
" \\draw[color=purple,<->, line width=1pt] (bob) to (alice2);\n",
" \\draw[color=purple,<->, line width=1pt] (alice) to (bob2);\n",
" \\draw[color=purple,<->, line width=1pt] (bob) to (bob2);\n",
" \n",
" \\node at (3.5,1) {\\parbox{2cm}{\\centering Leaked interactions}}; \n",
" \n",
"\\end{tikzpicture}\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "028f7e76-fd03-4c15-adc3-ce1d4a8b0ae1",
"metadata": {},
"source": [
"## Information contained in speech signals\n",
"\n",
"Speech signals contain a wide variety of information which are often\n",
"potentially private or even sensitive and it is difficult or impossible\n",
"to list all categories of potential information. However, information\n",
"which speech at least contains includes for example;\n",
"\n",
"- The linguistic (text) content of a speech signal can contain almost\n",
" anything\n",
" - If you reveal private information about yourself in a\n",
" conversation, then clearly that information is contained in a\n",
" transcription of the conversation.\n",
" - Word-choices and manners of speaking [can reveal things about the\n",
" speaker](https://www.theguardian.com/lifeandstyle/2009/mar/08/language-voice), without the speaker realizing it himself or herself.\n",
"- Para-linguistic (i.e. non-linguistic viz. non-text) content has a\n",
" wide variety of information:\n",
" - Speaker identity\n",
" - Physical traits, including gender, body size and age\n",
" - Emotional state (happy, sad, angry, excited etc.)\n",
" - Speaking style (public, intimate, theatrical etc.)\n",
" - State of health, such as flu, mental health and diseases like\n",
" [Alzheimer's](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4876915/),\n",
" but also including tiredness and intoxication.\n",
" - Association and affiliation with reference groups, including\n",
" groups of gender-identity, ethnicity, culture background,\n",
" geographic background, nationality, political and religious\n",
" affiliations etc.\n",
"- In interaction with other speakers:\n",
" - Proof that you have met with the other speaker\n",
" - Level of familiarity, intimacy and\n",
" [trust](https://arxiv.org/abs/2007.15711).\n",
" - Power structures (leader/follower/partner)\n",
" - Family, romantic and other relationships\n",
"\n",
"In other words, speech contains or can contain just about all types of\n",
"private and sensitive information you could imagine. As speech is a tool\n",
"for communication, this is not surprising; anything we can communicate\n",
"about, can be spoken. Conversely, if we find that (and we do find that)\n",
"privacy is important, then speech is among the most important signals to\n",
"protect.\n",
"\n",
"\n",
"## Types of privacy\n",
"\n",
"In a more general scope than just speech, privacy can be categorized\n",
"into seven types: {cite:p}`finn2013seven`\n",
"\n",
"- **Privacy of the person**, which refers to the privacy with respect\n",
" to our physical body as well as any information about our body such\n",
" as fingerprints, voice characteristics and medical history.\n",
"- **Privacy of behaviour and action**, refers to privacy with respect\n",
" to what we do; for example, nobody needs to know what I do within\n",
" the confine of my home.\n",
"- **Privacy of communication**, is particularly important in speech\n",
" communication and refers to privacy of the content and meta-content\n",
" of communication. That is, it is not only about the literal content\n",
" of a communication, but also about the style of communication and\n",
" also about the fact that communication has happened.\n",
"- **Privacy of data, including images and sound**, which extends the\n",
" privacy of possessions to immaterial things. In particular, this\n",
" addresses privacy with regard to sharing information about a third\n",
" person.\n",
"- **Privacy of thoughts and** **feelings**, is paraphrasing, the\n",
" privacy of thinking. We have the right to share our thoughts and\n",
" emotions with whom we like, or to choose not to share our feelings\n",
" with anyone. This does not mean that people would have to listen to\n",
" you, but that your are allowed to offer them your thoughts.\n",
"- **Privacy of location and space**, has become increasingly\n",
" important, since so many of our mobile devices has the ability to\n",
" track your location. However, this type of privacy covers both your\n",
" location and location history (as in location tracking).\n",
"- **Privacy of association**, refers to the privacy of whether you\n",
" belong or to which extent you otherwise associate yourself to a\n",
" particular group (religious, political, ethnic, gender identity,\n",
" professional, interest groups etc.).\n",
"\n",
"Note that this list does not make any claims with respect to *rights* to\n",
"these types of privacy, but that privacy-issues can be often be split\n",
"into these sub-topics. Whether someone has a right to privacy is a\n",
"society-level decision and political choice, where psychological and\n",
"cultural aspects play a big role.\n",
"\n",
"\n",
"## Threat and attack scenarios\n",
"\n",
"Threats to privacy in speech communication can almost always be defined\n",
"as covert extraction of information as well as storage, processing and\n",
"usage of that information in ways of which the speaker is not aware,\n",
"and/or with which the speaker does not agree. Variability in scenarios\n",
"is then almost entirely due to the type of information involved as well\n",
"as the stakeholders. In particular,\n",
"\n",
"- Companies can extract information from private users for unethical\n",
" advantage\n",
" - Insurance policies and mortgages can be denied based on\n",
" covertly extracted information\n",
" - The price of services can be increased for vulnerable groups\n",
" - Access to information (search results) can be restricted and\n",
" information can be targeted (advertisement) to covertly\n",
" influence users for unethical advantage. Such behaviour is often\n",
" advertised under the pretence of customizing services to the\n",
" users preferences, but without giving the user any tools for\n",
" choosing how services are customized.\n",
"- State operators can use surveillance and access restrictions on\n",
" their own citizens\n",
" - Authoritative regimes can track and eavesdrop the political\n",
" opposition, dissent and other groups such as religious, ethnic\n",
" and sexual orientations.\n",
" - Also legitimate uses by police for surveillance in criminal\n",
" investigations\n",
"- State operators can use surveillance on foreign citizens\n",
" - Spies can use speech technology for (remote) eavesdropping and\n",
" information extraction\n",
"- Criminals can steal information and use it for their advantage\n",
" - Identity theft\n",
" - Paparazzi's can steal private information of famous people\n",
" - Private information can potentially be used for extortion\n",
" - Explicit content could be sold as entertainment\n",
"- Private persons can covertly use speech technology for eavesdropping\n",
" and extracting information of other persons\n",
" - For example, the command history of smart speakers can give\n",
" access to past commands of all users, also when speech commands\n",
" have been made in private.\n",
"\n",
"\n",
"## Privacy and security scandals\n",
"\n",
"Most of the threats and attack scenarios are not familiar to the common\n",
"public and some of them might be too abstract to be relevant to average\n",
"users. Typically, we can hypothesize that scenarios which do not touch\n",
"directly on the life of an individual, probably do not get much\n",
"attention in the media. Topics in privacy and security which however\n",
"have received attention in the public media include:\n",
"\n",
"- [Amazon workers are listening to what you tell\n",
" Alexa](https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio)\n",
" (Bloomberg, 2019). Later it was revealed that Google, Apple and\n",
" others are doing the same.\n",
"\n",
"- [Amazon Sends 1,700 Alexa Voice Recordings to a Random\n",
" Person](https://threatpost.com/amazon-1700-alexa-voice-recordings/140201/)\n",
" (Threatpost, 2018).\n",
"\n",
"- [Amazon's Alexa recorded private conversation and sent it to random\n",
" contact](https://www.theguardian.com/technology/2018/may/24/amazon-alexa-recorded-conversation)\n",
" (The Guardian, 2018).\n",
"\n",
"- [Fraudsters Used AI to Mimic CEO’s Voice in Unusual Cybercrime\n",
" Case](https://www.wsj.com/articles/fraudsters-use-ai-to-mimic-ceos-voice-in-unusual-cybercrime-case-11567157402)\n",
" (The Wall Street Journal, 2019).\n",
"\n",
"Note that the fact that many of the above examples are related to\n",
"Amazon/Alexa is probably more coincidence than an indication that Alexa\n",
"would treat privacy differently than its competitors.\n",
"\n",
"\n",
"## Approaches for safeguarding privacy in and improving usability of speech technology\n",
"\n",
"### Basic design concepts\n",
"\n",
"At least from the European perspective, the following design concepts\n",
"are seen as basis of good design for privacy. In fact, they are mandated\n",
"by the General Data Protection Regulations of the European Union.\n",
"\n",
"- *Privacy by default*; systems should always default to the least\n",
" invasive configuration. Additional services, which are more\n",
" invasive, can be chosen (opt-in) if the user so chooses.\n",
"- *Privacy by design*; privacy should not be an after-thought but\n",
" systems should be designed for privacy. The overall systems\n",
" structure should be chosen such that it supports privacy.\n",
"- *Data minimization*; data can be extracted from users only the\n",
" extent explicitly required by the system. For example, if you order\n",
" a pizza through a web-portal, you need to tell which pizza you want\n",
" and where it should be delivered. Otherwise you cannot receive the\n",
" service. In other words, all services require some level of\n",
" information transfer, but the services cannot ask about information\n",
" which is irrelevant to the service. For example, the pizza service\n",
" cannot ask you about your gender-identity. Data minimization cab\n",
" also be interpreted as *data ecology*, which would underline the\n",
" fact that \"more data\" usually means also a higher consumption of\n",
" energy and other resources. In fact, data could (or should) be\n",
" treated as a natural resource itself. This line of argumentation\n",
" connects privacy with the [UN sustainability\n",
" goals](https://www.un.org/development/desa/disabilities/envision2030.html). \n",
" An important aspect of data minimization is that information should\n",
" stored only as long as it is necessary for the service.\n",
"- *Informed and meaningful consent*; when the user chooses a service,\n",
" he should receive information about its implications to privacy in\n",
" an understandable and accessible way, and the service provider\n",
" should receive the users consent without any form of coercion.\n",
"\n",
"Typically it is reasonable that service providers have access to\n",
"aggregated data such as ensemble averages, but not to information about\n",
"individuals. For example, in a hypothetical case, a smart speaker\n",
"operator could receive the information that 55% of users are male, but\n",
"would not get the gender of any individual user.\n",
"\n",
"### Definitions\n",
"\n",
"A central problematic in privacy with speech signals is the concept of\n",
"\"uniquely identifiable\". Legal frameworks such as the GDPR state that\n",
"private information is such data where individual users are \"uniquely\n",
"identifiable\", but there is no accurate definition of what it really\n",
"means. If your partner recognizes your \"Hello\" on the phone, it means\n",
"that for her, your \"Hello\" is uniquely identifiable. However, if you\n",
"give 10.000 speech samples to your partner, one of which is your\n",
"\"Hello\", then there's a significant likelihood that your partner would\n",
"not find your \"Hello\" from the pile. An unanswered question is thus,\n",
"\"What is the size of the group where a user should be uniquely\n",
"identifiable?\".\n",
"\n",
"A more detailed aspect is that of significance. The speaker recognition\n",
"approach is to find the most likely speaker, out of the reference group\n",
"of size *N*, whereas speaker verification tries to determine whether we,\n",
"within some confidence intervals, can be sure that you are who you claim\n",
"you are. In engineering terms, this means that we want to find the\n",
"speaker with the highest likelihood, but with a sufficient margin to all\n",
"other speakers. In the opposite direction, we can also use a lower\n",
"threshold; we could say that statistically significant correlation\n",
"already exposes the users privacy. For example, if we find that the\n",
"speaker is either you or your father/mother, then we have a significant\n",
"statistical correlation, but your are not uniquely identified.\n",
"\n",
"A further consideration is that of adjoining data; Suppose there is a\n",
"recording of a speaker A, and that you happen to know a speaker A very\n",
"well. Then it will be easy for you to recognize the voice of A in that\n",
"recording. That is, you have a lot of experience (stored data) about how\n",
"A sounds, therefore it is easy for you to identify A. Does that mean\n",
"that A is uniquely identifiable in that recording? After all, A would\n",
"not be identifiable if you did not know A (= if you would not have\n",
"prior, stored data about A). \n",
"A slight variation of the above case is a recording of a speaker B,\n",
"where B is relatively famous public person, such that there are readily\n",
"available sound samples of his voice on-line. Does that make the\n",
"recording of B uniquely identifiable? Or if there is a recording of a\n",
"currently non-famous person C, who later becomes famous. Does that\n",
"change the status of the recording of C to uniquely identifiable?\n",
"\n",
"Today, this question remains unanswered and we have no commonly agreed\n",
"interpretation of what \"uniquely identifiable\" really means. What level\n",
"of statistical confidence is assumed? What level of adjoining data is\n",
"assumed (in terms of GDPR probably: any and all data which exists)? Can\n",
"it change over time if new information becomes public (probably: yes)?\n",
"Can it change over time if new technologies are developed (probably:\n",
"yes)?"
]
},
{
"cell_type": "markdown",
"id": "2e7de610-355a-46a2-ad7a-af5f12167eee",
"metadata": {},
"source": [
"### Local/edge processing\n",
"\n",
"Privacy is an issue only if some other party has access to data about you. Data which resides on a device which is in your control is therefore relatively safe, assuming that no outsider has access to that device. If data is sent to a cloud server then there are more entities which could potentially have access to your data. Therefore all storage and processing which can be done on your local device is usually by design more private than any cloud server. Typically, a cloud server would only provide software updates (downlink), but no data would be sent in the other direction (uplink). {cite:p}`shi2016edge`"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "073e0c4c-7dc9-4b9c-9d9b-ae087f57ba85",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt,minimum height=35pt] at (-.5,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=35pt] at (2,0) (bob) {\\parbox{1cm}{\\centering Local device}};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.8,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (5.5,0) (eve) {\\parbox{1cm}{\\centering Cloud servce}};\n",
" %\\draw[color=purple,->, dashed, line width=1pt] (eve) to node[above]{Model update} [left] (bob);\n",
"\\path[color=purple,very thick,dashed,<-,every node/.style={font=\\small}] \n",
" (bob) edge [] node[yshift=.5cm] {\\parbox{1cm}{model update}} (eve);\n",
" %\\draw[color=purple,->, line width=1pt] (bob) to [bend left] (eve);\n",
" \n",
" \\node at (.8,1.5) {\\parbox{3cm}{\\centering Private speech interaction}};\n",
" \\node at (5.5,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "06f8aa63-6955-4640-ad96-ed6291d49cb2",
"metadata": {},
"source": [
"Observe that this does not protect you from other local users. For\n",
"example, if multiple persons are using one smart speaker at home, then\n",
"the other users could have access to information about you through that\n",
"device and any connected other devices.\n",
"\n",
"Central limitations of edge processing are\n",
"\n",
"- Many services require outside access; say if you ask your phone\n",
" \"What's the weather tomorrow?\", the phone cannot know that by\n",
" itself, but has to retrieve the information from a cloud server. The\n",
" essential content of your speech is therefore relayed to the cloud\n",
" and you don't have much benefit from edge processing.\n",
"- Improving voice operated services requires a lot of data. Moreover,\n",
" data which reflects features of actual users is much better than any\n",
" simulations. Service providers thus argue that they need to collect\n",
" data from users to provide high-quality services, and local\n",
" processing could prevent the services providers from getting that\n",
" data.\n",
"- Edge devices would use their full capacity only when they pick up\n",
" speech in their microphone, which means that most of the time, edge\n",
" devices would lie dormant, waiting for speech commands. This is a\n",
" wasteful use of resources; a cloud server can better balance the\n",
" load because with a large number of users, the resource requirements\n",
" would likely be more stable."
]
},
{
"cell_type": "markdown",
"id": "d41bce5f-0cc2-459c-8ff2-931f665767dc",
"metadata": {},
"source": [
"### Anonymization, pseudonymization and disentanglement\n",
"\n",
"A central issue with voice communication is that, in addition to the intended message, it also contains so much other information. For example, if you want to order a pizza delivery, the service provider needs to know only the content of the order, destination where the order should be delivered and how it is paid. The provider does not need to know, for example, your state of health or your cultural affiliation. *Anonymization* refers to methods which try to strip away such private and extra information such that only the intended message remains. With *pseudonymization* we refer to similar methods, where private information is replaced by some other information. For example, we could replace personal identifying information of a user *Alice* with an avatar-identity *Adam*. The process and methodology of separating the different streams of information is known as *disentanglement*."
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "d349485f-48d1-490b-9ef2-fb9e047de5f3",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt,minimum height=35pt] at (-.5,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=35pt] at (2,0) (bob) {\\parbox{1cm}{\\centering Local device}};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.8,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (6,0) (eve) {\\parbox{1cm}{\\centering Cloud servce}};\n",
" \\draw[color=purple,dashed,->, line width=1pt] (eve) to [bend left] (bob);\n",
" \\draw[color=purple,dashed,->, line width=1pt] (bob) to [bend left] (eve);\n",
" \n",
" \\node at (.8,1.5) {\\parbox{3cm}{\\centering Private speech interaction}};\n",
" \\node at (4.1,0) {\\parbox{2cm}{\\centering Reduced data}};\n",
" \\node at (6,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "4e32ed9d-62cf-4500-bf97-dce78994d3e3",
"metadata": {},
"source": [
"An issue with current methodology is that there are no theoretical guarantees of anonymity. That is, we can try to deduce private information from the anonymized data stream, and if we fail, we can say that the anonymization has succeeded with respect our attempts to break it. However, we have no guarantee that some other more advanced method for breaking the anonynization would not succeed. "
]
},
{
"cell_type": "markdown",
"id": "f700f636-1f9c-4370-9789-8ee72fe95930",
"metadata": {},
"source": [
"### Differential privacy\n",
"\n",
"Even when operating with aggregate data, like the mean user age, it is\n",
"still possible to extract private information in some scenarios. For\n",
"example, if we know the mean user age and the number of users at a time\n",
"$t$, and we also know that the age of user $X$ was added to the mean at\n",
"time $t+1$, as well as the mean user age at $t+1$, then we can deduce\n",
"the age of user $X$ with basic algebra. As a safeguard against such\n",
"differential attacks, to provide [differential\n",
"privacy](https://en.wikipedia.org/wiki/Differential_privacy), it is\n",
"possible to add noise to any data transfers. Individual data points are\n",
"then obfuscated and cannot be exactly recovered. However, the ensemble\n",
"average can still be deduced if the distribution of the added noise is\n",
"known."
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "84701c05-f898-4bc0-b591-7c662f1a47aa",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=50pt] at (0,0) (alice) {Alice\\strut };\n",
" \\node[ellipse,draw,minimum width=50pt] at (2.6,0) (bob) {Agent 1\\strut };\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
"\n",
" \\node[draw=gray,very thick,ellipse,minimum width=140pt, minimum height = 45pt] at (1.35,0) (private) {};\n",
"\n",
" \\node[circle,draw,style={font=\\small}] at (5,0) (diffplus) {+};\n",
"\n",
" \\node[ellipse,draw,minimum width=50pt] at (7,0) (eve) {Agent 2\\strut};\n",
" \\draw[color=purple,->, line width=1pt] (bob) to [left] (diffplus);\n",
" \\draw[color=purple,->, dashed,line width=1pt] (diffplus) to [left] (eve);\n",
" %\\path[color=purple,very thick,dashed,<-,every node/.style={font=\\small}] (bob) edge [] node[yshift=.5cm] {\\parbox{1cm}{model update}} (eve);\n",
" %\\draw[color=purple,->, line width=1pt] (bob) to [bend left] (eve);\n",
" \n",
" \\node at (1.35,1.5) {\\parbox{3cm}{\\centering Private speech interaction}};\n",
" \\node at (7,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
" \\node at (5,1.5) (noise) {\\parbox{2cm}{\\centering Noise}};\n",
" \\draw[color=purple,->, line width=1pt] (noise) to [left] (diffplus);\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "406edf60-7174-4d73-adbc-3af04016b845",
"metadata": {},
"source": [
"The required compromise here is that the level of privacy corresponds to\n",
"amount of noise, which is inversely proportional to the accuracy of the\n",
"ensemble mean. That is, if the amount of noise is large, then we need a\n",
"huge number of users to determine an accurate ensemble average. On the\n",
"other hand, if the amount of noise is small, then we can get a fair\n",
"guess of an individual data point, but also the ensemble average is\n",
"accurate."
]
},
{
"cell_type": "markdown",
"id": "69dea416-ce85-4075-97af-b0c8c1f41fbe",
"metadata": {},
"source": [
"### Federated learning\n",
"\n",
"To enable machine learning in the cloud without the need to provide access to private data, we can use [federated learning](https://en.wikipedia.org/wiki/Federated_learning), where private data remains on the local device, but only model updates are sent to the cloud. Clearly this is approach has better privacy than one where all private data is sent to the cloud. At the same time, by combining information from a large number of local devices in the cloud, the machine learning models can be optimized to be highly accurate. However, currently we do not yet have clear understanding of the extent of privacy with this type of methods; some data is sent to the cloud, but can some private data still be traced back to the user?"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "76b949c7-e047-46ba-8f81-3aee7a1630e2",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[draw=gray,dotted,fill=white,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (1.2,.8) (private5) {};\n",
" \\node[draw=gray,dotted,fill=white,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (1.1,.6) (private4) {};\n",
" \\node[draw=gray,fill=white,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (1,.4) (private3) {};\n",
" \\node[draw=gray,fill=white,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.9,.2) (private2) {};\n",
" \\node[draw=gray,fill=white,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.8,0) (private) {};\n",
" \\node[ellipse,draw,minimum width=40pt,minimum height=35pt] at (-.5,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=35pt] at (2,0) (bob) {\\parbox{1cm}{\\centering Local device}};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \n",
"\n",
" \\node[ellipse,draw] at (6,0) (eve) {\\parbox{1cm}{\\centering Cloud servce}};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (bob);\n",
" %\\path[color=purple,very thick,<-,every node/.style={font=\\small}] (bob) edge [] node[yshift=.5cm] {\\parbox{1cm}{model update}} (eve);\n",
" \\draw[color=purple,->, line width=1pt] (bob) to [bend left] (eve);\n",
" \n",
" \\node at (.8,2.5) {\\parbox{3cm}{\\centering Private speech interactions}};\n",
" \\node at (4.2,0) {\\parbox{1.3cm}{\\centering Model updates}};\n",
" \\node at (6,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "2ceb79d1-36d9-4b5e-8d96-eac69f51ef0f",
"metadata": {
"tags": []
},
"source": [
"### Homomorphic encryption\n",
"\n",
"Suppose a service provider has a proprietary model, say an analysis method for Alzheimer's disease from the voice, and your doctor would like to analyse your voice with that method. Naturally your voice is also private, so you do not want to send your voice to the third-party service provider, but also the service provider does not want to send the model to you. [Homomorphic encryption](https://en.wikipedia.org/wiki/Homomorphic_encryption) provides a method for applying the secret model on *encrypted* data, such that you have to only send your data in an encrypted form to the service provider. Your doctor would then receive only the final diagnosis, but not the model nor your speech data. The concept is in principle beautiful, it solves the problem of mutual distrust very nicely. However, the compromise is that currently available homomorphic encryption methods require that all processing functions can be written as polynomial functions. In theory, we can transform any function to a corresponding polynomial, but the increase in complexity is often dramatic. Consequently, privacy-preserving methods based on homomorphic encryption typically have a prohibitively high computational complexity. {cite:p}`armknecht2015guide`"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "24a8d8cc-6d94-44db-ae97-efd724475bd0",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt,minimum height=35pt] at (-.5,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=35pt] at (2,0) (bob) {\\parbox{1cm}{\\centering Local device}};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.8,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (6,0) (eve) {\\parbox{1cm}{\\centering Cloud servce}};\n",
" \\draw[color=purple,->, dotted,line width=1pt] (eve) to [bend left] (bob);\n",
" \\draw[color=purple,->, dotted,line width=1pt] (bob) to [bend left] (eve);\n",
" \n",
" \\node at (.8,1.5) {\\parbox{3cm}{\\centering Private speech interaction}};\n",
" \\node at (4.1,0) {\\parbox{2cm}{\\centering Encrypted data}};\n",
" \\node at (6,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "5db2d7e4-4f82-4daf-9fe2-413ffed3c7cc",
"metadata": {},
"source": [
"### myData\n",
"\n",
"In addition to privacy-preserving algorithms, we can also design privacy-preserving architectures. The [myData](https://mydata.org/) paradigm is based on a three-tier design, where the user can choose where all his/her data is stored and where the user can give access for service providers to his/her data when required. The idea is to separate service providers from data storage, such that users have better control over his/her data. To transform existing services to adhere with the myData concept requires that new storage services for private data are created and that APIs between storage and processing services are specified. {cite:p}`poikola2016mydatb`"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "8fc221fe-46db-44e1-a76e-3f227af031dc",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
""
],
"text/plain": [
""
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%itikz --temp-dir --file-prefix dual-primary-\n",
"\\documentclass{standalone}\n",
"\\usepackage[utf8]{inputenc}\n",
"\\usepackage{tikz}\n",
"\\usepackage{verbatim}\n",
"\n",
"\n",
"\\usepackage{pgfplots}\n",
"\\DeclareUnicodeCharacter{2212}{−}\n",
"\\usepgfplotslibrary{groupplots,dateplot}\n",
"\\usetikzlibrary{patterns,shapes.arrows}\n",
"\\usetikzlibrary {fit} \n",
"\\usetikzlibrary{shapes.geometric,positioning}\n",
"\\usetikzlibrary{bending}\n",
"\\pgfplotsset{compat=newest}\n",
"\n",
"\\begin{document}\n",
"\n",
"\n",
"\\begin{tikzpicture}\n",
" \\node[ellipse,draw,minimum width=40pt,minimum height=35pt] at (-.5,0) (alice) {Alice};\n",
" \\node[ellipse,draw,minimum width=35pt] at (2,0) (bob) {\\parbox{1cm}{\\centering Local device}};\n",
" \\draw[->, line width=1pt] (alice) to [bend left] (bob);\n",
" \\draw[->, line width=1pt] (bob) to [bend left] (alice);\n",
" %\\node[fit=(alice) (bob), inner sep=1pt] (p) {};% {\\parbox{2.4cm}{\\centering Desired speech interaction\\vspace{-1cm}}};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob),label={[label distance=.01cm]180:{\\parbox{1.7cm}{\\centering Desired\\\\ interaction}}}] (private) {};\n",
" %\\node[draw=gray,inner sep=6pt,very thick,ellipse,fit=(alice) (bob) (p)] (private) {};\n",
" \\node[draw=gray,very thick,ellipse,minimum width=130pt, minimum height = 60pt] at (.8,0) (private) {};\n",
"\n",
" \\node[ellipse,draw] at (5.1,1) (mydata) {\\parbox{1cm}{\\centering Private cloud}};\n",
" \\node[ellipse,draw] at (8,0) (eve) {\\parbox{1cm}{\\centering Cloud servce}};\n",
" \\draw[color=purple,->, line width=1pt] (eve) to [bend left] (bob);\n",
" \\path[color=purple,very thick,->,every node/.style={font=\\small}] \n",
" (bob) edge [] node[yshift=.5cm] {\\parbox{1cm}{\\centering raw data}} (mydata);\n",
" \\path[color=purple,very thick,dashed,->,every node/.style={font=\\small}] \n",
" (mydata) edge [] node[xshift=.25cm,yshift=.5cm] {\\parbox{1cm}{\\centering reduced data}} (eve);\n",
" \n",
" \\node at (.8,1.5) {\\parbox{3cm}{\\centering Private speech interaction}}; \n",
" %\\node at (6,1.5) {\\parbox{2cm}{\\centering Restricted interactor}};\n",
"\\end{tikzpicture}\n",
"\n",
"\\end{document}"
]
},
{
"cell_type": "markdown",
"id": "2f8e4b5b-9803-486c-8b5d-c39485deef4d",
"metadata": {},
"source": [
"Note that, if a user chooses to store private data on a cloud-server, then it is still susceptible for abuse by the storage-service-provider, unless appropriate encryption methods are used. However, the user could in principle choose to store private data on a edge device, such that the storage-service-provider is cut out of the loop.\n",
"\n",
"A further risk is that in the myData concept, we usually assume that data is stored at a single central location, which becomes a central point of weakness. Should someone gain illegitimate access to the storage, then all your data would be compromised. Distributing data to several different storage locations might therefore be reasonable."
]
},
{
"cell_type": "markdown",
"id": "9f40c805-7e98-4950-b0ff-d323b1add280",
"metadata": {
"tags": []
},
"source": [
"## Design goals, human computer interfaces and user experience\n",
"\n",
"A common prejudice is that privacy and security requirements cause\n",
"problems for developers and make systems more difficult for users to\n",
"use. Such prejudice are unfortunate and patently misguided. The problem\n",
"is that many privacy problems are not visible to the casual observer and\n",
"their effects become apparent only when it already is too late. Another\n",
"argument is \"*privacy is* *not my concern* *because I haven't seen any\n",
"privacy problems*\", which is like saying that \"*rape is not my concern\n",
"because I haven't seen any rapes\".* This is thus an absurd argument.\n",
"Privacy safeguards are meant to protect users and developers from [very bad\n",
"consequences](https://www.reuters.com/article/us-apple-cyber-idUSKCN1VR29K).\n",
"These problems are real. *You* cannot ignore them.\n",
"\n",
"However, designing for privacy is also *not only* about protection of\n",
"users. It is also very much about designing technology which is *easy to\n",
"use* and where the user experience feels intuitive and natural. For\n",
"example, speaker recognition can be used to grant access to voice\n",
"technology such that the user does not have to be bothered with\n",
"passwords, PIN-codes or other cumbersome authentication methods. Overall\n",
"speech technology promises to give access to services without the need\n",
"scroll through menus on your washing machine to find that one mode which\n",
"is optimized for white curtains made out of cotton.\n",
"\n",
"The overall design goal could be that people should be able to *trust*\n",
"the system. In particular, a trustworthy system will be {cite:p}`Chen2003,Xie2009`\n",
"\n",
"- *consistent*; It does what it says, and it says what it does.\n",
"- *competent*; It is able to do what it should do and what it says it\n",
" does.\n",
"- *benevolent*; It cares for the user and it shows that in both *what*\n",
" is says and *how* it says it.\n",
"\n",
"These goals are best illustrated by examples;\n",
"\n",
"- We should avoid cognitive dissonance; if the computer has the\n",
" intellectual capacity corresponding to a 3-year-old child, but if it\n",
" then speaks with the authoritative voice of a middle-aged person, it\n",
" is giving mixed signals which can confuse users. Similarly, if a\n",
" computer leaks out all your information to the world, while speaking\n",
" with an intimate voice, it is also giving the wrong impression to\n",
" the user. Conversely, the intuitive impression which a device gives\n",
" should be consistent with its true nature.\n",
"- Consistency is extremely important and one mistake can be very\n",
" costly; If your friend Greg once tells about your intimate\n",
" health-incident to your friends, it casts a shadow of doubt over all\n",
" future and all of the past 10 years. Did he already earlier tell my\n",
" secrets to them? Who else has he told my secrets to? It takes a very\n",
" long time to patch such breaches of trust."
]
},
{
"cell_type": "markdown",
"id": "f3743886-9bcb-44cd-bd78-b3f72844925e",
"metadata": {
"tags": []
},
"source": [
"## Ethical dilemmas\n",
"\n",
"The following is a list of hypothetical questions which can (and do)\n",
"arise in the design of speech operated systems:\n",
"\n",
"- If a device hears cries of help, should it call the police\n",
" automatically, even when it breaches privacy and the trust of the\n",
" user?\n",
"- If a device hears indications of domestic abuse, but has no direct\n",
" evidence, should it call the police? What level of evidence is\n",
" required?\n",
"- If a device or service recognizes that you have Alzheimer's or some\n",
" other serious illness, should it tell it to you? Even if your doctor\n",
" would not yet have noticed it? Even if the diagnosis might be\n",
" incorrect? Even if you might choose not to want to hear it? Even if\n",
" you'd have a history of depression and such bad news might trigger\n",
" suicidal thoughts?\n",
"- Suppose you have been dreaming of buying a new bicycle, but haven't\n",
" told anyone. Your smart TV, though, knows about it because you have\n",
" searched for information about that bicycle. Suppose that your\n",
" spouse is simultaneously trying to come up with a nice surprise\n",
" birthday present. Should your smart TV suggest to your spouse that\n",
" she buys the bicycle?\n",
"- Suppose your local cafeteria has automatic speech operated ordering\n",
" of drinks. On this day, last year, you bought a birthday-surprise\n",
" drink from this cafeteria. Should the computer remember that and\n",
" congratulate you today about your birthday?\n",
"- The better your services know you, the better they could serve you.\n",
" That's undoubtedly a fact. (The fact that current services are not\n",
" optimal is not a contradiction.) However, do you want to have\n",
" privacy from devices in the same way you have privacy from your\n",
" friends. For example, your friends do not follow you to the toilet\n",
" or the bedroom; is it ok if your device does that?\n",
"- Suppose all your friends are using a service \"TakTik\" for in-group\n",
" communication. Further suppose that the TakTik service is known\n",
" for privacy-invasive practices. In choice of service, can you then,\n",
" as an individual, make a free choice based on your own preferences \n",
" to privacy? Overall, can you make free choices about communication services?\n",
"\n",
"\n",
"## Security and privacy in speech research\n",
"\n",
"Scientific research is based on arguments supported by evidence, where\n",
"evidence, in the speech sciences, is recordings of speech. Access to\n",
"speech data is therefore a mandatory part of research in the speech\n",
"sciences. To obtain trustworthy results, independent researchers have to\n",
"be able to verify each others results, which means that they have to\n",
"have access to the same or practically identical data sources. Shared\n",
"data is the gold standard for [reproducible\n",
"research](https://en.wikipedia.org/wiki/Reproducibility). However, the\n",
"sharing of speech data can be problematic with respect to speaker\n",
"privacy.\n",
"\n",
"The concept of \"uniquely identifiable\" is here the key. If an individual\n",
"is *not* uniquely identifiable in a data set, then you are allowed to\n",
"share that data. Conversely, if you remove all identifying data, then\n",
"you can share data relatively freely. However, in perspective of the\n",
"discussions above, it should be clear that it is not clear what\n",
"constitutes identifying data nor is it clear what makes that data\n",
"\"uniquely\" identifying.\n",
"\n",
"A second important consideration is *consent*. The persons whose voices\n",
"are recorded must be allowed to choose freely whether they want to\n",
"participate and that choice has to be explicit; you need to ask them\n",
"clearly whether they want to participate in a recording. The research\n",
"needs to be able to prove that consent has been given, and therefore\n",
"that consent must be documented carefully. Consent must also be given\n",
"freely such that there are no explicit or hidden penalties of rejecting\n",
"consent. Furthermore, if any uniquely identifying data of a participant\n",
"is stored, then the participant must be allowed to withdraw consent\n",
"afterwards. There are however some important exemptions to this rule;\n",
"the right to withdraw consent can be rejected, for example, if that\n",
"would corrupt the integrity of the data set, such as\n",
"\n",
"- If withdrawal of a participant could bias the results, then that\n",
" *could* be grounds for denying withdrawal. For example, if a dataset\n",
" is constructed in a way that it represents a balanced subset of the\n",
" population (the amount of say, males and females, different age,\n",
" cultural and education backgrounds are chosen to match the general\n",
" population), then we cannot remove any participants without\n",
" corrupting the distribution. Moreover, people more educated in\n",
" questions of privacy could hypothetically be more prone to\n",
" withdrawing their consent, such that the population becomes biased.\n",
"- If withdrawal of a participant could jeopardize the reproducibility\n",
" of results, then that *could* be grounds for denying withdrawal. For\n",
" example, if a machine learning algorithm is used on a dataset, then\n",
" we can recreate that algorithm only if we have *exactly* the same\n",
" data available. This is especially problematic if the dataset is\n",
" relatively small, where small changes in the dataset can have big\n",
" consequences on the output.\n",
"\n",
"To allow plausible grounds for denying the right to withdraw consent,\n",
"datasets can then be designed to be either balanced or relatively small.\n",
"Collecting balanced datasets is good practice in any case, such that\n",
"this is not a limitation but can actually improve quality. Conversely,\n",
"good data is balanced and that should be our goal; A consequence is that\n",
"we *might be forced* to deny the right to withdraw consent. Avoiding the\n",
"collection of excessively large data sets is also good from the\n",
"perspective of *data minimization* and data ecology.\n",
"\n",
"In a request for consent, the data collector should state the purpose of\n",
"the dataset (i.e. *purpose binding*). For instance, a dataset could be\n",
"collected for development of wake-word detection methods and consent is\n",
"received for that purpose. Then *it is not permissible* to use the same\n",
"data set for speaker detection experiments or medical analysis of the\n",
"voice. Period. It is therefore good practice to ask for consent in a\n",
"sufficiently wide way, such that researchers have some flexibility in\n",
"using the data. Blanket consent to all research purposes is however not\n",
"good practice. In particular, it is recommend that processing of\n",
"sensitive information such as health, ethnic, political information is\n",
"excluded if it is not the express purpose of the dataset (cf. data\n",
"minimization).\n",
"\n",
"If a dataset by nature does include uniquely identifiable data, then the\n",
"researchers need to apply stronger layers of safeguards. In particular,\n",
"typically researchers have to keep track of who has access to the data,\n",
"to ensure purpose binding and to allow withdrawal of consent. This could\n",
"also require that any researcher who downloads the data signs a contract\n",
"with the data provider, where the terms of usage are defined. Such a\n",
"contract can be required in any case, not only with uniquely\n",
"identifiable data.\n",
"\n",
"Data such as medical information, data about children or other exposed\n",
"groups, political, religious and gender-identity affiliations etc. are\n",
"particularly *sensitive*. If your dataset contains *any* such\n",
"information, then you have to apply stronger safeguards. To begin with,\n",
"access to such data has to be, in practice, always limited to only\n",
"persons who are included in a legally binding contract specifying access\n",
"rights and allowable uses, processing and storage.\n",
"\n",
"As an overall principle, note that the principal investigator (research\n",
"group leader) is legally responsible for the use of the data that is\n",
"collected, stored and processed. In particular, if a third party\n",
"downloads the data and misuses it, for example by analysing health\n",
"information even if no consent has been acquired for that purpose, then\n",
"it is the principal investigator who is responsible. However, the\n",
"principal investigator is only required to apply *reasonable*\n",
"*safeguards* to ensure that data is not misused. What level of\n",
"safeguards are sufficient has however not yet been agreed. It is likely\n",
"that there will never be rules which specify exactly a sufficient level\n",
"of safeguards.\n",
"\n",
"In the above discussion it has become clear that the nature of *unique\n",
"identifiability* can change over time, when new information is published\n",
"and new technologies emerge. This means that datasets which previously\n",
"were adequately protected, over time become exposed to privacy problems.\n",
"It is therefore important that researchers monitor their published\n",
"datasets over time such that if new threats emerge, they can take\n",
"appropriate action. For example, they could withdraw an dataset\n",
"entirely. Reasonable ways for implementing this could be:\n",
"\n",
"- *Expiry date*; All datasets should have a clearly stated shelf-life\n",
" and use of the dataset after the expiry date should be prohibited.\n",
" The manager of the dataset could update the expiry date if no new\n",
" threats are discovered. Academic publications based on expired\n",
" datasets should not be accepted.\n",
"- *Controlled access to datasets*; To enforce purpose binding and to\n",
" enable withdrawal of datasets, the data manager can require that all\n",
" users are registered and sign a formal contract which specifies\n",
" accepted uses.\n",
"\n",
"As a last resort, when data is so sensitive and private that it cannot\n",
"be publicly released, it is possible to require on-site processing of\n",
"data. For example, you can design a computing architecture, where data\n",
"resides on a secure server, to which researcher have access through a\n",
"secure\n",
"[API](https://en.wikipedia.org/wiki/Application_programming_interface).\n",
"Data never leaves the server such that privacy is always preserved. For\n",
"an even higher level of security, data can be stored on an\n",
"[air-gapped](https://en.wikipedia.org/wiki/Air_gap_%28networking%29)\n",
"computer system, which means that access to the data requires that\n",
"researchers physically come to the computer (no network access). This\n",
"level of security is usually the domain of military-grade systems."
]
},
{
"cell_type": "markdown",
"id": "4238dd54-3300-4973-986e-ea3b1aca3470",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"source": [
"## References\n",
"\n",
"- ISCA Special Interest Group \"Security and Privacy in Speech\n",
" Communication\", \n",
"\n",
"\n",
"\n",
"```{bibliography}\n",
":filter: docname in docnames\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}