{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Neuron class\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Resources\n",
"\n",
"***\n",
"\n",
"- [Diagram of a neuron](https://upload.wikimedia.org/wikipedia/commons/1/10/Blausen_0657_MultipolarNeuron.png)\n",
"- [Neurons under a microscope](https://previews.123rf.com/images/tonaquatic19/tonaquatic191903/tonaquatic19190300040/120852799-neurons-cells-from-the-brain-under-the-microscope-view-for-education-.jpg)\n",
"- [One hidden layer](https://i.stack.imgur.com/HUQ8n.png)\n",
"- [Deep neural network](https://www.kdnuggets.com/wp-content/uploads/deep-neural-network.jpg)\n",
"- [Structure of artificial neuron](https://www.researchgate.net/publication/323775654/figure/fig1/AS:631604661739544@1527597694416/A-diagram-to-show-the-work-of-a-neuron-input-x-weights-w-bias-b-activation-function-f.png)\n",
"- [Artificial neural network with biases](https://www.researchgate.net/profile/Daniela_Guimaraes4/publication/328369102/figure/fig1/AS:683451942457344@1539959049109/Representation-of-the-artificial-neural-networks-showing-the-weights-w-and-the-bias-b.png)\n",
"- [Weight and height predicting gender](https://victorzhou.com/media/neural-network-post/network3.png)\n",
"- [Sigmoid function on Wikipedia](https://en.wikipedia.org/wiki/Sigmoid_function)\n",
"\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# Numerical arrays and operations.\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Simple neuron\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"![An artificial neuron](https://github.com/ianmcloughlin/images/raw/master/clean-neuron.png)\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Just one input for now.\n",
"def neuron(x, w, b):\n",
" return b + x * w"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.1"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"neuron(4.0, 0.5, 0.1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"#### Class\n",
"\n",
"***"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"# Represents a neuron.\n",
"# Note how we can now save the weight and bias with the neuron.\n",
"# The neuron has state now.\n",
"class Neuron:\n",
" # This is the constructor.\n",
" def __init__(self, winit, binit):\n",
" self.w = winit\n",
" self.b = binit\n",
" \n",
" # The neuron takes an input and gives an output.\n",
" # Uses the internal state (the weight and bias).\n",
" def fire(self, x):\n",
" return self.b + x * self.w"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# Create a neuron using the above class blueprint.\n",
"n = Neuron(0.5, 0.1)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<__main__.Neuron at 0x7f1d382f5d00>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# n is a variable an instance of the class.\n",
"# Memory has been allocated for it and it has value for w and b.\n",
"n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.5"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# n's w\n",
"n.w"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.1"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# n's b\n",
"n.b"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.1"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Call n's fire method with an x input.\n",
"n.fire(4.0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"#### Activation function\n",
"\n",
"***"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"# Let's add a function to be called just before the output is generated.\n",
"class Neuron:\n",
" def __init__(self, winit, binit, finit):\n",
" self.w = winit\n",
" self.b = binit\n",
" self.f = finit\n",
" \n",
" def fire(self, x):\n",
" return self.f(self.b + x * self.w)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"# A not so interesting function - it just return it's input.\n",
"def identity(x):\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"# In Python we can use lambda to create quick/anonymous functions.\n",
"# This is a different way of writing the above code.\n",
"# Typically you don't give the function a name when using lambda, but here we do.\n",
"identity = lambda x: x"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"# Create the neuron.\n",
"n = Neuron(0.5, 0.1, identity)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.1"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Same output as before, because f is the identity function.\n",
"n.fire(4.0)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# The Sigmoid function, as per WikiPedia.\n",
"def sigmoid(x):\n",
" return 1.0 / (1.0 + np.exp(-x))"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"# Create a neuron that users the sigmoid function instead of the identity function.\n",
"n = Neuron(0.5, 0.1, sigmoid)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.8909031788043871"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Now it gives a different output - a value between 0 and 1.\n",
"n.fire(4.0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"#### Arrays\n",
"\n",
"***"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"# What if we have more than one input value?\n",
"class Neuron:\n",
" # Incorporate the bias in the weight array.\n",
" # We'll have to adjust x to incorporate this.\n",
" def __init__(self, winit, finit):\n",
" self.w = winit\n",
" self.f = finit\n",
" \n",
" def fire(self, x):\n",
" # Append a 1.0 to x.\n",
" x = np.append(x, np.array([1.0]))\n",
" # Now we can use the dot product.\n",
" # This is the calculation we've been using in all but name all along.\n",
" return self.f(np.dot(x, self.w))"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"# Create the neuron with our weight vector and sigmoid activation.\n",
"n = Neuron(np.array([0.5, 0.1]), sigmoid)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.8909031788043871"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Fire the neuron.\n",
"n.fire(np.array([4.0]))"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.740774899182154"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# We can use the neuron class with more than one input now.\n",
"n = Neuron(np.array([0.4, 0.5, 0.1]), sigmoid)\n",
"n.fire(np.array([4.0, -1.3]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"#### What next?\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- How do we use more than one neuron?\n",
"- How do we update the w array to make the neuron(s) match known inputs and outputs?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"***\n",
"\n",
"#### End"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}