{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## EMV is the worldwide standard for smart card payments\n", "- EMV (named after its original developers – Europay, MasterCard and Visa), is now used for over 80% of in person card payments worldwide\n", "- Initially EMV was a standard for contact smart-cards, but has since expanded to include the closely related contactless payment standards, and payment standards like 3D-Secure (for online payments)\n", "- Standards are maintained by [EMVCo](https://www.emvco.com/) which makes the current version publicly available\n", "- Over the past 15 years colleagues and I have found [numerous security vulnerabilities](https://www.cl.cam.ac.uk/research/security/banking/)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# EMV is primarily a compatibility standard, not security\n", "\n", "- It is designed to allow terminals and cards work, even without fully understanding the data processed, where cards have very limited RAM and processing power\n", "- *Be conservative in what you do, be liberal in what you accept from others* (Postel's law)\n", "- This has the potential to create security vulnerabilities, by increasing complexity and risking that important data will not be properly interpreted" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# One way EMV achieves compatibility is through the TLV format\n", "- Data is encoded as **T**ag, **L**ength, **V**alue\n", "- Very efficient, compared to JSON or XML but not easy to read by eye\n", "- Tree structure can be decoded without knowing all tags\n", "- Unknown tags can be ignored (for better or worse)\n", "- `0x00` values are ignored between TLV items, allowing in-place deletion (historically `0xff` too)\n", "- Also known as ASN.1 BER (Basic Encoding Rules) format from X.208, used for example in X.509 HTTPS certificates" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# It is sometimes helpful to manually decode TLV data\n", "- Data might be incomplete or corrupt\n", "- You have to explain why the decoded version is correct\n", "- You might have to write your own decoder (though I wouldn't recommend it)\n", "- Doing things for yourself can help you find where others might have slipped up\n", "\n", "Follow along yourself at https://murdoch.is/:/emvdecode with notes at https://murdoch.is/:/emvdecodenotes and repository at https://murdoch.is/:/emvdecoderepo " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [], "source": [ "## Some helpful utilities for processing hex data\n", "from hexutils import *" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "'10101010'" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Convert hex to binary\n", "to_bin('AA')" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "'303132'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Strip whitespace around and within hex\n", "strip_bytes(\"303\\n 132 \")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "'30 31 32'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Split hex into bytes for display\n", "split_bytes(\"303\\n 132 \")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "3" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Count how many bytes in a hex string\n", "len_bytes(\"303\\n 132 \")" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/plain": [ "'012'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Covert bytes to text (using ISO8859-1)\n", "decode_bytes(\"303\\n 132 \")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/html": [ "\n", "
0xaa =b8b7b6b5b4b3b2b1
10101010
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Format a byte into a binary table\n", "format_bytes('aa')" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "
0xaa =b8b7b6b5b4b3b2b1
1-------
-01-----
--------
---01010
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Split a byte into fields of specified length\n", "format_bytes('aa', [1,2,0,5])" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "'30 31'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Take a certain number of bytes from a hex string\n", "take(\"303\\n 132 \", 2)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'31 32'" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Take a certain number of bytes from a hex string with an offset\n", "take(\"303\\n 132 \", 2, 1)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Output of [cardpeek](http://pannetrat.com/Cardpeek/) log, requesting EMV record 2 from Short File Identifier (SFI 2)\n", "`C:00B2 02 14 00` `:6C97:` \n", "`C:00B2 02 14 97` `:9000:` `7081948C219F0206...`" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
0x14 =b8b7b6b5b4b3b2b1
00010---
-----100
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Reference control parameter (SFI 2)\n", "format_bytes('14', [5,3])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "151" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Length of record is 0x97 = 151 bytes\n", "0x97" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [], "source": [ "## Response as a Python string\n", "response=\"7081948C219F02069F03069F1A0295055F2A029A039C019F37049F35019F45029F4C089F34038D0C910A8A0295059F37049F4C089F08020002571352AAAAAAAAAAAA47D15122011407992700000F5F20134D5552444F43482F53544556454E204A2E44525F300202019F1F183134303739303030303030303030303932373030303030309F420208269F4401029F49039F37049F470103\"" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'70'" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Look at the first byte of the response (a tag)\n", "take(response, 1)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
0x70 =b8b7b6b5b4b3b2b1
01------
--1-----
---10000
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Application class, constructed, one-byte tag\n", "format_bytes(_, [2,1,5]) # 0x70 is a READ RECORD response message template" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'81'" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## First byte of length is 0x81...\n", "take(response, 1, 1)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/html": [ "\n", "
0x81 =b8b7b6b5b4b3b2b1
10000001
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## b8 is 1, so the actual length is in the next byte\n", "format_bytes(_)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'94'" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The actual length is 0x94...\n", "take(response, 1, 2)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "148" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## which is 148 in decimal\n", "int(_, 16)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'8c 21 9f 02 06 9f 03 06 9f 1a 02 95 05 5f 2a 02 9a 03 9c 01 9f 37 04 9f 35 01 9f 45 02 9f 4c 08 9f 34 03 8d 0c 91 0a 8a 02 95 05 9f 37 04 9f 4c 08 9f 08 02 00 02 57 13 52 aa aa aa aa aa aa 47 d1 51 22 01 14 07 99 27 00 00 0f 5f 20 13 4d 55 52 44 4f 43 48 2f 53 54 45 56 45 4e 20 4a 2e 44 52 5f 30 02 02 01 9f 1f 18 31 34 30 37 39 30 30 30 30 30 30 30 30 30 30 39 32 37 30 30 30 30 30 30 9f 42 02 08 26 9f 44 01 02 9f 49 03 9f 37 04 9f 47 01 03'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The tag value is 148 bytes, starting after tag (1 byte) and length (2 bytes)...\n", "take(response, 148, 1+2)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "148" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## which is the whole response from the card\n", "len_bytes(response) - 3" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'8c'" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The value is constructed so the next byte is a tag\n", "take(response, 1, 1+2)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
0x8c =b8b7b6b5b4b3b2b1
10------
--0-----
---01100
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## Context-specific class, primitive, 1-byte tag\n", "format_bytes(_, [2,1,5]) # 0x8c - CDOL1" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'21'" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Next byte will be the length\n", "take(response, 1, 1+2+1)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "
0x21 =b8b7b6b5b4b3b2b1
00100001
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## b8 is 0 so this is a 1 byte length (0x21)...\n", "format_bytes(_)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "33" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## which is 16 in decimal\n", "int(_, 16)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Response is `7081948C219F02069F03...`" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'9f 02 06 9f 03 06 9f 1a 02 95 05 5f 2a 02 9a 03 9c 01 9f 37 04 9f 35 01 9f 45 02 9f 4c 08 9f 34 03'" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The CDOL1 is 33 bytes, skipping the tags and lengths \n", "cdol1 = take(response, 33, 1+2+1+1)\n", "cdol1" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'8d'" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## After the CDOL1 the next tag is the CDOL2\n", "take(response, 1, 1+2+1+1 + 33) # 0x8d - CDOL2" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'0c'" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## with length 0x0c (12)\n", "take(response, 1, 1+2+1+1 + 33 + 1)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'91 0a 8a 02 95 05 9f 37 04 9f 4c 08'" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## So the CDOL2 can be extracted\n", "cdol2 = take(response, 0x0c, 1+2+1+1 + 33 + 1+1)\n", "cdol2" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "CDOL1 is `9f 02 06 9f 03 06 9f 1a 02 95 05...`" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'9f'" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## DOL objects are a list of tags and lengths that describe how to\n", "## send data to a card possibly unable to decode TLV data\n", "take(cdol1, 1)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
0x9f =b8b7b6b5b4b3b2b1
10------
--0-----
---11111
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## 9f starts a context-specific class, primitive, multi-byte tag\n", "format_bytes(_, [2,1,5])" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'02'" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## The next byte of the tag is 0x02\n", "take(cdol1, 1, 1)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
0x02 =b8b7b6b5b4b3b2b1
0-------
-0000010
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## 0x02 is the last byte of the tag, giving 0x9f02\n", "format_bytes(_, [1,7]) # 0x9f02 - Amount, Authorised (Numeric)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'06'" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Next is the length of the data expected: 0x06\n", "take(cdol1, 1, 1 + 1)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "'5f 20'" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Going back to the response, another 2-byte tag is at offset 78...\n", "take(response, 2, 78) # 0x5f20 – Cardholder Name" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'13'" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## which has length 0x13 (19)\n", "take(response, 1, 80)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'4d 55 52 44 4f 43 48 2f 53 54 45 56 45 4e 20 4a 2e 44 52'" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## This tag is ASCII encoded\n", "take(response, 0x13, 81)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "'MURDOCH/STEVEN J.DR'" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "decode_bytes(_)" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "'5f 30'" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Another 2-byte tag is at offset 100...\n", "take(response, 2, 100) # 0x5f30 – Service Code" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'02'" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## with length 0x02\n", "take(response, 1, 102)" ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'0201'" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## and in binary-coded decimal format: 201\n", "strip_bytes(take(response, 2, 103))" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [ { "data": { "text/plain": [ "'57'" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## At offset 57 we have a 1-byte tag (with length 59)...\n", "take(response, 1, 57) # 0x57 – Track 2 Equivalent Data" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "text/plain": [ "'52aaaaaaaaaaaa47d15122011407992700000f'" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## which is also in binary-coded decimal\n", "## I've removed the middle of my card number ;-)\n", "strip_bytes(take(response, 0x13, 59))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# TLV decoding properly is very tricky to get right\n", "- Different encodings are used in different contexts\n", "- Overflow and underflow errors could easily occur\n", "- Mistakes do happen and so banks do accept transactions that should in theory be invalid\n", "- Maybe you would like to do this yourself, whether professionally or just out of curiosity\n", "- “The only way to understand the wheel is to reinvent it.” — Mike Bond\n", "\n", "More on my research – https://murdoch.is/ \n", "Research group blog – http://www.benthamsgaze.org/" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.3" } }, "nbformat": 4, "nbformat_minor": 4 }