{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Recipe 1a - Provisioning an Azure storage account using the Azure portal" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Azure Blob storage is one of the four storage services available in Azure Storage. The other storage services are Table, Queue, and File Share. Table storage is used to store non-relational structured data as key-value pairs, queue storage is used to store messages as queues, and file share is used for creating file share directories/mount points that can be accessed using the NFS/SMB protocols. This recipe will focus on storing data using the Blob storage service." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "1. In the Azure portal, select **Create a resource** and choose **Storage account -- blob, file, table, queue** (or search for **storage account** in the search bar; do not choose **Storage accounts (classic)**).\n", "2. A new page, **Create a storage account**, will open. There are six tabs on the **Create a storage account** page -- **Basics**, **Advanced**, **Networking,** **Data protection**, **Tags**, and **Review + create**.\n", "3. In the **Basics** tab, we need to provide the Azure **Subscription**, **Resource group**, **Storage account name**, **Region**, **Performance**, and **Redundancy** values.\n", "4. In the **Advanced** tab, we need to select **Enable hierarchical namespace** under the **Data Lake Storage Gen2** settings.\n", "5. In the **Networking** tab, we need to provide the connectivity method.\n", "6. In the **Review + create** tab, review the configuration settings and select **Create** to provision the Azure storage account." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "The Azure storage account is deployed in the selected subscription, resource group, and location. The **Performance** tier can be either **Standard** or **Premium**. A **Standard **performance tier is a low-cost magnetic drive-backed storage. It's suitable for applications such as static websites and bulk storing flat files. The **Premium** tier is a high-cost SSD-backed storage service. The **Premium** tier can only be used with Azure virtual machine disks for I/O-intensive applications.\n", "\n", "The **Replication** options available are **Locally-redundant storage (LRS)**, **Zone-redundant storage (ZRS)**, **Geo-redundant storage (GRS)**, and **Geo-zone-redundant storage (GZRS)**. Local redundancy stores three local copies within the data center and provides fault tolerance for failures within it. Zone-redundant storage provides fault tolerance by copying data to additional data centers within the same region, while geo-redundant storage maintains copies across regions. Geo-zone-redundant storage combines geo- and zone-redundant features and offers the highest fault tolerance. The default **Geo-redundant storage (GRS)** option was selected, as it provides fault tolerance across regions.\n", "\n", "Azure storage accounts can be accessed publicly over the internet, through selected networks (selected IPs and IP ranges), and from private endpoints." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Recipe 1b - Provisioning an Azure storage account using PowerShell" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "PowerShell is a scripting language used to programmatically manage various tasks. In this recipe, we will learn how to provision an Azure storage account using PowerShell." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Before you start, you need to log in to the Azure subscription from the PowerShell console. To do this, execute the following command in a new PowerShell window:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "Connect-AzAccount" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command in a PowerShell window to create a new resource group. If you want to create the Azure storage account in an existing resource group, this step isn't required:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "New-AzResourceGroup -Name sparshde-powershell -Location 'East US'" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command to create a new Azure storage account in the sparshade-powershell resource group:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "New-AzStorageAccount -ResourceGroupName sparshde-powershell -Name sparshdestoragepowershellv2 -SkuName Standard_LRS -Location 'East US' -Kind StorageV2" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "There is a single command to create an Azure storage account using PowerShell -- **New-AzStorageAccount**. The **SkuName** parameter specifies the performance tier, and the **Kind** parameter specifies the account kind." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Recipe 2 - Creating containers and uploading files to Azure Blob storage using PowerShell" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "In this recipe, we will create a new container and upload files to Azure Blob storage using PowerShell.\n", "\n", "Execute the following commands to create the container in an Azure storage account:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "$storageaccountname=\"sparshstorage1\"\n", "$containername=\"logfiles\"\n", "$resourcegroup=\"sparsh-resource-1\"\n", "\n", "#Get the Azure Storage account context\n", "$storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccountname).Context;\n", "\n", "#Create a new container\n", "New-AzStorageContainer -Name $containername -Context $storagecontext" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Container creation is usually very quick. You should get the following output:\n", "\n", "```\n", "PS /Users/sparshagarwal/Desktop/projects/de> $storageaccountname=\"sparshstorage1\"\n", "PS /Users/sparshagarwal/Desktop/projects/de> $containername=\"logfiles\"\n", "PS /Users/sparshagarwal/Desktop/projects/de> $resourcegroup=\"sparsh-resource-1\"\n", "PS /Users/sparshagarwal/Desktop/projects/de> \n", "PS /Users/sparshagarwal/Desktop/projects/de> #Get the Azure Storage account context\n", "PS /Users/sparshagarwal/Desktop/projects/de> $storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccountname).Context;\n", "PS /Users/sparshagarwal/Desktop/projects/de> \n", "PS /Users/sparshagarwal/Desktop/projects/de> #Create a new container\n", "PS /Users/sparshagarwal/Desktop/projects/de> New-AzStorageContainer -Name $containername -Context $storagecontext\n", " \n", " Storage Account Name: sparshstorage1\n", "\n", "Name PublicAccess LastModified IsDeleted VersionId\n", "---- ------------ ------------ --------- ---------\n", "logfiles Off 2/10/2023 6:27:00 PM +00:00 \n", "\n", "PS /Users/sparshagarwal/Desktop/projects/de> \n", "``` " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to upload a text file to an existing container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "#upload single file to container\n", "\n", "Set-AzStorageBlobContent -File \".\\data\\Logfiles\\Logfile1.txt\" -Context $storagecontext -Blob logfile1.txt -Container $containername" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /Users/sparshagarwal/Desktop/projects/de> Set-AzStorageBlobContent -File \".\\data\\Logfiles\\Logfile1.txt\" -Context $storagecontext -Blob logfile1.txt -Container $containername\n", " \n", " AccountName: sparshstorage1, ContainerName: logfiles \n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDelete\n", " d\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------\n", "logfile1.txt BlockBlob 48 application/octet-stream 2023-02-10 18:30:50Z Hot False \n", "\n", "PS /Users/sparshagarwal/Desktop/projects/de> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to upload all the files in a directory to an Azure container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "#get files to be uploaded from the directory\n", "\n", "$files = Get-ChildItem -Path \".\\data\\Logfiles\";\n", "\n", "#iterate through each file in the folder and upload it to the azure container\n", "\n", "foreach($file in $files){\n", "\n", "Set-AzStorageBlobContent -File $file.FullName -Context $storagecontext -Blob $file.BaseName -Container $containername -Force\n", "\n", "}" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> #iterate through each file in the folder and upload it to the azure container\n", "PS /> \n", "PS /> foreach($file in $files){\n", ">> \n", ">> Set-AzStorageBlobContent -File $file.FullName -Context $storagecontext -Blob $file.BaseName -Container $containername -Force\n", ">> \n", ">> }\n", " \n", " AccountName: sparshstorage1, ContainerName: logfiles \n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDelete\n", " d\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------\n", "Logfile1 BlockBlob 48 application/octet-stream 2023-02-10 18:35:09Z Hot False \n", "Logfile2 BlockBlob 78 application/octet-stream 2023-02-10 18:35:09Z Hot False \n", " \n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "The storage container is created using the **New-AzStorageContainer** command. It takes two parameters -- the container name and the storage context. The storage context can be set using the **Get-AzStorageAccount** command context property.\n", "\n", "To upload files to the container, we used the **Set-AzStorageBlobContent** command. This command requires the storage context, a file path to be uploaded, and the container name. To upload multiple files, we can iterate through the folder and upload each file using the **Set-AzStorageBlobContent** command." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Recipe 3 - Managing blobs in Azure Storage using PowerShell" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "In this recipe, we will learn how to perform various management tasks on an Azure blob. We will perform operations such as copying, listing, modifying, deleting, and downloading files from Azure Blob storage.\n", "\n", "Let's perform the following operations in this recipe:\n", "\n", "1. Copy files/blobs between two blob storage containers.\n", "2. List files from a blob container.\n", "3. Modify the storage access tier of a blob from **Hot** to **Cool**.\n", "4. Download a file/blob from a container.\n", "5. Delete a file/blob from a container." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Copying blobs between containers" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to create a new container in an Azure storage account:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "#set the parameter values\n", "$storageaccountname=\"sparshstorage1\"\n", "$resourcegroup=\"sparsh-resource-1\"\n", "$sourcecontainername=\"logfiles\"\n", "$destcontainername=\"textfiles\"\n", "\n", "#Get storage account context\n", "$storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccountname).Context\n", "\n", "# create the container\n", "$destcontainer = New-AzStorageContainer -Name $destcontainername -Context $storagecontext\n", "\n", "$destcontainer" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> #set the parameter values \n", "PS /> $storageaccountname=\"sparshstorage1\"\n", "PS /> $resourcegroup=\"sparsh-resource-1\"\n", "PS /> $sourcecontainername=\"logfiles\"\n", "PS /> $destcontainername=\"textfiles\"\n", "PS /> \n", "PS /> #Get storage account context\n", "PS /> $storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccountname).Context\n", "PS /> \n", "PS /> # create the container\n", "PS /> $destcontainer = New-AzStorageContainer -Name $destcontainername -Context $storagecontext\n", "PS /> \n", "PS /> $destcontainer\n", "\n", " Storage Account Name: sparshstorage1\n", "\n", "Name PublicAccess LastModified IsDeleted VersionId\n", "---- ------------ ------------ --------- ---------\n", "textfiles Off 2/10/2023 6:43:15 PM +00:00 \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command to copy the Logfile1 blob from the source container to the destination container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "Start-CopyAzureStorageBlob -SrcBlob \"Logfile1\" -SrcContainer $sourcecontainername -DestContainer $destcontainername -Context $storagecontext -DestContext $storagecontext" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> Start-CopyAzureStorageBlob -SrcBlob \"Logfile1\" -SrcContainer $sourcecontainername -DestContainer $destcontainername -Context $storagecontext -DestContext $storagecontext\n", " \n", " AccountName: sparshstorage1, ContainerName: textfiles\n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------\n", "Logfile1 BlockBlob 48 application/octet-stream 2023-02-10 18:46:34Z Hot False \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command to copy all the blobs from the source container to the destination container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "Get-AzStorageBlob -Container $sourcecontainername -Context $storagecontext | Start-CopyAzureStorageBlob -DestContainer $destcontainername -DestContext $storagecontext -force" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "\n", "PS /> Get-AzStorageBlob -Container $sourcecontainername -Context $storagecontext | Start-CopyAzureStorageBlob -DestContainer $destcontainername -DestContext $storagecontext -force\n", " \n", " AccountName: sparshstorage1, ContainerName: textfiles\n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------\n", "Logfile1 BlockBlob 48 application/octet-stream 2023-02-10 18:46:34Z Hot False \n", "Logfile2 BlockBlob 78 application/octet-stream 2023-02-10 18:47:52Z Hot False \n", "logfile1.txt BlockBlob 48 application/octet-stream 2023-02-10 18:47:52Z Hot False \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Listing blobs in an Azure storage container" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command to list the blobs from the destination container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "(Get-AzStorageContainer -Name $destcontainername -Context $storagecontext).CloudBlobContainer.ListBlobs()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> (Get-AzStorageContainer -Name $destcontainername -Context $storagecontext).CloudBlobContainer.ListBlobs()\n", " \n", " Container Uri: https://sparshstorage1.blob.core.windows.net/textfiles\n", "\n", "Name BlobType Length IsDeleted RemainingD ContentType LastModified AccessTier SnapshotTime\n", " aysBeforeP\n", " ermanentDe\n", " lete\n", "---- -------- ------ --------- ---------- ----------- ------------ ---------- ------------\n", "Logfile1 BlockBlob 48 False application/octet-stream 2023-02-10 18:47:52Z \n", "Logfile2 BlockBlob 78 False application/octet-stream 2023-02-10 18:47:52Z \n", "logfile1.txt BlockBlob 48 False application/octet-stream 2023-02-10 18:47:52Z \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Modifying a blob access tier" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to change the access tier of a blob:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "$blob = Get-AzStorageBlob -Blob *Logfile2* -Container $sourcecontainername -Context $storagecontext\n", "\n", "#Get current access tier\n", "$blob\n", "\n", "#change access tier to cool\n", "$blob.ICloudBlob.SetStandardBlobTier(\"cool\")\n", "\n", "#Get the modified access tier\n", "Get-AzStorageBlob -Blob *Logfile2* -Container $sourcecontainername -Context $storagecontext" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> $blob = Get-AzStorageBlob -Blob *Logfile2* -Container $sourcecontainername -Context $storagecontext\n", "PS /> \n", "PS /> #Get current access tier\n", "PS /> $blob\n", "\n", " AccountName: sparshstorage1, ContainerName: logfiles\n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------\n", "Logfile2 BlockBlob 78 application/octet-stream 2023-02-10 18:35:09Z Hot False \n", "\n", "PS /> \n", "PS /> #change access tier to cool\n", "PS /> $blob.ICloudBlob.SetStandardBlobTier(\"cool\")\n", "PS /> \n", "PS /> #Get the modified access tier\n", "PS /> Get-AzStorageBlob -Blob *Logfile2* -Container $sourcecontainername -Context $storagecontext\n", " \n", " AccountName: sparshstorage1, ContainerName: logfiles\n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------\n", "Logfile2 BlockBlob 78 application/octet-stream 2023-02-10 18:35:09Z Cool False \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to change the access tier of all the blobs in the container:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "$blobs = Get-AzStorageBlob -Container $destcontainername -Context $storagecontext\n", "\n", "#change the access tier of all the blobs in the container\n", "$blobs.icloudblob.setstandardblobtier(\"Cool\")\n", "\n", "#verify the access tier\n", "Get-AzStorageBlob -Container $destcontainername -Context $storagecontext" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You should get an output similar to the following:\n", "\n", "```\n", "PS /> $blobs = Get-AzStorageBlob -Container $destcontainername -Context $storagecontext\n", "PS /> \n", "PS /> #change the access tier of all the blobs in the container\n", "PS /> $blobs.icloudblob.setstandardblobtier(\"Cool\")\n", "PS /> \n", "PS /> #verify the access tier\n", "PS /> Get-AzStorageBlob -Container $destcontainername -Context $storagecontext\n", "\n", " \n", " AccountName: sparshstorage1, ContainerName: textfiles\n", "\n", "Name BlobType Length ContentType LastModified AccessTier SnapshotTime IsDeleted VersionId\n", "---- -------- ------ ----------- ------------ ---------- ------------ --------- ---------\n", "Logfile1 BlockBlob 48 application/octet-stream 2023-02-10 18:47:52Z Cool False \n", "Logfile2 BlockBlob 78 application/octet-stream 2023-02-10 18:47:52Z Cool False \n", "logfile1.txt BlockBlob 48 application/octet-stream 2023-02-10 18:47:52Z Cool False \n", "\n", "PS /> \n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Downloading a blob" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following commands to download a blob from Azure Storage to your local computer:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "$storagecontext = (Get-AzStorageAccount -ResourceGroupName\n", "$resourcegroup -Name $storageaccountname).Context\n", "\n", "#download the blob\n", "Get-AzStorageBlobContent -Blob \"Logfile1\" -Container\n", "\n", "$sourcecontainername -Destination .\\data\\Logfiles\\ -Context $storagecontext -Force" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Deleting a blob" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Execute the following command to remove/delete a blob:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [], "source": [ "$storagecontext = (Get-AzStorageAccount -ResourceGroupName\n", "$resourcegroup -Name $storageaccountname).Context\n", "Remove-AzStorageBlob -Blob \"Logfile2\" -Container\n", "$sourcecontainername -Context $storagecontext" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Copying blobs across containers in the same storage account or a different storage account can be done easily by the PowerShell **Start-CopyAzureStorageBlob** command. The command takes the source and destination blobs, the source and destination containers, and the source and destination storage accounts as parameters. To copy all blobs in a container, we can run **Get-AzStorageBlob** to get all the blobs in the container and pipe the blobs to the **Start-CopyAzureStorageBlob** command.\n", "\n", "A blob access tier can be modified by first getting the reference to the blob object using **Get-AzStorageBlob** and then modifying the access tier using the **setstandardblobtier** property. There are three access tiers -- **Hot**, **Cool**, and **Archive**:\n", "\n", "- The **Hot tier** is suitable for files that are accessed frequently. It has a higher storage cost and low access cost.\n", "- The **Cool tier** is suitable for infrequently accessed files and has a lower access cost and a lower storage cost.\n", "- The **Archive tier**, as the name suggests, is used for long-term archival and should be used for files that are seldom required. It has the highest access cost and the lowest storage cost.\n", "\n", "To download a blob from Azure to a local system, we use **Get-AzStorageBlobContent**. The command accepts the blob name, the container name, the local file path, and the storage context.\n", "\n", "To delete a blob, run **Remove-AzStorageBlob**. Provide the blob name, the container name, and the storage context." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Recipe 4 - Configuring blob lifecycle management for blob objects using the Azure portal" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Azure Storage provides different blob access tiers such as Hot, Cool, and Archive. Each access tier has a different storage and data transfer cost. Applying a proper lifecycle rule to move a blob among different access tiers helps optimize the cost. In this recipe, we will learn how to apply a lifecycle rule to a blob using the Azure portal." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "1. In the Azure portal, find and open the Azure Storage case\n", "1. Search for Data management and select Lifecycle Management under Data management\n", "1. On the Add a rule page, create a rule to provide the lifecycle configuration. A lifecycle defines when to move a blob from a Hot to a Cool access tier, when to move a blob from a Cool to a Storage access tier, and when to delete the blob. Select Limit blobs with filters to create a lifecycle policy for a particular container. Click Next\n", "1. Specify the condition you would like to use as a lifecycle policy. For example, a rule that moves the blobs that haven't been modified in the last 30 days to Cool storage and 60 days to Archive storage\n", "1. In Filter set, specify the specific container or filename for which you need to apply the rule" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "A blob lifecycle management rule helps in managing storage costs by modifying the access tier of blobs as per the specified rule. Consider a log processing application that reads the log file from Azure Storage, analyzes it, and saves the result in a database. As the log file is read and processed, it may not be needed any further. Therefore, moving it to a Cool access tier from a Hot access tier will save on storage costs.\n", "\n", "Blob lifecycle management helps in automating the access tier modification as per the application requirement and is, therefore, a must-have for any storage-based application." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.10" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" } } }, "nbformat": 4, "nbformat_minor": 2 }