{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "57fe8246", "metadata": {}, "source": [ "# Offline reinforcement learning with Ray AIR\n", "In this example, we'll train a reinforcement learning agent using offline training.\n", "\n", "Offline training means that the data from the environment (and the actions performed by the agent) have been stored on disk. In contrast, online training samples experiences live by interacting with the environment." ] }, { "attachments": {}, "cell_type": "markdown", "id": "edc8d8ac", "metadata": {}, "source": [ "Let's start with installing our dependencies:" ] }, { "cell_type": "code", "execution_count": 1, "id": "0ef2e884", "metadata": {}, "outputs": [], "source": [ "# !pip install -qU \"ray[rllib]\" gymnasium" ] }, { "attachments": {}, "cell_type": "markdown", "id": "503b1b55", "metadata": {}, "source": [ "Now we can run some imports:" ] }, { "cell_type": "code", "execution_count": 2, "id": "db0a45ff", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " DESCRIPTOR = _descriptor.FileDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:43: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:29: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _TENSORSHAPEPROTO_DIM = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:73: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:80: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:66: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _TENSORSHAPEPROTO = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:19: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " DESCRIPTOR = _descriptor.FileDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:33: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:37: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:41: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:45: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:49: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:53: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:57: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:61: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:65: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:69: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:73: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:77: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:81: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:85: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:89: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:93: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:97: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:101: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:105: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:109: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:113: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:117: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:121: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:125: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:129: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:133: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:137: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:141: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:145: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:149: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:153: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:157: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:161: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:165: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:169: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:173: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:177: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:181: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:185: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:189: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:193: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:197: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:201: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:205: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:209: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:213: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:217: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.EnumValueDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:27: DeprecationWarning: Call to deprecated create function EnumDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _DATATYPE = _descriptor.EnumDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:287: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:280: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _SERIALIZEDDTYPE = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:20: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " DESCRIPTOR = _descriptor.FileDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:39: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:46: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:32: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _RESOURCEHANDLEPROTO_DTYPEANDSHAPE = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:76: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:83: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:90: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:97: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:104: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:111: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:69: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _RESOURCEHANDLEPROTO = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " DESCRIPTOR = _descriptor.FileDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:40: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:47: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:54: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:61: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:68: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:75: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:89: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:96: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:103: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:110: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:117: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:124: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:131: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:138: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:145: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:152: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:33: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _TENSORPROTO = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:183: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:190: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:197: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:176: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _VARIANTTENSORDATAPROTO = _descriptor.Descriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " DESCRIPTOR = _descriptor.FileDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:40: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:47: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:54: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:61: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:68: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:75: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", " _descriptor.FieldDescriptor(\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:61: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", " if (distutils.version.LooseVersion(tf.__version__) <\n", "/Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:62: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", " distutils.version.LooseVersion(required_tensorflow_version)):\n", "WARNING:tensorflow:From /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py:561: calling function (from tensorflow.python.eager.polymorphic_function.polymorphic_function) with experimental_relax_shapes is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "experimental_relax_shapes is deprecated, use reduce_retracing instead\n" ] } ], "source": [ "import gymnasium as gym\n", "\n", "import numpy as np\n", "import ray\n", "from ray.air import Checkpoint\n", "from ray.air.config import CheckpointConfig, RunConfig\n", "from ray.train.rl.rl_predictor import RLPredictor\n", "from ray.train.rl.rl_trainer import RLTrainer\n", "from ray.air.config import ScalingConfig\n", "from ray.air.result import Result\n", "from ray.rllib.algorithms.bc import BC\n", "from ray.tune.tuner import Tuner" ] }, { "attachments": {}, "cell_type": "markdown", "id": "184fe936", "metadata": {}, "source": [ "We will be training on offline data - this means we have full agent trajectories stored somewhere on disk and want to train on these past experiences.\n", "\n", "Usually this data could come from external systems, or a database of historical data. But for this example, we'll generate some offline data ourselves and store it using RLlibs `output_config`." ] }, { "cell_type": "code", "execution_count": 3, "id": "5aeed761", "metadata": {}, "outputs": [], "source": [ "def generate_offline_data(path: str):\n", " print(f\"Generating offline data for training at {path}\")\n", " trainer = RLTrainer(\n", " algorithm=\"PPO\",\n", " run_config=RunConfig(stop={\"timesteps_total\": 5000}),\n", " config={\n", " \"env\": \"CartPole-v1\",\n", " \"output\": \"dataset\",\n", " \"output_config\": {\n", " \"format\": \"json\",\n", " \"path\": path,\n", " \"max_num_samples_per_file\": 1,\n", " },\n", " \"batch_mode\": \"complete_episodes\",\n", " \"framework\": \"torch\"\n", " },\n", " )\n", " trainer.fit()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "8bca906c", "metadata": {}, "source": [ "Here we define the training function. It will create an `RLTrainer` using the `PPO` algorithm and kick off training on the `CartPole-v1` environment. It will use the offline data provided in `path` for this." ] }, { "cell_type": "code", "execution_count": 4, "id": "f5071ce0", "metadata": {}, "outputs": [], "source": [ "def train_rl_bc_offline(path: str, num_workers: int, use_gpu: bool = False) -> Result:\n", " print(\"Starting offline training\")\n", " dataset = ray.data.read_json(\n", " path, parallelism=num_workers, ray_remote_args={\"num_cpus\": 1}\n", " )\n", "\n", " trainer = RLTrainer(\n", " run_config=RunConfig(\n", " stop={\"training_iteration\": 5},\n", " checkpoint_config=CheckpointConfig(checkpoint_at_end=True)\n", " ),\n", " scaling_config=ScalingConfig(num_workers=num_workers, use_gpu=use_gpu),\n", " datasets={\"train\": dataset},\n", " algorithm=BC,\n", " config={\n", " \"env\": \"CartPole-v1\",\n", " \"framework\": \"tf\",\n", " \"evaluation_num_workers\": 1,\n", " \"evaluation_interval\": 1,\n", " \"evaluation_config\": {\"input\": \"sampler\"},\n", " \"framework\": \"torch\"\n", " },\n", " )\n", " result = trainer.fit()\n", " return result" ] }, { "attachments": {}, "cell_type": "markdown", "id": "d935cdee", "metadata": {}, "source": [ "Once we trained our RL policy, we want to evaluate it on a fresh environment. For this, we will also define a utility function:" ] }, { "cell_type": "code", "execution_count": 5, "id": "2628f3b0", "metadata": {}, "outputs": [], "source": [ "def evaluate_using_checkpoint(checkpoint: Checkpoint, num_episodes) -> list:\n", " predictor = RLPredictor.from_checkpoint(checkpoint)\n", "\n", " env = gym.make(\"CartPole-v1\")\n", "\n", " rewards = []\n", " for i in range(num_episodes):\n", " obs, _ = env.reset()\n", " reward = 0.0\n", " terminated = truncated = False\n", " while not terminated and not truncated:\n", " action = predictor.predict(np.array([obs]))\n", " obs, r, terminated, truncated, _ = env.step(action[0])\n", " reward += r\n", " rewards.append(reward)\n", "\n", " return rewards" ] }, { "attachments": {}, "cell_type": "markdown", "id": "84f4bebe", "metadata": {}, "source": [ "Let's put it all together. First, we initialize Ray and create the offline data:" ] }, { "cell_type": "code", "execution_count": 10, "id": "cae1337e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Generating offline data for training at /tmp/out\n" ] }, { "data": { "text/html": [ "
\n", "
\n", "
\n", "

Tune Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "
Current time:2023-03-30 10:01:06
Running for: 00:00:16.15
Memory: 18.1/32.0 GiB
\n", "
\n", "
\n", "
\n", "

System Info

\n", " Using FIFO scheduling algorithm.
Logical resource usage: 0/8 CPUs, 0/0 GPUs\n", "
\n", " \n", "
\n", "
\n", "
\n", "

Trial Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name status loc iter total time (s) ts reward episode_reward_max episode_reward_min episode_len_mean
AIRPPO_6c451_00000TERMINATED127.0.0.1:82000 2 7.148298179 43.54 143 10 43.54
\n", "
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 5x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\u001B[32m [repeated 5x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 40x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\u001B[32m [repeated 40x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:176: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:217: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 47x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.EnumValueDescriptor(\u001B[32m [repeated 47x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(ReadJSON pid=81351)\u001B[0m 2023-03-30 09:58:43,544\tINFO worker.py:837 -- Task failed with retryable exception: TaskID(85bd3840f615b0e4ffffffffffffffffffffffff01000000).\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(ReadJSON pid=81351)\u001B[0m Traceback (most recent call last):\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(ReadJSON pid=81351)\u001B[0m File \"python/ray/_raylet.pyx\", line 651, in ray._raylet.execute_dynamic_generator_and_store_task_outputs\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(ReadJSON pid=81351)\u001B[0m File \"python/ray/_raylet.pyx\", line 2533, in ray._raylet.CoreWorker.store_task_outputs\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/execution/operators/map_operator.py\", line 374, in _map_task\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m for b_out in fn(iter(blocks), ctx):\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/execution/legacy_compat.py\", line 287, in do_map\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m yield from block_fn(blocks, ctx, *fn_args, **fn_kwargs)\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/plan.py\", line 1317, in wrapper\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m yield from fn(block, ctx, *args, **kwargs)\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/plan.py\", line 1195, in block_fn\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m for block in read_fn():\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/datasource/file_based_datasource.py\", line 500, in read_files\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m yield output_buffer.next()\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/output_buffer.py\", line 74, in next\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m block = self._buffer.build()\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/delegating_block_builder.py\", line 70, in build\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m return self._builder.build()\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/table_block.py\", line 118, in build\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m return self._concat_tables(tables)\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/arrow_block.py\", line 127, in _concat_tables\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m return transform_pyarrow.concat(tables)\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"/Users/avnish/ray/python/ray/data/_internal/arrow_ops/transform_pyarrow.py\", line 255, in concat\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m table = pyarrow.concat_tables(blocks, promote=True)\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"pyarrow/table.pxi\", line 2338, in pyarrow.lib.concat_tables\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"pyarrow/error.pxi\", line 143, in pyarrow.lib.pyarrow_internal_check_status\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m File \"pyarrow/error.pxi\", line 99, in pyarrow.lib.check_status\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRBC pid=81380)\u001B[0m pyarrow.lib.ArrowInvalid: Unable to merge: Field policy_batches has incompatible types: struct>, action_logp: list, action_prob: list, actions: list, advantages: list, agent_index: list, eps_id: list, new_obs: string, obs: string, prev_actions: list, prev_rewards: list, rewards: list, t: list, terminateds: list, truncateds: list, unroll_id: list, value_targets: list, vf_preds: list>> vs struct>, action_logp: list, action_prob: list, actions: list, advantages: list, agent_index: list, eps_id: list, infos: list>>>, new_obs: string, obs: string, prev_actions: list, prev_rewards: list, rewards: list, t: list, terminateds: list, truncateds: list, unroll_id: list, value_targets: list, vf_preds: list>>\u001B[32m [repeated 8x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:61: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m if (distutils.version.LooseVersion(tf.__version__) <\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:62: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m distutils.version.LooseVersion(required_tensorflow_version)):\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m WARNING:tensorflow:From /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py:561: calling function (from tensorflow.python.eager.polymorphic_function.polymorphic_function) with experimental_relax_shapes is deprecated and will be removed in a future version.\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m Instructions for updating:\n", "\u001B[2m\u001B[36m(pid=82000)\u001B[0m experimental_relax_shapes is deprecated, use reduce_retracing instead\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(AIRPPO pid=82000)\u001B[0m 2023-03-30 10:00:54,374\tWARNING algorithm_config.py:636 -- Cannot create PPOConfig from given `config_dict`! Property __stdout_file__ not supported.\n", "\u001B[2m\u001B[36m(AIRPPO pid=82000)\u001B[0m 2023-03-30 10:00:54,479\tINFO algorithm.py:527 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _TENSORSHAPEPROTO_DIM = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _TENSORSHAPEPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:27: DeprecationWarning: Call to deprecated create function EnumDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _DATATYPE = _descriptor.EnumDescriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _SERIALIZEDDTYPE = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _RESOURCEHANDLEPROTO_DTYPEANDSHAPE = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _RESOURCEHANDLEPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _TENSORPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m _VARIANTTENSORDATAPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 10x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\u001B[32m [repeated 10x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 80x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m _descriptor.FieldDescriptor(\u001B[32m [repeated 80x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:176: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 14x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:217: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 94x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m _descriptor.EnumValueDescriptor(\u001B[32m [repeated 94x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(AIRPPO pid=82000)\u001B[0m 2023-03-30 10:00:59,026\tWARNING util.py:67 -- Install gputil for GPU system monitoring.\n", "\u001B[2m\u001B[36m(RolloutWorker pid=82019)\u001B[0m 2023-03-30 10:01:00,115\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:62: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\u001B[32m [repeated 4x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m if (distutils.version.LooseVersion(tf.__version__) <\u001B[32m [repeated 2x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m distutils.version.LooseVersion(required_tensorflow_version)):\u001B[32m [repeated 2x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m WARNING:tensorflow:From /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py:561: calling function (from tensorflow.python.eager.polymorphic_function.polymorphic_function) with experimental_relax_shapes is deprecated and will be removed in a future version.\u001B[32m [repeated 2x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m Instructions for updating:\u001B[32m [repeated 2x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m experimental_relax_shapes is deprecated, use reduce_retracing instead\u001B[32m [repeated 2x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m Resource usage vs limits 0: 0%| | 0/1 [00:00\n", "

Trial Progress

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name agent_timesteps_totalconnector_metrics counters custom_metrics date done episode_len_meanepisode_media episode_reward_max episode_reward_mean episode_reward_min episodes_this_iter episodes_totalhostname info iterations_since_restorenode_ip num_agent_steps_sampled num_agent_steps_trained num_env_steps_sampled num_env_steps_sampled_this_iter num_env_steps_trained num_env_steps_trained_this_iter num_faulty_episodes num_healthy_workers num_in_flight_async_reqs num_remote_worker_restarts num_steps_trained_this_iterperf pidpolicy_reward_max policy_reward_mean policy_reward_min sampler_perf sampler_results time_since_restore time_this_iter_s time_total_stimers timestamp timesteps_total training_iterationtrial_id
AIRPPO_6c451_00000 8179{'ObsPreprocessorConnector_ms': 0.0020761489868164062, 'StateBufferConnector_ms': 0.0016286373138427734, 'ViewRequirementAgentConnector_ms': 0.0386197566986084}{'num_env_steps_sampled': 8179, 'num_env_steps_trained': 8179, 'num_agent_steps_sampled': 8179, 'num_agent_steps_trained': 8179}{} 2023-03-30_10-01-06True 43.54{} 143 43.54 10 90 266avnishs-mbp-3.lan{'learner': {'default_policy': {'learner_stats': {'allreduce_latency': 0.0, 'grad_gnorm': 1.0311362160602584, 'cur_kl_coeff': 0.29999999999999993, 'cur_lr': 5.0000000000000016e-05, 'total_loss': 8.954141706724961, 'policy_loss': -0.029519098582871568, 'vf_loss': 8.978031371037165, 'vf_explained_var': 0.04644986521452665, 'kl': 0.018764853765244092, 'entropy': 0.6120760556931297, 'entropy_coeff': 0.0}, 'model': {}, 'custom_metrics': {}, 'num_agent_steps_trained': 128.0, 'num_grad_updates_lifetime': 1410.5, 'diff_num_grad_updates_vs_sampler_policy': 479.5}}, 'num_env_steps_sampled': 8179, 'num_env_steps_trained': 8179, 'num_agent_steps_sampled': 8179, 'num_agent_steps_trained': 8179} 2127.0.0.1 8179 8179 8179 4153 8179 4153 0 2 0 0 4153{'cpu_util_percent': 29.48, 'ram_util_percent': 56.42}82000{} {} {} {'mean_raw_obs_processing_ms': 0.1494050133826096, 'mean_inference_ms': 0.3030270243531414, 'mean_action_processing_ms': 0.04484232997177898, 'mean_env_wait_ms': 0.020527675791925257, 'mean_env_render_ms': 0.0}{'episode_reward_max': 143.0, 'episode_reward_min': 10.0, 'episode_reward_mean': 43.54, 'episode_len_mean': 43.54, 'episode_media': {}, 'episodes_this_iter': 90, 'policy_reward_min': {}, 'policy_reward_max': {}, 'policy_reward_mean': {}, 'custom_metrics': {}, 'hist_stats': {'episode_reward': [10.0, 21.0, 20.0, 12.0, 15.0, 35.0, 16.0, 32.0, 21.0, 19.0, 72.0, 35.0, 32.0, 102.0, 76.0, 21.0, 15.0, 17.0, 15.0, 43.0, 25.0, 34.0, 16.0, 26.0, 71.0, 28.0, 52.0, 69.0, 10.0, 21.0, 12.0, 130.0, 56.0, 18.0, 35.0, 32.0, 108.0, 61.0, 97.0, 47.0, 31.0, 23.0, 18.0, 20.0, 59.0, 13.0, 69.0, 58.0, 38.0, 22.0, 83.0, 25.0, 22.0, 34.0, 53.0, 29.0, 18.0, 115.0, 25.0, 59.0, 20.0, 28.0, 64.0, 43.0, 40.0, 36.0, 93.0, 13.0, 20.0, 35.0, 29.0, 38.0, 10.0, 58.0, 26.0, 126.0, 17.0, 54.0, 35.0, 55.0, 35.0, 37.0, 110.0, 28.0, 41.0, 31.0, 45.0, 41.0, 20.0, 123.0, 17.0, 136.0, 41.0, 59.0, 51.0, 18.0, 14.0, 98.0, 35.0, 143.0], 'episode_lengths': [10, 21, 20, 12, 15, 35, 16, 32, 21, 19, 72, 35, 32, 102, 76, 21, 15, 17, 15, 43, 25, 34, 16, 26, 71, 28, 52, 69, 10, 21, 12, 130, 56, 18, 35, 32, 108, 61, 97, 47, 31, 23, 18, 20, 59, 13, 69, 58, 38, 22, 83, 25, 22, 34, 53, 29, 18, 115, 25, 59, 20, 28, 64, 43, 40, 36, 93, 13, 20, 35, 29, 38, 10, 58, 26, 126, 17, 54, 35, 55, 35, 37, 110, 28, 41, 31, 45, 41, 20, 123, 17, 136, 41, 59, 51, 18, 14, 98, 35, 143]}, 'sampler_perf': {'mean_raw_obs_processing_ms': 0.1494050133826096, 'mean_inference_ms': 0.3030270243531414, 'mean_action_processing_ms': 0.04484232997177898, 'mean_env_wait_ms': 0.020527675791925257, 'mean_env_render_ms': 0.0}, 'num_faulty_episodes': 0, 'connector_metrics': {'ObsPreprocessorConnector_ms': 0.0020761489868164062, 'StateBufferConnector_ms': 0.0016286373138427734, 'ViewRequirementAgentConnector_ms': 0.0386197566986084}} 7.14829 3.57113 7.14829{'training_iteration_time_ms': 3571.167, 'sample_time_ms': 1119.964, 'load_time_ms': 0.253, 'load_throughput': 16189340.451, 'learn_time_ms': 2449.34, 'learn_throughput': 1669.634, 'synch_weights_time_ms': 1.166} 1680195666 8179 26c451_00000
\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82020)\u001B[0m 2023-03-30 10:01:03,714\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\n", "\u001B[2m\u001B[36m(pid=82019)\u001B[0m Resource usage vs limits 0: 0%| | 0/1 [00:00\n", "
\n", "
\n", "

Tune Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "
Current time:2023-03-30 10:02:06
Running for: 00:00:14.18
Memory: 18.3/32.0 GiB
\n", "
\n", "
\n", "
\n", "

System Info

\n", " Using FIFO scheduling algorithm.
Logical resource usage: 0/8 CPUs, 0/0 GPUs\n", "
\n", " \n", "
\n", "
\n", "
\n", "

Trial Status

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name status loc iter total time (s) ts reward episode_reward_max episode_reward_min episode_len_mean
AIRBC_914d8_00000TERMINATED127.0.0.1:82274 5 0.82942920384 nan nan nan nan
\n", "
\n", "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:43: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:29: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _TENSORSHAPEPROTO_DIM = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:73: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:80: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:66: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _TENSORSHAPEPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:19: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:33: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:37: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:41: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:45: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:49: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:53: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:57: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:61: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:65: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:69: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:73: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:77: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:81: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:85: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:89: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:93: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:97: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:101: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:105: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:109: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:113: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:117: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:121: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:125: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:129: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:133: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:137: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:141: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:145: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:149: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:153: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:157: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:161: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:165: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:169: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:173: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:177: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:181: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:185: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:189: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:193: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:197: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:201: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:205: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:209: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:213: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:217: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.EnumValueDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:27: DeprecationWarning: Call to deprecated create function EnumDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _DATATYPE = _descriptor.EnumDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:287: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:280: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _SERIALIZEDDTYPE = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:20: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:39: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:46: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:32: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _RESOURCEHANDLEPROTO_DTYPEANDSHAPE = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:76: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:83: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:90: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:97: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:104: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:111: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py:69: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _RESOURCEHANDLEPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:40: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:47: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:54: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:61: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:68: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:75: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:89: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:96: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:103: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:110: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:117: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:124: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:131: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:138: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:145: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:152: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:33: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _TENSORPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:183: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:190: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:197: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:176: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _VARIANTTENSORDATAPROTO = _descriptor.Descriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:40: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:47: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:54: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:61: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:68: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:75: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m _descriptor.FieldDescriptor(\n", "\u001B[2m\u001B[36m(RolloutWorker pid=82020)\u001B[0m 2023-03-30 10:01:03,685\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[Repartition] -> TaskPoolMapOperator[Write]\u001B[32m [repeated 3x across cluster]\u001B[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:61: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m if (distutils.version.LooseVersion(tf.__version__) <\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:62: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m distutils.version.LooseVersion(required_tensorflow_version)):\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m WARNING:tensorflow:From /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py:561: calling function (from tensorflow.python.eager.polymorphic_function.polymorphic_function) with experimental_relax_shapes is deprecated and will be removed in a future version.\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m Instructions for updating:\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m experimental_relax_shapes is deprecated, use reduce_retracing instead\n", "\u001B[2m\u001B[36m(AIRBC pid=82274)\u001B[0m 2023-03-30 10:01:56,521\tWARNING algorithm_config.py:636 -- Cannot create BCConfig from given `config_dict`! Property __stdout_file__ not supported.\n", "\u001B[2m\u001B[36m(AIRBC pid=82274)\u001B[0m 2023-03-30 10:01:56,624\tINFO algorithm.py:527 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.\n", "\u001B[2m\u001B[36m(AIRBC pid=82274)\u001B[0m 2023-03-30 10:01:56,630\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> TaskPoolMapOperator[ReadJSON] -> AllToAllOperator[Repartition]\n", "\u001B[2m\u001B[36m(pid=82274)\u001B[0m Resource usage vs limits 0: 0%| | 0/1 [00:00 AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(AIRBC pid=82274)\u001B[0m 2023-03-30 10:02:05,550\tWARNING deprecation.py:50 -- DeprecationWarning: `remote_workers()` has been deprecated. Accessing the list of remote workers directly through remote_workers() is strongly discouraged. Please try to use one of the foreach accessors that is fault tolerant. This will raise an error in the future!\n" ] }, { "data": { "text/html": [ "
\n", "

Trial Progress

\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
Trial name agent_timesteps_totalconnector_metrics counters custom_metrics date done episode_len_meanepisode_media episode_reward_max episode_reward_mean episode_reward_min episodes_this_iter episodes_totalevaluation hostname info iterations_since_restorenode_ip num_agent_steps_sampled num_agent_steps_trained num_env_steps_sampled num_env_steps_sampled_this_iter num_env_steps_trained num_env_steps_trained_this_iter num_faulty_episodes num_healthy_workers num_in_flight_async_reqs num_remote_worker_restarts num_steps_trained_this_iterperf pidpolicy_reward_max policy_reward_mean policy_reward_min sampler_perf sampler_results time_since_restore time_this_iter_s time_total_stimers timestamp timesteps_total training_iterationtrial_id
AIRBC_914d8_00000 20384{} {'num_env_steps_sampled': 20384, 'num_env_steps_trained': 20384, 'num_agent_steps_sampled': 20384, 'num_agent_steps_trained': 20384}{} 2023-03-30_10-02-06True nan{} nan nan nan 0 0{'episode_reward_max': 76.0, 'episode_reward_min': 11.0, 'episode_reward_mean': 29.6, 'episode_len_mean': 29.6, 'episode_media': {}, 'episodes_this_iter': 10, 'policy_reward_min': {}, 'policy_reward_max': {}, 'policy_reward_mean': {}, 'custom_metrics': {}, 'hist_stats': {'episode_reward': [13.0, 13.0, 11.0, 76.0, 48.0, 36.0, 14.0, 34.0, 14.0, 37.0], 'episode_lengths': [13, 13, 11, 76, 48, 36, 14, 34, 14, 37]}, 'sampler_perf': {'mean_raw_obs_processing_ms': 0.12616764114882342, 'mean_inference_ms': 0.2590957092866947, 'mean_action_processing_ms': 0.04460762030537042, 'mean_env_wait_ms': 0.021067194550248115, 'mean_env_render_ms': 0.0}, 'num_faulty_episodes': 0, 'connector_metrics': {'ObsPreprocessorConnector_ms': 0.0019359588623046875, 'StateBufferConnector_ms': 0.0017905235290527344, 'ViewRequirementAgentConnector_ms': 0.03647565841674805}, 'num_agent_steps_sampled_this_iter': 296, 'num_env_steps_sampled_this_iter': 296, 'timesteps_this_iter': 296, 'num_healthy_workers': 1, 'num_in_flight_async_reqs': 0, 'num_remote_worker_restarts': 0}avnishs-mbp-3.lan{'learner': {'default_policy': {'learner_stats': {'allreduce_latency': 0.0, 'grad_gnorm': 0.0902162678539753, 'policy_loss': 0.6931167542934418, 'total_loss': 0.6931167542934418}, 'model': {}, 'custom_metrics': {}, 'num_agent_steps_trained': 2000.0, 'num_grad_updates_lifetime': 9.5, 'diff_num_grad_updates_vs_sampler_policy': 8.5}}, 'num_env_steps_sampled': 20384, 'num_env_steps_trained': 20384, 'num_agent_steps_sampled': 20384, 'num_agent_steps_trained': 20384} 5127.0.0.1 20384 20384 20384 4026 20384 4026 0 2 0 0 4026{'cpu_util_percent': 31.2, 'ram_util_percent': 57.0}82274{} {} {} {} {'episode_reward_max': nan, 'episode_reward_min': nan, 'episode_reward_mean': nan, 'episode_len_mean': nan, 'episode_media': {}, 'episodes_this_iter': 0, 'policy_reward_min': {}, 'policy_reward_max': {}, 'policy_reward_mean': {}, 'custom_metrics': {}, 'hist_stats': {'episode_reward': [], 'episode_lengths': []}, 'sampler_perf': {}, 'num_faulty_episodes': 0, 'connector_metrics': {}} 0.829429 0.184188 0.829429{'training_iteration_time_ms': 45.667, 'sample_time_ms': 26.685, 'load_time_ms': 0.193, 'load_throughput': 21115508.208, 'learn_time_ms': 17.555, 'learn_throughput': 232232.678, 'synch_weights_time_ms': 1.169} 1680195726 20384 5914d8_00000
\n", "
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "\u001B[2m\u001B[36m(pid=82313)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:21: DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 5x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m DESCRIPTOR = _descriptor.FileDescriptor(\u001B[32m [repeated 5x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py:82: DeprecationWarning: Call to deprecated create function FieldDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 40x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m _descriptor.FieldDescriptor(\u001B[32m [repeated 40x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py:176: DeprecationWarning: Call to deprecated create function Descriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 7x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m /Users/avnish/miniforge3/envs/ray/lib/python3.8/site-packages/tensorflow/core/framework/types_pb2.py:217: DeprecationWarning: Call to deprecated create function EnumValueDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.\u001B[32m [repeated 47x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m _descriptor.EnumValueDescriptor(\u001B[32m [repeated 47x across cluster]\u001B[0m\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "\u001B[2m\u001B[36m(pid=82313)\u001B[0m 2023-03-30 10:02:05,854\tINFO streaming_executor.py:83 -- Executing DAG InputDataBuffer[Input] -> AllToAllOperator[RandomShuffle]\n", "2023-03-30 10:02:06,881\tINFO tune.py:945 -- Total run time: 14.67 seconds (14.17 seconds for the tuning loop).\n" ] } ], "source": [ "result = train_rl_bc_offline(path=path, num_workers=2, use_gpu=False)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "71d7f318", "metadata": {}, "source": [ "And then, using the obtained checkpoint, we evaluate the policy on a fresh environment:" ] }, { "cell_type": "code", "execution_count": 12, "id": "53e412cc", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-03-30 10:02:11,829\tWARNING checkpoints.py:109 -- No `rllib_checkpoint.json` file found in checkpoint directory /var/folders/jr/6lgb7_ln64v1kppw9szl17rc0000gn/T/tmp5kq688t7! Trying to extract checkpoint info from other files found in that dir.\n", "2023-03-30 10:02:11,841\tINFO policy.py:1285 -- Policy (worker=local) running on CPU.\n", "2023-03-30 10:02:11,841\tINFO torch_policy_v2.py:110 -- Found 0 visible cuda devices.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Average reward over 3 episodes: 24.666666666666668\n" ] } ], "source": [ "num_eval_episodes = 3\n", "\n", "rewards = evaluate_using_checkpoint(result.checkpoint, num_episodes=num_eval_episodes)\n", "print(f\"Average reward over {num_eval_episodes} episodes: \" f\"{np.mean(rewards)}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e6b264f3", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "jupytext": { "cell_metadata_filter": "-all", "main_language": "python", "notebook_metadata_filter": "-all" }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 5 }