:github_url: https://github.com/ros-controls/ros2_control/blob/{REPOS_FILE_BRANCH}/controller_manager/doc/userdoc.rst .. _controller_manager_userdoc: Controller Manager ================== Controller Manager is the main component in the ros2_control framework. It manages lifecycle of controllers, access to the hardware interfaces and offers services to the ROS-world. ROS Interface ------------- Publishers ^^^^^^^^^^ ~/activity [controller_manager_msgs::msg::ControllerManagerActivity] A topic that is published every time there is a change of state of the controllers or hardware components managed by the controller manager. The message contains the list of the controllers and the hardware components that are managed by the controller manager along with their lifecycle states. The topic is published using the "transient local" quality of service, so subscribers should also be "transient local". Subscribers ^^^^^^^^^^^ robot_description [std_msgs::msg::String] String with the URDF xml, e.g., from ``robot_state_publisher``. Reloading of the URDF is not supported yet. All joints defined in the ````-tag have to be present in the URDF. Parameters ^^^^^^^^^^ .type Name of a plugin exported using ``pluginlib`` for a controller. This is a class from which controller's instance with name "``controller_name``" is created. .params_file The absolute path to the YAML file with parameters for the controller. The file should contain the parameters for the controller in the standard ROS 2 YAML format. .fallback_controllers List of controllers that are activated as a fallback strategy, when the spawned controllers fail by returning ``return_type::ERROR`` during the ``update`` cycle. It is recommended to add all the controllers needed for the fallback strategy to the list, including the chainable controllers whose interfaces are used by the main fallback controllers. .. warning:: The fallback controllers activation is subject to the availability of the state and command interfaces at the time of activation. It is recommended to test the fallback strategy in simulation before deploying it on the real robot. .. generate_parameter_library_details:: ../src/controller_manager_parameters.yaml parameters_context.yaml **An example parameter file:** .. generate_parameter_library_default:: ../src/controller_manager_parameters.yaml Determinism ----------- For best performance when controlling hardware you want the controller manager to have as little jitter as possible in the main control loop. Independent of the kernel installed, the main thread of Controller Manager attempts to configure ``SCHED_FIFO`` with a priority of ``50``. Read more about the scheduling policies `for example here `__. For real-time tasks, a priority range of 0 to 99 is expected, with higher numbers indicating higher priority. By default, users do not have permission to set such high priorities. To give the user such permissions, add a group named realtime and add the user controlling your robot to this group: .. code-block:: console $ sudo addgroup realtime $ sudo usermod -a -G realtime $(whoami) Afterwards, add the following limits to the realtime group in ``/etc/security/limits.conf``: .. code-block:: console @realtime soft rtprio 99 @realtime soft memlock unlimited @realtime hard rtprio 99 @realtime hard memlock unlimited The limits will be applied after you log out and in again. You can run ros2_control with real-time requirements also from a docker container. Pass the following capability options to allow the container to set the thread priority and lock memory, e.g., .. code-block:: console $ docker run -it \ --cap-add=sys_nice \ --ulimit rtprio=99 \ --ulimit memlock=-1 \ --rm --net host For more information, see the Docker engine documentation about `resource_constraints `__ and `linux capabilities `__. The normal linux kernel is optimized for computational throughput and therefore is not well suited for hardware control. Alternatives to the standard kernel include - `Real-time Ubuntu `_ on Ubuntu - `linux-raspi-realtime `_ on Ubuntu on Raspberry Pi: ``sudo apt install linux-raspi-realtime`` - `linux-image-rt-amd64 `__ or `linux-image-rt-arm64 `__ on Debian for 64-bit PCs - `lowlatency kernel `__ (``sudo apt install linux-lowlatency``) on any Ubuntu Though installing a realtime-kernel will definitely get the best results when it comes to low jitter, using a lowlatency kernel can improve things a lot with being really easy to install. .. note:: Avoid using the get_lifecycle_state() method in the real-time control loop of the controllers and the hardware components as it is not real-time safe. Factors that affect Determinism ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When run under the conditions determined in the above section, the determinism is assured up to the limitations of the hardware and the real-time kernel. However, there are some situations that can affect determinism: * When a controller fails to activate in the realtime loop, the controller_manager will call the methods ``prepare_command_mode_switch`` and ``perform_command_mode_switch`` to stop the started interfaces. These calls can cause jitter in the main control loop. * If a controller does not complete a successful update cycle in the realtime loop (for example, returns ``return_type::ERROR``), the controller manager will deactivate that controller (or) the entire controller chain it is part of. It will then invoke ``prepare_command_mode_switch`` and ``perform_command_mode_switch`` to stop the interfaces used by the affected controller(s). These actions can introduce jitter into the main control loop. Launching controller_manager ---------------------------- Using ros2_control_node ^^^^^^^^^^^^^^^^^^^^^^^ The controller_manager can be launched with the ros2_control_node executable. The following example shows how to launch the controller_manager with the ros2_control_node executable: .. code-block:: python control_node = Node( package="controller_manager", executable="ros2_control_node", parameters=[robot_controllers], output="both", ) The ros2_control_node executable uses the following parameters from the ``controller_manager`` node: lock_memory (optional; bool; default: false for a non-realtime kernel, true for a realtime kernel) Locks the memory of the ``controller_manager`` node at startup to physical RAM in order to avoid page faults and to prevent the node from being swapped out to disk. Find more information about the setup for memory locking in the following link: `How to set ulimit values `_ The following command can be used to set the memory locking limit temporarily: ``ulimit -l unlimited``. cpu_affinity (optional; int (or) int_array;) Sets the CPU affinity of the ``controller_manager`` node to the specified CPU core. If it is an integer, the node's affinity will be set to the specified CPU core. If it is an array of integers, the node's affinity will be set to the specified set of CPU cores. thread_priority (optional; int; default: 50) Sets the thread priority of the ``controller_manager`` node to the specified value. The value must be between 0 and 99. use_sim_time (optional; bool; default: false) Enables the use of simulation time in the ``controller_manager`` node. overruns.manage (optional; bool; default: true) Enables or disables the handling of overruns in the real-time loop of the ``controller_manager`` node. If set to true, the controller manager will detect overruns caused by system time changes or longer execution times of the controllers and hardware components. If an overrun is detected, the controller manager will print a warning message to the console. When used with ``use_sim_time`` set to true, this parameter is ignored and the overrun handling is disabled. hardware_synchronization.expect_blocking_read_write (optional; bool; default: false) If true, the controller manager will not sleep actively. Use this, when there is a hardware interface running that will block during its read or write operation until receiving new data from the hardware. hardware_synchronization.minimum_cycle_time (optional; double; default: 0.0001) The minimum sleep time in seconds for the control node's real-time loop. This is used to prevent the control node from running too fast, which can cause high CPU usage if the hardware doesn't block in read / write. This is only used when ``expect_blocking_read_write`` is set to true. If the cycle is shorter than this, it will sleep for this period and print a warning. Handling Multiple Controller Managers (single/multiple) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When dealing with multiple controller managers, you have two options for managing different robot descriptions: 1. **Using Namespaces:** You can place both the ``robot_state_publisher`` and the ``controller_manager`` nodes into the same namespace. .. code-block:: python control_node = Node( package="controller_manager", executable="ros2_control_node", parameters=[robot_controllers], output="both", namespace="rrbot", ) robot_state_pub_node = Node( package="robot_state_publisher", executable="robot_state_publisher", output="both", parameters=[robot_description], namespace="rrbot", ) 2. **Using Remappings:** You can use remappings to handle different robot descriptions. This involves relaying topics using the ``remappings`` tag, allowing you to specify custom topics for each controller manager. .. code-block:: python control_node = Node( package="controller_manager", executable="ros2_control_node", parameters=[robot_controllers], output="both", remappings=[('robot_description', '/rrbot/robot_description')] ) robot_state_pub_node = Node( package="robot_state_publisher", executable="robot_state_publisher", output="both", parameters=[robot_description], namespace="rrbot", ) Using the Controller Manager in a Process ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The ``ControllerManager`` may also be instantiated in a process as a class, but proper care must be taken when doing so. The reason for this is because the ``ControllerManager`` class inherits from ``rclcpp::Node``. If there is more than one Node in the process, global node name remap rules can forcibly change the ``ControllerManager's`` node name as well, leading to duplicate node names. This occurs whether the Nodes are siblings or exist in a hierarchy. .. image:: images/global_general_remap.png The workaround for this is to specify another node name remap rule in the ``NodeOptions`` passed to the ``ControllerManager`` node (causing it to ignore the global rule), or ensure that any remap rules are targeted to specific nodes. .. image:: images/global_specific_remap.png .. TODO: (methylDragon) Update the proposed solution when https://github.com/ros2/ros2/issues/1377 is resolved .. code-block:: cpp auto options = controller_manager::get_cm_node_options(); options.arguments({ "--ros-args", "--remap", "_target_node_name:__node:=dst_node_name", "--log-level", "info"}); auto cm = std::make_shared( executor, "_target_node_name", "some_optional_namespace", options); Helper Scripts -------------- There are two scripts to interact with controller manager from launch files: 1. ``spawner`` - loads, configures and start a controller on startup. 2. ``unspawner`` - stops and unloads a controller. 3. ``hardware_spawner`` - activates and configures a hardware component. ``spawner`` ^^^^^^^^^^^ .. code-block:: console $ ros2 run controller_manager spawner -h usage: spawner [-h] [-c CONTROLLER_MANAGER] [-p PARAM_FILE] [-n NAMESPACE] [--load-only] [--inactive] [-u] [--controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT] [--switch-timeout SWITCH_TIMEOUT] [--service-call-timeout SERVICE_CALL_TIMEOUT] [--activate-as-group] [--switch-asap | --no-switch-asap] [--controller-ros-args CONTROLLER_ROS_ARGS] [--controller [CONTROLLER ...]] controller_names [controller_names ...] positional arguments: controller_names List of controllers options: -h, --help show this help message and exit -c CONTROLLER_MANAGER, --controller-manager CONTROLLER_MANAGER Name of the controller manager ROS node -p PARAM_FILE, --param-file PARAM_FILE Controller param file to be loaded into controller node before configure. Pass multiple times to load different files for different controllers or to override the parameters of the same controller. -n NAMESPACE, --namespace NAMESPACE DEPRECATED Namespace for the controller_manager and the controller(s) --load-only Only load the controller and leave unconfigured. --inactive Load and configure the controller, however do not activate them -u, --unload-on-kill Wait until this application is interrupted (SIGINT or SIGTERM) and deactivate/unload controllers --controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT Time to wait for the controller manager service to be available --switch-timeout SWITCH_TIMEOUT Time to wait for a successful state switch of controllers. Useful when switching cannot be performed immediately, e.g., paused simulations at startup --service-call-timeout SERVICE_CALL_TIMEOUT Time to wait for the service response from the controller manager --activate-as-group Activates all the parsed controllers list together instead of one by one. Useful for activating all chainable controllers altogether --switch-asap, --no-switch-asap Option to switch the controllers in the realtime loop at the earliest possible time or in the non-realtime loop. --controller-ros-args CONTROLLER_ROS_ARGS The --ros-args to be passed to the controller node, e.g., for remapping topics. Pass multiple times for every argument. When launching ``spawner`` with ROS parameter files that use substitutions (for example, launch ``allow_substs=True``), the resolved ``--params-file`` path(s) used by the spawner node are automatically forwarded to each controller along with any explicit ``--param-file`` arguments passed to the spawner command. .. note:: If a single parameter file is used for multiple controllers, the spawner will automatically forward the resolved path(s) to each controller. The following methods are recommended: .. code-block:: console Node( package="controller_manager", executable="spawner", arguments=[ "my_controller", "--param-file", PathSubstitution(FindPackageShare("my_config_pkg")) / "config" / "controllers.yaml", ], ), (or) .. code-block:: console Node( package="controller_manager", executable="spawner", parameters=[ ParameterFile( PathSubstitution(FindPackageShare("my_config_pkg")) / "config" / "controllers.yaml", ), ], arguments=[ "my_controller" ], ), The ``spawner`` now supports per controller arguments, while parsing the arguments for multiple controllers using ``--controller`` option. For example, to spawn two controllers with different parameter files and remapping topics, the following command can be used: .. code-block:: console $ ros2 run controller_manager spawner --controller position_trajectory_controller \ --param-file /path/to/position_trajectory_controller_params.yaml \ --controller-ros-args "--ros-args --remap /joint_states:=/rrbot/joint_states" \ --controller velocity_trajectory_controller \ --param-file /path/to/velocity_trajectory_controller_params.yaml \ --controller-ros-args "--ros-args --remap /joint_states:=/rrbot/joint_states" .. code-block:: console $ ros2 run controller_manager spawner --controller -h Usage: spawner [global_options] --controller [controller_options] --controller ... Global Options: usage: spawner [-c CONTROLLER_MANAGER] [--controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT] [--switch-timeout SWITCH_TIMEOUT] [--service-call-timeout SERVICE_CALL_TIMEOUT] [--activate-as-group] [--switch-asap | --no-switch-asap] [-u] [-h] options: -c CONTROLLER_MANAGER, --controller-manager CONTROLLER_MANAGER Name of the controller manager --controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT Timeout for controller manager services --switch-timeout SWITCH_TIMEOUT Timeout for switch controller service --service-call-timeout SERVICE_CALL_TIMEOUT Timeout for service calls --activate-as-group Activate controllers as a group --switch-asap, --no-switch-asap Switch controllers as soon as possible -u, --unload-on-kill Deactivate the active controllers and unload them on SIGINT or SIGTERM -h, --help Show help Controller Options: usage: spawner [-p PARAM_FILE] [--load-only] [--inactive] [--controller-ros-args CONTROLLER_ROS_ARGS] controller_name positional arguments: controller_name Name of the controller options: -p PARAM_FILE, --param-file PARAM_FILE Parameter files to load for the controller --load-only Load the controller but do not configure/activate it --inactive Configure the controller but do not switch it --controller-ros-args CONTROLLER_ROS_ARGS ROS arguments to pass to the controller The parsed controller config file can follow the same conventions as the typical ROS 2 parameter file format. Now, the spawner can handle config files with wildcard entries and also the controller name in the absolute namespace. See the following examples on the config files: .. code-block:: yaml /**: ros__parameters: type: joint_trajectory_controller/JointTrajectoryController command_interfaces: - position ..... position_trajectory_controller_joint1: ros__parameters: joints: - joint1 position_trajectory_controller_joint2: ros__parameters: joints: - joint2 .. code-block:: yaml /**/position_trajectory_controller: ros__parameters: type: joint_trajectory_controller/JointTrajectoryController joints: - joint1 - joint2 command_interfaces: - position ..... .. code-block:: yaml /position_trajectory_controller: ros__parameters: type: joint_trajectory_controller/JointTrajectoryController joints: - joint1 - joint2 command_interfaces: - position ..... .. code-block:: yaml position_trajectory_controller: ros__parameters: type: joint_trajectory_controller/JointTrajectoryController joints: - joint1 - joint2 command_interfaces: - position ..... .. code-block:: yaml /rrbot_1/position_trajectory_controller: ros__parameters: type: joint_trajectory_controller/JointTrajectoryController joints: - joint1 - joint2 command_interfaces: - position ..... ``unspawner`` ^^^^^^^^^^^^^ .. code-block:: console $ ros2 run controller_manager unspawner -h usage: unspawner [-h] [-c CONTROLLER_MANAGER] [--switch-timeout SWITCH_TIMEOUT] controller_names [controller_names ...] positional arguments: controller_names Name of the controller options: -h, --help show this help message and exit -c CONTROLLER_MANAGER, --controller-manager CONTROLLER_MANAGER Name of the controller manager ROS node --switch-timeout SWITCH_TIMEOUT Time to wait for a successful state switch of controllers. Useful if controllers cannot be switched immediately, e.g., paused simulations at startup ``hardware_spawner`` ^^^^^^^^^^^^^^^^^^^^ .. code-block:: console $ ros2 run controller_manager hardware_spawner -h usage: hardware_spawner [-h] [-c CONTROLLER_MANAGER] [--controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT] (--activate | --configure) hardware_component_names [hardware_component_names ...] positional arguments: hardware_component_names The name of the hardware components which should be activated. options: -h, --help show this help message and exit -c CONTROLLER_MANAGER, --controller-manager CONTROLLER_MANAGER Name of the controller manager ROS node --controller-manager-timeout CONTROLLER_MANAGER_TIMEOUT Time to wait for the controller manager --activate Activates the given components. Note: Components are by default configured before activated. --configure Configures the given components. Color Output Handling ^^^^^^^^^^^^^^^^^^^^^ The helper scripts (``spawner`` and ``hardware_spawner``) now use an environment-aware ``bcolors`` class. The color output automatically adapts to the environment: * ``RCUTILS_COLORIZED_OUTPUT=0`` -> disables color output * ``RCUTILS_COLORIZED_OUTPUT=1`` -> forces color output * Unset -> automatically detects TTY and enables color only in interactive terminals Utility scripts --------------- In addition to the ``spawner``, ``unspawner``, and ``hardware_spawner`` helper scripts, the controller_manager package provides utility modules for programmatic interaction with the controller manager. Programmatic controller lifecycle (recommended) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Prefer the high-level service wrapper functions in ``controller_manager.controller_manager_services`` for lifecycle scripts (e.g. ``load_controller``, ``configure_controller``, ``switch_controllers``, ``cleanup_controller``, ``unload_controller``) instead of exposing internal service-client singletons. **Example lifecycle script:** .. code-block:: python import rclpy from controller_manager.controller_manager_services import ( load_controller, configure_controller, switch_controllers, cleanup_controller, unload_controller ) from controller_manager_msgs.srv import SwitchController def main(): rclpy.init() node = rclpy.create_node('controller_ops') cm_name = 'controller_manager' try: resp = load_controller(node, cm_name, 'my_controller') if getattr(resp, 'success', False): node.get_logger().info("Controller loaded") resp = configure_controller(node, cm_name, 'my_controller') if getattr(resp, 'ok', getattr(resp, 'success', False)): node.get_logger().info("Controller configured") resp = switch_controllers( node, cm_name, deactivate_controllers=[], activate_controllers=['my_controller'], strictness=SwitchController.Request.STRICT, activate_asap=False, timeout=0.0 ) if getattr(resp, 'ok', False): node.get_logger().info('Controller activated') finally: switch_controllers( node, cm_name, deactivate_controllers=['my_controller'], activate_controllers=[], strictness=SwitchController.Request.STRICT, activate_asap=False, timeout=0.0 ) cleanup_controller(node, cm_name, 'my_controller') unload_controller(node, cm_name, 'my_controller') node.destroy_node() rclpy.shutdown() if __name__ == '__main__': main() Notes: - Import path: controller_manager.controller_manager_services (matches repository layout). - Response fields differ between services; examples use getattr to handle ``success`` vs ``ok``. launch_utils ^^^^^^^^^^^^ The ``launch_utils`` module provides helper functions to generate launch descriptions for loading controllers. These functions simplify launch file creation by reducing boilerplate code. **generate_controllers_spawner_launch_description()** Generates a launch description that automatically loads and activates controllers using the spawner utility. **Function Signature:** .. code-block:: python def generate_controllers_spawner_launch_description( controller_names: list, controller_params_files=None, extra_spawner_args=[] ) -> LaunchDescription **Python Usage Example:** .. code-block:: python from controller_manager.launch_utils import generate_controllers_spawner_launch_description from launch import LaunchDescription from launch.substitutions import PathSubstitution from launch_ros.substitutions import FindPackageShare def generate_launch_description(): config_dir = PathSubstitution(FindPackageShare('my_robot_bringup')) / 'config' # Example 1: Load with controller parameters already in controller_manager spawner = generate_controllers_spawner_launch_description( ['joint_state_broadcaster'] ) # Example 2: Load with parameter file and extra arguments spawner = generate_controllers_spawner_launch_description( ['joint_trajectory_controller'], controller_params_files=[config_dir / 'controllers.yaml'], extra_spawner_args=['--load-only'] ) return spawner **Parameters:** - ``controller_names`` (list[str]): Names of controllers to load. Example: ``['joint_state_broadcaster', 'joint_trajectory_controller']`` - ``controller_params_files`` (list, optional): Paths to YAML parameter files for controllers. Each item can be a plain string path or a Launch substitution (e.g., ``PathJoinSubstitution([...])``). Defaults to ``None`` - ``extra_spawner_args`` (list[str], optional): Additional arguments to pass to spawner (e.g., ``['--load-only', '--inactive']``). Defaults to ``[]`` **Return Value:** Returns a ``LaunchDescription`` containing: - Launch argument for ``controller_manager_name`` (default: "controller_manager") - allows overriding the controller manager node name - Launch argument for ``unload_on_kill`` (default: "false") - waits for interrupt signal to unload controllers - Spawner node that loads and activates the specified controllers **Complete Launch File Example:** .. code-block:: python from launch import LaunchDescription from launch_ros.actions import Node from controller_manager.launch_utils import generate_controllers_spawner_launch_description from launch.substitutions import PathSubstitution from launch_ros.substitutions import FindPackageShare def generate_launch_description(): # Get configuration paths config_dir = PathSubstitution(FindPackageShare('my_robot_bringup')) / 'config' # Create controller manager node control_node = Node( package='controller_manager', executable='ros2_control_node', parameters=[config_dir / 'ros2_controllers.yaml'], output='both', ) # Create spawner for multiple controllers controllers_spawner = generate_controllers_spawner_launch_description( ['joint_state_broadcaster', 'joint_trajectory_controller'], controller_params_files=[config_dir / 'controllers.yaml'], ) return LaunchDescription([ control_node, controllers_spawner, ]) **Launching:** .. code-block:: console $ ros2 launch my_robot_bringup my_robot.launch.py [controller_manager-1] [INFO] [ros2_control_node]: Loaded controller 'joint_state_broadcaster' [spawner-1] [INFO] [spawner]: Configured and activated 'joint_state_broadcaster' [spawner-1] [INFO] [spawner]: Configured and activated 'joint_trajectory_controller' **generate_controllers_spawner_launch_description_from_dict()** Provides an alternative way to specify controllers using a dictionary format, allowing different parameter files per controller. **Function Signature:** .. code-block:: python def generate_controllers_spawner_launch_description_from_dict( controller_info_dict: dict, extra_spawner_args=[] ) -> LaunchDescription **Parameters:** - ``controller_info_dict`` (dict[str, str | list[str] | None]): Dictionary with controller names as keys and parameter file path(s) as values. Each value may be a ``str`` (single YAML), ``list[str]`` (multiple YAMLs applied in order), ``[]`` (no parameter file), or ``None`` if no parameter file is needed - ``extra_spawner_args`` (list[str], optional): Additional arguments to pass to spawner. Defaults to ``[]`` **Python Usage Example:** .. code-block:: python from controller_manager.launch_utils import generate_controllers_spawner_launch_description_from_dict from launch import LaunchDescription from launch.substitutions import PathSubstitution from launch_ros.substitutions import FindPackageShare def generate_launch_description(): config_dir = PathSubstitution(FindPackageShare('my_robot_bringup')) / 'config' # Define controllers with per-controller configurations controller_info_dict = { 'joint_state_broadcaster': config_dir / 'common_params.yaml', 'position_trajectory_controller': [ config_dir / 'common_params.yaml', config_dir / 'position_controller.yaml', ], 'velocity_trajectory_controller': config_dir / 'velocity_controller.yaml', } spawner = generate_controllers_spawner_launch_description_from_dict( controller_info_dict, extra_spawner_args=['--load-only'] ) return LaunchDescription([spawner]) **Use Cases:** - Define different parameter files for different controllers - Apply common parameter files across multiple controllers - Override controller parameters with multiple YAML files - Simplify complex launch file configurations rqt_controller_manager ---------------------- A GUI tool to interact with the controller manager services to be able to switch the lifecycle states of the controllers as well as the hardware components. .. image:: images/rqt_controller_manager.png It can be launched independently using the following command or as rqt plugin: .. code-block:: console ros2 run rqt_controller_manager rqt_controller_manager * Double-click on a controller or hardware component to show the additional info. * Right-click on a controller or hardware component to show a context menu with options for lifecycle management. Operations ---------- Restarting all controllers ^^^^^^^^^^^^^^^^^^^^^^^^^^ The simplest way to restart all controllers is by using ``switch_controllers`` services or CLI and adding all controllers to ``start`` and ``stop`` lists. Note that not all controllers have to be restarted, e.g., broadcasters. Restarting hardware ^^^^^^^^^^^^^^^^^^^ If hardware gets restarted then you should go through its lifecycle again in order to reconfigure and export the interfaces Hardware and Controller Errors ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If the hardware during it's ``read`` or ``write`` method returns ``return_type::ERROR``, the controller manager will stop all controllers that are using the hardware's command and state interfaces. Likewise, if a controller returns ``return_type::ERROR`` from its ``update`` method, the controller manager will deactivate the respective controller (or) the entire controller chain it is part of, then the controller manager will try to start any available fallback controllers.