caffe2.python.modeling.gradient_clipping.GradientClipping Class Reference

Inheritance diagram for caffe2.python.modeling.gradient_clipping.GradientClipping:

Public Member Functions
def	__init__ (self, grad_clip_method, clip_norm_type, clip_threshold, use_parameter_norm=False, compute_norm_ratio=False)
def	modify_net (self, net, init_net=None, grad_map=None, blob_to_device=None)
Public Member Functions inherited from caffe2.python.modeling.net_modifier.NetModifier
def	__init__ (self)
def	modify_net (self, net, init_net=None, grad_map=None, blob_to_device=None)
def	__call__ (self, net, init_net=None, grad_map=None, blob_to_device=None)

Public Attributes
	grad_clip_method
	clip_norm_type
	clip_threshold
	use_parameter_norm
	compute_norm_ratio

Static Public Attributes
string	L1_NORM = 'l1_norm'
string	L2_NORM = 'l2_norm'
string	BY_NORM = 'by_norm'
list	GRAD_CLIP_METHODS = [BY_NORM]
list	CLIP_GRADIENT_NORM_TYPES = [L2_NORM, L1_NORM]

Detailed Description

Definition at line 16 of file gradient_clipping.py.

Constructor & Destructor Documentation

def caffe2.python.modeling.gradient_clipping.GradientClipping.__init__	(		self,
			grad_clip_method,
			clip_norm_type,
			clip_threshold,
			use_parameter_norm = False,
			compute_norm_ratio = False
	)

Clips gradient to avoid gradient magnitude explosion or vanishing gradient.

Args:
grad_clip_method: ways to clip the gradients
clip_norm_type: type of norm used in the necessary computation
clip_threshold: threshold used to determine whether to clip
use_parameter_norm: a boolean to indicate whether to incorporate
    the norm of the parameter
compute_norm_ratio: a boolean to compute the ratio between gradient norm
    and parameter norm explicitly for debugging purpose

Definition at line 27 of file gradient_clipping.py.

---

The documentation for this class was generated from the following file:

• caffe2/python/modeling/gradient_clipping.py