o ?e6@sdZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd lmZd d ZGd dde ZGddde ZGddde ZGddde ZddZGddde ZGddde ZdS)z(Layers that act as activation functions.)dtypes)backend) constraints) initializers) regularizers)Layer) InputSpec)tf_utils)math_opscCstSN)globalsr r t/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/keras/layers/advanced_activations.py get_globalssrcDeZdZdZd fdd ZddZfddZejd d Z Z S) LeakyReLUaLeaky version of a Rectified Linear Unit. It allows a small gradient when the unit is not active: ``` f(x) = alpha * x if x < 0 f(x) = x if x >= 0 ``` Usage: >>> layer = tf.keras.layers.LeakyReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.9, -0.3, 0.0, 2.0] >>> layer = tf.keras.layers.LeakyReLU(alpha=0.1) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.3, -0.1, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Float >= 0. Negative slope coefficient. Default to 0.3. 333333?c @tt|jdi||durtd|d|_t||_dS)NzKThe alpha value of a Leaky ReLU layer cannot be None, needs a float. Got %sTr )superr__init__ ValueErrorsupports_maskingrcast_to_floatxalphaselfrkwargs __class__r rrCszLeakyReLU.__init__cCstj||jdS)N)rrrelurrinputsr r rcallLszLeakyReLU.callc8dt|ji}tt|}tt|t|SNr)floatrrr get_configdictlistitemsrconfigZ base_configrr rr'OzLeakyReLU.get_configcC|Sr r r input_shaper r rcompute_output_shapeTzLeakyReLU.compute_output_shape)r __name__ __module__ __qualname____doc__rr#r'r shape_type_conversionr1 __classcell__r r rrr!s! rcsZeZdZdZ    dfdd ZejddZdd Zfd d Z ejd d Z Z S)PReLUaParametric Rectified Linear Unit. It follows: ``` f(x) = alpha * x for x < 0 f(x) = x for x >= 0 ``` where `alpha` is a learned array with the same shape as x. Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha_initializer: Initializer function for the weights. alpha_regularizer: Regularizer for the weights. alpha_constraint: Constraint for the weights. shared_axes: The axes along which to share learnable parameters for the activation function. For example, if the incoming feature maps are from a 2D convolution with output shape `(batch, height, width, channels)`, and you wish to share parameters across space so that each filter only has one set of parameters, set `shared_axes=[1, 2]`. zerosNc sztt|jdi|d|_t||_t||_t ||_ |dur)d|_ dSt |t tfs6|g|_ dSt ||_ dSNTr )rr:rrrgetalpha_initializerralpha_regularizerralpha_constraint shared_axes isinstancer)tuple)rr>r?r@rArrr rr{s     zPReLU.__init__cCst|dd}|jdur|jD]}d||d<q|j|d|j|j|jd|_i}|jrAtdt|D] }||jvr@||||<q3t t||d|_ d|_ dS)Nr)shapenameZ initializerZ regularizer constraint)ndimaxesT) r)rAZ add_weightr>r?r@rrangelenrZ input_specZbuilt)rr0Z param_shapeirIr r rbuilds&     z PReLU.buildcCs&t|}|j t| }||Sr r)rr"posnegr r rr#s z PReLU.callcsRt|jt|jt|j|jd}tt | }t t | t | S)N)r>r?r@rA)r serializer>rr?rr@rArr:r'r(r)r*r+rr rr's   zPReLU.get_configcCr.r r r/r r rr1r2zPReLU.compute_output_shape)r;NNN) r4r5r6r7rr r8rMr#r'r1r9r r rrr:Ys"   r:cr) ELUaExponential Linear Unit. It follows: ``` f(x) = alpha * (exp(x) - 1.) for x < 0 f(x) = x for x >= 0 ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Scale for the negative factor. ?c r)Nz>Alpha of an ELU layer cannot be None, requires a float. Got %sTr )rrQrrrrrrrrr rrsz ELU.__init__cCst||jSr )rZelurr!r r rr#szELU.callcr$r%)r&rrrQr'r(r)r*r+rr rr'r-zELU.get_configcCr.r r r/r r rr1r2zELU.compute_output_shaperRr3r r rrrQs rQcr) ThresholdedReLUaThresholded Rectified Linear Unit. It follows: ``` f(x) = x for x > theta f(x) = 0 otherwise` ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: theta: Float >= 0. Threshold location of activation. rRc sTtt|jdi||durtd||dkrtd|d|_t||_dS)NzJTheta of a Thresholded ReLU layer cannot be None, requires a float. Got %srzAThe theta value of a Thresholded ReLU layer should be >=0, got %sTr )rrTrrrrrtheta)rrUrrr rrszThresholdedReLU.__init__cCs*t|j|j}|tt|||jSr )r castrUdtypeZgreater)rr"rUr r rr#szThresholdedReLU.callcr$)NrU)r&rUrrTr'r(r)r*r+rr rr'r-zThresholdedReLU.get_configcCr.r r r/r r rr1 r2z$ThresholdedReLU.compute_output_shaperSr3r r rrrTs rTcCs|tjkr tjjSdS)a Large negative number as Tensor. This function is necessary because the standard value for epsilon in this module (-1e9) cannot be represented using tf.float16 Args: tensor_type: a dtype to determine the type. Returns: a large negative number. ge)rZfloat16min)Z tensor_typer r r_large_compatible_negatives rYcsFeZdZdZd fdd Zd ddZfdd Zejd d Z Z S)SoftmaxaSoftmax activation function. Example without mask: >>> inp = np.asarray([1., 2., 1.]) >>> layer = tf.keras.layers.Softmax() >>> layer(inp).numpy() array([0.21194157, 0.5761169 , 0.21194157], dtype=float32) >>> mask = np.asarray([True, False, True], dtype=bool) >>> layer(inp, mask).numpy() array([0.5, 0. , 0.5], dtype=float32) Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: axis: Integer, or list of Integers, axis along which the softmax normalization is applied. Call arguments: inputs: The inputs, or logits to the softmax layer. mask: A boolean mask of the same shape as `inputs`. Defaults to `None`. The mask specifies 1 to keep and 0 to mask. Returns: softmaxed output with the same shape as `inputs`. c s&tt|jdi|d|_||_dSr<)rrZrraxis)rr\rrr rrBs zSoftmax.__init__NcCs|durdt||jt|j}||7}t|jttfr=t|jdkr3t |tj ||jddSt j ||jddSt j ||jdS)NrRrDT)r\Zkeepdimsr)r\) r rVrWrYrBr\rCr)rKexpZreduce_logsumexprZsoftmax)rr"maskZadderr r rr#Gs  z Softmax.callcs4d|ji}tt|}tt|t|S)Nr\)r\rrZr'r(r)r*r+rr rr'Zs zSoftmax.get_configcCr.r r r/r r rr1_r2zSoftmax.compute_output_shape)r[r r3r r rrrZ!s  rZcsDeZdZdZd fdd ZddZfdd Zejd d Z Z S) ReLUaYRectified Linear Unit activation function. With default values, it returns element-wise `max(x, 0)`. Otherwise, it follows: ``` f(x) = max_value if x >= max_value f(x) = x if threshold <= x < max_value f(x) = negative_slope * (x - threshold) otherwise ``` Usage: >>> layer = tf.keras.layers.ReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(max_value=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 1.0] >>> layer = tf.keras.layers.ReLU(negative_slope=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-3.0, -1.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(threshold=1.5) >>> output = layer([-3.0, -1.0, 1.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: max_value: Float >= 0. Maximum activation value. Default to None, which means unlimited. negative_slope: Float >= 0. Negative slope coefficient. Default to 0. threshold: Float >= 0. Threshold value for thresholded activation. Default to 0. Nrc stt|jdi||dur|dkrtd||dus!|dkr'td||dus/|dkr5td|d|_|durAt|}||_t||_t||_ dS)Ngz=max_value of a ReLU layer cannot be a negative value. Got: %szBnegative_slope of a ReLU layer cannot be a negative value. Got: %sz=threshold of a ReLU layer cannot be a negative value. Got: %sTr ) rr_rrrrr max_valuenegative_slope threshold)rr`rarbrrr rrs&  z ReLU.__init__cCstj||j|j|jdS)N)rr`rb)rr rar`rbr!r r rr#s z ReLU.callcs<|j|j|jd}tt|}tt|t|S)N)r`rarb) r`rarbrr_r'r(r)r*r+rr rr's zReLU.get_configcCr.r r r/r r rr1r2zReLU.compute_output_shape)Nrrr3r r rrr_ds/  r_N)r7Ztensorflow.python.frameworkrZtensorflow.python.kerasrrrrZ)tensorflow.python.keras.engine.base_layerrZ)tensorflow.python.keras.engine.input_specrZtensorflow.python.keras.utilsr Ztensorflow.python.opsr rrr:rQrTrYrZr_r r r rs$         8]+/C