o 7?e&@sdZddlZddlmmZddlmZdZdZ dZ dZ dZ dZ d Zd Zd Zd d ZGdddZd%ddZddZddZddZddZddZddZdd Zd!d"Zd#d$ZdS)&z0Utilities used by both the GRU and LSTM classes.N)get_device_nameZapi_implementsZapi_preferred_deviceZCPUZGPUz4Layer %s will use cuDNN kernels when running on GPU.zLayer %s will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.cCsdS)NFrrrd/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/keras/src/layers/rnn/gru_lstm_utils.pyuse_new_gru_lstm_impl1src@s eZdZdZddZddZdS) DefunWrapperz;A wrapper with no deep copy of the Defun in LSTM/GRU layer.c Cs||_||_||_|jdvrtd|jd|jd|jt|jdtti}|jdkr8ddl m }|j }n dd l m }|j }tj|d |d |_dS) N)lstmgruz>Defun wrapper only applies to LSTM and GRU layer, but given {} time_major go_backwards_r r)r )r FZ autographZexperimental_attributes)r r layer_name ValueErrorformat_FUNCTION_API_NAME_ATTRIBUTEstruuiduuid4Zkeras.src.layers.rnnr Zlstm_with_backend_selectionr Zgru_with_backend_selectiontffunctionZ defun_layer)selfr r rsupportive_attributesr Z layer_funcr rrr__init__:s4      zDefunWrapper.__init__cCs&t||j|j|j}||t|<|SN)typer r rid)rmemoZ new_wrapperrrr __deepcopy__^s   zDefunWrapper.__deepcopy__N)__name__ __module__ __qualname____doc__rrrrrrr7s $rFcsDfddfdd|D}fdd|D}tj||ddS)aUtility function convert variable to cuDNN compatible parameter. Note that Keras weights for kernels are different from the cuDNN format. Eg.: ``` Keras cuDNN [[0, 1, 2], <---> [[0, 2, 4], [3, 4, 5]] [1, 3, 5]] ``` If the input weights need to be in a unified format, then set `transpose_weights=True` to convert the weights. Args: weights: list of weights for the individual kernels and recurrent kernels. biases: list of biases for individual gate. shape: the shape for the converted variables that will be feed to cuDNN. transpose_weights: boolean, whether to transpose the weights. Returns: The converted weights that can be feed to cuDNN ops as param. csrt|S|Sr)r transpose)w)transpose_weightsrrconvertsz$canonical_to_params..convertcsg|] }t|qSrrZreshape.0x)r'shaperr sz'canonical_to_params..csg|]}t|qSrr(r))r,rrr-srZaxis)rconcat)weightsZbiasesr,r&r)r'r,r&rcanonical_to_paramsfs r1cCsFt|d}tjt|tjdd}tj||d}tt||S)a_Check the mask tensor and see if it right padded. For cuDNN kernel, it uses the sequence length param to skip the tailing timestep. If the data is left padded, or not a strict right padding (has masked value in the middle of the sequence), then cuDNN kernel won't be work properly in those cases. Left padded data: [[False, False, True, True, True]]. Right padded data: [[True, True, True, False, False]]. Mixture of mask/unmasked data: [[True, False, True, False, False]]. Note that for the mixed data example above, the actually data RNN should see are those 2 Trues (index 0 and 2), the index 1 False should be ignored and not pollute the internal states. Args: mask: the Boolean tensor with shape [batch, timestep] Returns: boolean scalar tensor, whether the mask is strictly right padded. rr.)maxlen)rr, reduce_sumcastint32Z sequence_mask reduce_allequal)maskZmax_seq_lengthZ count_of_trueZright_padded_maskrrris_sequence_right_paddedsr9cCsttjt|ddS)Nrr.)rZ reduce_anyr6 logical_not)r8rrrhas_fully_masked_sequencesr;cCstjdr+|s|durdS|durt|S|dur)tjt|t|SdS|dur1dS|r8t|}t t |t t |S)NZ is_rocm_buildFT) r sysconfigZget_build_infor6mathr7Z reduce_minZ reduce_maxr$ logical_andr9r:r;)r8r Zsequence_lengthsrrris_cudnn_supported_inputss$    r?cCs$|rdnd}tjt|tj|dS)aOCalculate the sequence length tensor (1-D) based on the masking tensor. The masking tensor is a 2D boolean tensor with shape [batch, timestep]. For any timestep that should be masked, the corresponding field will be False. Consider the following example: a = [[True, True, False, False], [True, True, True, False]] It is a (2, 4) tensor, and the corresponding sequence length result should be 1D tensor with value [2, 3]. Note that the masking tensor must be right padded that could be checked by, e.g., `is_sequence_right_padded()`. Args: mask: Boolean tensor with shape [batch, timestep] or [timestep, batch] if time_major=True. time_major: Boolean, which indicates whether the mask is time major or batch major. Returns: sequence_length: 1D int32 tensor. rrr.)rr3r4r5)r8r Ztimestep_indexrrrcalculate_sequence_by_masks r@cCs&t|t|i}||tj|d|dS)NFr)r_FUNCTION_DEVICE_ATTRIBUTEupdaterr)Zunique_api_nameZpreferred_devicefuncrZfunction_attributesrrrgenerate_defun_backends rDcCs$t}|dur dStjjj|jS)zAParse the current context and return the device type, eg CPU/GPU.N)rrcompatv1Z DeviceSpecZ from_stringZ device_type)Zcurrent_devicerrrget_context_device_typesrGcCs@tdtj|tjddWdS1swYdS)Nz/cpu:0runtime)Zdtypename)rZdeviceZconstantZfloat32)Z runtime_namerrrrHs $rHcCst|tjr |S|S)z/Read the value of a variable if it is variable.) isinstancerVariableZ read_value)vrrrread_variable_values rMcOs$|j|i|}|||S)aRegister a specialization of a `Function` into the graph. This won't actually call the function with the inputs, and only put the function definition into graph. Register function with different input param will result into multiple version of functions registered in graph. Args: func: the `Function` instance that generated by a @defun *args: input arguments for the Python function. **kwargs: input keyword arguments for the Python function. Returns: a `ConcreteFunction` object specialized to inputs and execution context. Raises: ValueError: When the input function is not a defun wrapped python function. )Zget_concrete_functionZ add_to_graphZadd_gradient_functions_to_graph)rCargskwargsZ concrete_funcrrrfunction_registersrP)F)r#rZtensorflow.compat.v2rEv2rZtensorflow.python.eager.contextrrrAZCPU_DEVICE_NAMEZGPU_DEVICE_NAMEZRUNTIME_UNKNOWNZ RUNTIME_CPUZ RUNTIME_GPUZCUDNN_AVAILABLE_MSGZCUDNN_NOT_AVAILABLE_MSGrrr1r9r;r?r@rDrGrHrMrPrrrrs4  /!