o ?eH@s|dZddlZddlmZddlmZddlmZddlmZ ddlm Z ddlm Z dd lm Z dd lm Zdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddl!m"Z"dZ#dZ$dbddZ%e&d e&d!d"d#Z' dcd$d%Z(d&d'Z)d(d)Z*d*d+Z+d,d-Z,d.d/Z-d0d1Z.d2d3Z/d4d5Z0d6d7Z1d8d9Z2d:d;Z3dd?Z5d@dAZ6dBdCZ7dDdEZ8dFdGZ9dHdIZ:dJdKZ;dLdMZdRdSZ?dTdUZ@GdVdWdWejAZB X dddYdXZCe&dZe&d[d\d]ZD  ded^d_ZEd`daZFdS)fzcond_v2 and gradient. This is a version of cond that emits a single If op, as well as the gradient function for If ops produced by cond_v2. This will eventually replace the current tf.cond implementation once it reaches feature and performance parity. N) types_pb2) backprop_util)auto_control_deps)auto_control_deps_utils) constant_op)dtypes) errors_impl func_graph)indexed_slices)opstensor) tensor_shape) tensor_util) type_spec) array_ops)control_flow_util)control_flow_util_v2)default_gradient)gen_functional_ops)gen_optional_ops)gradients_util)handle_data_util)math_ops)nestcondc Cst|tr td||sd}t|n}t|d}t|d}tj}t |}t |r>|j j dus9|j j r>t|}tj||gitj|tjd||d}tj||gitj|tjd||d} tt|| gt||| |j| jd|d WdS1swYdS) z*Like tf.cond, except emits a single If op.zpred must not be a Python boolrtruefalseN collectionsr add_control_dependenciesZop_return_valueF)building_gradientname) isinstancebool TypeErrorr name_scopeutilunique_fn_nameget_default_graph_add_control_dependenciesconvert_to_tensorrZ is_tf_typeshapedimsrZ squeeze_v2func_graph_modulefunc_graph_from_py_funcCondBranchFuncGraph _collectionsverify_captures_COND _build_condexternal_captures) predtrue_fnfalse_fnr&scopeZ true_nameZ false_namer$ true_graph false_graphr@^/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/ops/cond_v2.pycond_v2=sX            $rB StatelessIfIfcGs|jdj}t|\}}|j|jksJ|j|jksJt||t|j}t||t|j}t ||g||g|j s?|j rt t rX|j}|j}t||g||g\} } n|j}|j}t||g||g\} } |j| |j| tt||g|jd7_|jd7_|dt||dt||d|j|d|j|dd| Dd d| Dt||} t||} tt||gt|j d||| | d d } d g| S) z-The gradient of an If op produced by cond_v2.r _rewritten then_branch else_branchTout output_shapescSg|]}|jqSr@dtype.0tr@r@rA z_IfGrad..cSrJr@r0rMr@r@rArPrQT)r%N)!outputsopget_func_graphs outer_graphgraph_create_grad_funcr+unique_grad_fn_namer&_create_zeros_for_none_gradsop_needs_rewriterGraphOrParentsInXlaContextr r-xla_intermediates_make_intermediates_match_xlawrapped_intermediates_make_intermediates_matchextend_check_same_outputsr7Z_set_func_attrcreate_new_tf_function_set_type_list_attr output_types_set_shape_list_attrrI _add_outputs_resolve_grad_inputs$_make_output_composite_tensors_matchr8inputs)rTgradsif_opr>r?Ztrue_grad_graphZfalse_grad_graphtrue_intermediatesfalse_intermediatesextra_true_outputsextra_false_outputsZtrue_grad_inputsZfalse_grad_inputsrSr@r@rA_IfGradksl             rqcsTttgttgtg||g}|sMtrMt}t} t|} t| } tg| | g\} } j | j | ttgt t jjt jj5ddD}ddD}|ss|rwtjntjfdd}t||}Wdn1swYdd|D}t}t||S)a%Creates an If op from the specified predicate, branch functions and inputs. Note that this modifies true_graph and false_graph to make the inputs match, and to output all intermediates values so they're available for the gradient computation. true_graph and false_graph need not have the same input types, but they must have the same output types. Args: pred: boolean Tensor true_graph: FuncGraph false_graph: FuncGraph true_inputs: a list of Tensors to be passed to true_graph as input. false_inputs: a list of Tensors to be passed to false_graph as input. building_gradient: Whether this is a gradient If op. name: the name for the If op. Returns: A list of Tensors which are the outputs of the If op. Does not include added intermediate outputs. cSg|]}|jr|qSr@Z _is_statefulrNrTr@r@rArP z_build_cond..cSrrr@rsrtr@r@rArPruc st|ddjDtttjjd\}}t|jj|durTt_t_|_ |_ t |t |t |g|j||S)NcSrJr@rKrMr@r@rArPrQz1_build_cond.._make_op..rIr&)r+get_op_and_outputsrSrc_get_output_shapes_copy_handle_datar r-rV _true_graph _false_graphmaybe_set_lowering_attr*maybe_propagate_compile_time_consts_in_xla#_set_read_only_resource_inputs_attrrWprevent_fetching)rjrltensorsr?r&op_fnr:r>r@rA_make_ops*      z_build_cond.._make_opNcSg|]}t|qSr@ridentityrMr@r@rArP4)(_make_indexed_slices_indices_types_matchr7rb_make_inputs_matchr+Zoutput_all_intermediates_get_intermediates_wrap_intermediatesr`rSrar control_dependencieslistfunction_capturescontrolget_operationsrZ_ifZ stateless_if"run_as_function_for_tape_gradients'_get_compatible_structured_output_specs_pack_sequence_as)r:r>r?Z true_inputsZ false_inputsr%r&Z cond_inputsrmrnZwrapped_true_intermediatesZwrapped_false_intermediatesrorpZtrue_stateful_opsZfalse_stateful_opsrrZstructured_output_specsr@rrAr8sV    3 r8csndfdd jdvrddddfSjd vr/fd d td DStd j)zReturns `FuncGraph`s for the input op branches. Args: op: The If or Case Operation. Returns: A tuple of the `FuncGraph`s of the then_branch and else_branch (all branches for Case). Ncsd}|dur t|d}jdd}|dur&dd|D}t||j}t||jD] \}}t||q,|j ||j|_ |S)z7Generates and returns a FuncGraph for the given branch.NrcSrJr@rRrMr@r@rArPMrQzGget_func_graphs.._get_func_graph_for_branch..) getattrrjr+Zget_func_graphr&ziprZcopy_handle_datarreset_captures _forward_cond)Zname_attr_listZcached_attr_namer rjZ input_shapesZ external_tZ internal_trTr@rA_get_func_graph_for_branchFs z3get_func_graphs.._get_func_graph_for_branch)rDrCrFrzrGr{)Case StatelessCasecs g|] \}}|d|qS)_branch_graph_{}format)rNi branch_fn)rr@rArP\sz#get_func_graphs..brancheszUnsupported op type: {}N)typeZget_attr enumerate ValueErrorrrr@)rrTrArU;s      rUcCstt|j|jS)zGReturns the most specific compatible specs of graph structured outputs.)rZ map_structure_get_compatible_specstructured_outputs)r>r?r@r@rArbsrcCsFt|}t|}||g}|dur!td|d|d|S)a_Returns the most specific compatible spec. Args: value_or_spec1: A TypeSpecs or a value that has a defined TypeSpec. value_or_spec2: A TypeSpecs or a value that has a defined TypeSpec. Returns: The most specific compatible TypeSpecs of the input. Raises: ValueError: If value_or_spec1 is not compatible with value_or_spec2. NzNo common supertype of z and .) _get_spec_forZ_without_tensor_namesZmost_specific_common_supertyper))Zvalue_or_spec1Zvalue_or_spec2Zspec1Zspec2commonr@r@rAris rcCst|tjr|St|S)zBReturns TypeSpec of a value or itself if it is a TypeSpec already.)r'rZTypeSpecZtype_spec_from_value)Z value_or_specr@r@rArs  rcCslt|jt|ks Jg}g}t|j|D]\}}t|sq||||qtj||j||d}|S)aThe gradient function for each conditional branch. This function builds the gradient graph of the corresponding forward-pass conditional branch in `func_graph`. This is done by differentiating func_graph's outputs w.r.t. its inputs. Args: func_graph: FuncGraph. The corresponding forward-pass function. grads: The list of input gradient Tensors. Returns: The output gradient Tensors. )grad_ysZ src_graph) lenrSrrZ IsTrainableappendrZ_GradientsHelperrj)r rkZysryZgrad_yresultr@r@rA_grad_fns   rcs$tj|fddgit|dS)z1Returns the FuncGraph representation of _grad_fn.cs tSr)rr@r rkr@rAs z#_create_grad_func..r )r2r3_CondGradFuncGraph)r rkr&r@rrArXs rXcCsg}|jD]S}|j|jkrS|j|ksJt|jjD]\}}||ur+|jjj|}qKqt|jjD]\}}||urB|jj|}qKq2tdj|d|j|jksSJ| |q|S)aReturns the tensors to pass as inputs to `grad_graph`. The `grad_graph` may have external references to 1. Its outer graph containing the input gradients. These references are kept as is. 2. Tensors in the forward pass graph. These tensors may not be "live" when the gradient is being computed. We replace such references by their corresponding tensor in `cond_graph.outer_graph`. In the case of nested control flow or functions, the gradient logic handling `grad_graph.outer_graph` will make sure the tensor from `cond_graph.outer_graph` is also correctly captured. Args: cond_graph: FuncGraph. The forward-pass function. grad_graph: FuncGraph. The gradients function. Returns: A list of inputs tensors to be passed to grad_graph. zFCould not find external tensor capture {tensor} in captures or outputsr ) r9rWrVrrSrZinternal_capturesrrr)Z cond_graph grad_graph new_inputsrOroutputr@r@rArhs(    rhcCs^g}|D]&}|jD] }||jvrq ||jvrq |jtjur q |jdkr&q ||q q|S)zFReturns intermediate tensors of `func_graph` for gradient computation.Z MutexLock)rrSrjrLrresourcerr)r intermediatesrTrOr@r@rArs        rcCsTg}tdd|D}t|D]\}}t||t||}||||q|S)aReturns new optionals lists that have matching signatures. This is done by mirroring each list in the other using none optionals. There is no merging of like optionals. Args: branch_graphs: `list` of `FuncGraph`. branch_optionals: `list` of `list`s of optional `Tensor`s from other branch_graphs Returns: A `list` of `list`s of `Tensor`s for each branch_graph. Each list has the same number of `Tensor`s, all of which will be optionals of the same shape/type. cs|]}t|VqdSrr)rNor@r@rA z,_make_intermediates_match..)maxr_create_none_optionalsrr) branch_graphsZbranch_optionalsZnew_branch_optionalsZintermediates_sizer branch_graphZother_optionalsr@r@rAr`sr`cszg}t|D]4\}t|tfddDg}tdddD}||d|||dq|S)z4Like _make_intermediates_match but for the XLA case.c3s |] }|ur|VqdSrr@rNZbibranch_intermediatesrr@rArs z0_make_intermediates_match_xla..csrrrrr@r@rArrN)r_create_fakeparamssumr)rrZnew_branch_intermediatesrZother_fakeparamsZ num_precedingr@rrAr^s r^c Cst|t|ks Jt}g}|D]}|D]}t|}||vr*||||qqt||D];\}}dd|D}tt||j} g} |D]} | t| } | dur[t || } | | qH| |_|j ||jq1|S)aModifies branch_graphs so they have the same input signature. This method reorders and/or adds parameters to each graph in branch_graphs so they have the same input signature, and updates the 'inputs' and 'captured' fields of each graph accordingly. It uses the input tensors from the outer graph to avoid duplicating shared arguments. Args: branch_graphs: a `list` of `FuncGraph` branch_inputs: a `list` of `list`s of `Tensor`s in the outer graph. The inputs for the corresponding graph in `branch_graphs`. Returns: A new list of Tensors from the outer graph that are the new inputs for each branch_graph. This is a deduped version of `sum(branch_inputs)`. cSrr@)r tensor_idrMr@r@rArP@rz&_make_inputs_match..N) rsetr raddrrdictrjget_create_dummy_inputrr) r branch_inputsZ added_inputsrZ branch_inrrrZ input_idsZbranch_input_to_paramZ input_listZin_tparamr@r@rAr$s2     rc Cst|t|ks Jdd|D}dd|D}tt|dks$J|tt|D]J\}}tdd|Drttdd|Drtt|D]/\}}|durs||t||j|}|||j |<Wdn1snwYqDq*|D]} d dt | j D| _ qwdS) zCreates zeros for None out grads if at least one branch has non-None grad. Args: forward_graphs: List of forward FuncGraphs. grad_graphs: List of grad FuncGraphs. cSrJr@rrNgr@r@rArPZrQz0_create_zeros_for_none_grads..cSg|]}t|qSr@rrNZoutsr@r@rArP[rcss|]}|duVqdSrr@rMr@r@rAr^rz/_create_zeros_for_none_grads..css|]}|duVqdSrr@rMr@r@rAr_rNcSg|]}|dur|qSrr@rMr@r@rArPhs ) rrrrany as_defaultrZ zeros_likerjrr2flattenrS) Zforward_graphsZ grad_graphsbranch_outputsZnum_outputs_per_branch output_idx branch_outs branch_indexrOZzerosrr@r@rArZRs,   rZc CsD|sJdd|D}tdd|D}tt|dks J|tt|D]a\}}ttdd|Ddkr8q&tdd|DsBq&t|D]@\}}t|tjrQqFt|t j rw|| t ||||<Wdn1sqwYqFtd j|tkrd nd ||d q&t||D]\}}||_d dt|D|_qdS)aModifies each branch_graph's outputs to have the same output signature. Currently the only transformation implemented is turning a Tensor into an equivalent IndexedSlices if the other branch returns an IndexedSlices. Updates branch_graph.{outputs,structured_outputs} for each branch_graph in branch_graphs. Args: op_type: _COND or _CASE branch_graphs: `list` of `FuncGraph` Raises: TypeError: if a set of outputs cannot be rewritten. cSrJr@rrr@r@rArPrQz8_make_output_composite_tensors_match..csrrrrr@r@rArrz7_make_output_composite_tensors_match..rcsrr)rrNoutr@r@rArrcs|] }t|tjVqdSrr'r IndexedSlicesrr@r@rArs  NzZCannot reconcile {op_name} {output_idx}-th outputs: outputs from all branches: {outputs}tf.condtf.switch_caseop_namerrScSrrr@rMr@r@rArPs )rrrrrrr'r r tensor_libZTensorrrZ_as_indexed_slicesr)rr7rr2rrS) op_typerrZoutputs_per_branchrr branch_idxZ branch_outrr@r@rArinsH   ric s|sJg}d}dd|D}dd|D}tt|dks"J|tt|D]O\}}ttdd|DdkrItdj|tkrBd nd ||d t|dtj rX| |dt |drm|tt j |dd d 7}q(|ddurw|d7}q(|s|dS|t|djkrtd|t|djf|D]Ztfdd|Drtdtfdd|Dttfdd|Ddkr|D],}|jjtjkr|t|jtj|j<Wdn1swYqq|D] }t|j|j|_qdS)zAMatch dtype of IndexedSlices.indices in outputs of branch_graphs.rcSsg|] }tj|jddqS)FZexpand_composites)rrrrNrr@r@rArPsz<_make_indexed_slices_indices_types_match..cSrr@rrr@r@rArPrrcsrrrrr@r@rArs  z;_make_indexed_slices_indices_types_match..zUCannot reconcile tf.{op_name} {output_idx}-th outputs: branches returned: {outputs}rZ switch_caserTrNzJInsufficient elements in branch_graphs[0].outputs. Expected: %i Actual: %ic3s(|]}|jjtjtjfvVqdSr)rSrLrint32int64rNbgindexr@rArs z?Type of IndexedSlices.indices must be int32 or int64. Found: %scsg|]}|jjqSr@rSrLrrr@rArPc3s|] }|jjVqdSrrrrr@rAr)rrrrr)rr7r'r rrrZis_nested_or_compositerrSrrstrrLrrrrcastrrr) rrZindexed_slice_indicesZ current_indexZ#branch_outputs_flat_with_compositesZouts_per_branchrrrr@rrArst      rcCsRg}d}tj|ddD]}|dur|dq ||||d7}q t||S)aPacks the outputs of the gradient If/Case op. The branch functions may contain None's in the list of `structured_outputs`. `op_outputs` has those outputs missing. So we need to add those Nones to the list of `op_outputs` and then pack it in the same structure as `structured_outputs`. Args: structured_outputs: structured_outputs from one of the branch functions. op_outputs: List of output tensors of the op. Returns: `op_outputs` packed like `structured_outputs`. rTrNr)rrrr2Zpack_sequence_as)rZ op_outputsZoutputs_with_nonescounterrr@r@rArs  rcCs:|dd|DWdS1swYdS)NcSsg|]}t|gqSr@)roptional_from_valuerMr@r@rArPrz'_wrap_intermediates..r)r rr@r@rArs  $rcCs>|tj|j|jdWdS1swYdS)zCreates tensors in func_graph to represent template_tensors. Args: func_graph: FuncGraph. template_tensor: a tensor in the outer graph. Returns: A tensor in func_graph. rRN)rr placeholderrLr0)r Ztemplate_tensorr@r@rArs $rcCs>|ddt|DWdS1swYdS)zCreates `n` `None` optionals in func_graph. Args: func_graph: FuncGraph. n: `int` the number of `None` optionals to make. Returns: A list of tensors in func_graph. cSsg|]}tqSr@)rZ optional_none)rN_r@r@rArPrz*_create_none_optionals..N)rrange)r nr@r@rArs $rcCs&|jdur|Stdd|DS)aConverts dynamic dimensions in `shape` to zero. The fake params created to match the intermediates captured in other branches could have dynamic dimensions. But the XLA shape is not able to handle dynamic dimensions in TF TensorShape. Setting the dynamic dimensions to size zero will help avoid failing safety checks in bridge. When XLA DynamicConditional op reconciles branch differences, XLA will replace the dimension size 0 with a bounded dimension determined from the shape of real argument in the other branch. Note: Rank unknown shapes are returned as they are. Args: shape: The TensorShape of fake param. Returns: The new TensorShape with dynamic dimensions set to zero. NcSsg|] }|dur dn|qS)Nrr@)rNdr@r@rArP:sz6_convert_dynamic_dimension_to_zero..)Zrankr TensorShapeas_listrRr@r@rA"_convert_dynamic_dimension_to_zero#s rcCs:|dd|DWdS1swYdS)z$Creates FakeParams for the XLA case.cSs"g|] }tj|jt|jdqS))rLr0)rZ fake_paramrLrr0rMr@r@rArPAs  z&_create_fakeparams..Nr)r Ztemplate_tensorsr@r@rAr>s $rc sfdd}tdtD]x}ztjdj|jddWnttfy;}z ||t|WYd}~nd}~wwtkrBdnd }tdj t|j krgtd j |tdj |t|j d t dj |j D]\}}|j |j kr||d ||fqrqdS) z3Raises an error if `graphs` have different outputs.c sPtdjtkr dndtkrdnd|tkrdnddj|j|d ) Nz{b0_name} and {bn_name} arguments to {op_name} must have the same number, type, and overall structure of return values. {b0_name} output: {b0_out} {bn_name} output: {bn_out} Error details: {detail}r;z branches[0]r<z branches[{}]rrr)Zb0_nameZbn_namerb0_outbn_outdetail)r)rr7r)rZ error_detailgraphsrr@rAerrorJs z"_check_same_outputs..errorrrTrNrcasezuLengths of branch outputs of {op_type} must match. len(graphs[0].outputs): {len_0} len(graphs[{b}].outputs): {len_b} )rZlen_0blen_bz%s and %s have different types) rrrZassert_same_structurerrr)rr7rSrrrL)rrrreZ op_type_strrrr@rrArbGs6    rbcGsHg}t|D]}|dj}|ddD]}||j}q||q|S)Nrr)rr0most_specific_compatible_shaper)branch_graph_outputsrIZ out_by_branchr0Z other_outr@r@rArxss   rxc Gst|g|RD]t}|d}|dd}g}|D]}t|}|jr)t|jdkr+q|||qtd}d} |D]&}t|jdj } | | }| durU|jdj } q:|jdj | kr`t j } q:|d} | jdj || | jd_ t|| qdS)zGCombines shapes in handle data and sets metadata on `external_tensors`.rrN)rrZget_resource_handle_datais_setrZshape_and_typerrrr0r rLrZ DT_INVALIDZCopyFromZas_protoZset_handle_data) Zexternal_tensorsr rZexternalZinternalZinternal_handle_datar handle_dataZcombined_shapeZcombined_dtypeZ handle_shapeZcombined_handle_datar@r@rAry}s>        rycCsddt|D}t|D]:\}}|jD]2}t|tjsF|j|vrF|tkr)ddgn ddtt|D}t dj |j |||j||dqq d S) zCVerify that a branch's tensor is not accessed in another branch fn.cSsi|]\}}||qSr@r@)rNrrr@r@rA rz#verify_captures..r;r<cSsg|]}d|qS)z branch {}rrr@r@rArPs z#verify_captures..z5Tensor {tname} in {b0name} is accessed from {b1name}.)ZtnameZb0nameZb1nameN) rr9r'r Z EagerTensorrWr7rrrrr&)rrZother_branch_graphsrrrO branch_namesr@r@rAr6s"   r6csDeZdZdZfddZeddZeddZfdd ZZ S) ra]FuncGraph for the gradient function of the branch of an If op. Handles wrapping and unwrapping intermediate values that are captured by the gradient computation in optionals. Attributes: op_needs_rewrite: True if any intermediates were captured, meaning the forward If op needs to be written to output the wrapped intermediates. csFtt|j|tjdd|_||_i|_t |_ g|_ i|_ dS)Nr!F)superr__init__r r-r5r[_forward_graph_indirect_capturesr" OrderedDict_wrapped_intermediates_xla_intermediates_captured_constants)selfr&Z forward_graph __class__r@rArs    z_CondGradFuncGraph.__init__cCst|jS)zCThe optional-wrapped intermediates captured from the forward graph.)rrvaluesrr@r@rAr_sz(_CondGradFuncGraph.wrapped_intermediatescCs|jS)zDRaw intermediates captured from the forward graph if XLA is enabled.)rrr@r@rAr]sz$_CondGradFuncGraph.xla_intermediatescs2j|jus tfdd|jjDs tfdd|jjDr)tt||St }||j vr8|j |St rPt j tjd|j |<|j |Sttrutfdd|jDrl|jd|_tt||S|j|}|dur|Sjtjkrtjdd|jjDd d |j D|jj!}tt||jj||}nc||j"vr#D]j$d krtfd d|jjDrӈjd }qq|j%t&'g}Wdn1swYd|_||j"|<|j"|}tt|||}t&(|jgj)gd }||j|<|S)Nc3|]}|uVqdSrr@rMr r@rArrz5_CondGradFuncGraph._capture_helper..c3rrr@rMr r@rArrrKc3s|]}|uVqdSrr@)rNcapturer r@rArrTcSrJr@r&rMr@r@rArPrQz6_CondGradFuncGraph._capture_helper..cSsi|]}|j|jqSr@)r&Znode_defrtr@r@rArrz6_CondGradFuncGraph._capture_helper..ZOptionalFromValuec3s|] }jd|uVqdS)rNrS)rNr)consumerr@rAr sr)*rWrrrjrSrr_capture_helperr rrr is_constantZconstantrZconstant_valuerLrr\r-allr9r]rr[rrrrr+Zresource_input_indexr&rZ _functionsrZ consumersrrrrZoptional_get_valuer0)rrr&rZcaptured_tensorroptionalZcaptured_optionalr)r"rrAr#sv                     z"_CondGradFuncGraph._capture_helper) __name__ __module__ __qualname____doc__rpropertyr_r]r# __classcell__r@r@rrArs   r indexed_casec st|tr td|t|Wfddtt|D}tj}tj |dd}g}t ||D]\}}| t j ||gitj|tjd||dq0tt|t||dd|D|d Wd S1siwYd S) z6Like conv_v2, except emits a Case op instead of an If.z%branch_index must not be a Python intcsg|] }td|qS)zbranch{})r+r,r)rNrr=r@rArP,sz indexed_case..rr r!r#cSrJr@)r9rr@r@rArPGrQr&lower_using_switch_mergeN)r'intr)r r*rrr-r.r/rrr2r3r+r4r5r6_CASE _build_case) rZ branch_fnsr&r0rr$r branch_namerr@r.rAr-#s@       $rrc Gs|jdj}t|}|sJ|D] }|j|jksJqg}|D]}|t||t|j qt ||t dd|Drt trQdd|D}t||}n dd|D}t||}t||D] \}} |j| qbtt||D] }|j d7_ qt|dd d|D|d |dj|d |dj|d d|dDd d|dDddt||D} tt|z|d} Wn tjyd} Ynwt |j!d|| d| d} dg| S)z5The gradient of a Case op produced by tf.switch_case.rcss|]}|jVqdSr)r[rr@r@rAresz_CaseGrad..cSrJr@)r])rNbranch_grad_graphr@r@rArPqsz_CaseGrad..cSrJr@)r_rr@r@rArPxsrErcSrr@r+rcrr@r@rArPsrHrIcSrJr@rKrMr@r@rArPrQcSrJr@rRrMr@r@rArPrQcSsg|] \}}t||qSr@)rh)rNrr5r@r@rArPs Z_lower_using_switch_mergeNZgradientr/)"rSrTrUrVrWrrXr+rYr&rZrrr\r r-r^r`rrarbr2Z_set_func_list_attrrdrerfrIrgriZ_get_attr_boolrZ NotFoundErrorr3rj) rTrkcase_oprrZbranch_grad_graphsZbranches_intermediatesZextra_branch_outputsrZ extra_outputsZbranches_grad_inputsZloweringrSr@r@rA _CaseGradLsv       r8c sttttt|}g}D]}|dd|Dq|r(tjntjt t ddDgfdd}t ||} Wdn1sSwYdd| D} td j| S) aCreates an `Case` op from `branch_index`, branch graphs and inputs. Note that this modifies `branch_graphs` to make the inputs match, and to output all intermediates values so they're available for the gradient computation. `branch_graphs` need not have the same input types, but they must have the same output types. Args: branch_index: integer Tensor branch_graphs: List of FuncGraph branch_inputs: List of lists of Tensors to be passed to corresponding branch_graph as input. name: the name for the Case op. lower_using_switch_merge: Lower this op using switch merge ops (optional). Returns: A list of Tensors which are the outputs of the Case op. Does not include added intermediate outputs. cSsg|] }t|r|qSr@)rZop_is_statefulrtr@r@rArPs  z_build_case..css|] }t|jjVqdSr)rrrrr@r@rArrz_build_case..c st|dddjDddDtddDd\}}t|gddDR|durat|t|t||j |t D]\}}t |_ t|d||qN|S) NcSrJr@rKrMr@r@rArPrQz1_build_case.._make_op..rcSrr@r6rr@r@rArPrcSrJr@r!rr@r@rArPrQrvcSrJr@r!rr@r@rArPrQr)r+rwrSrxryr|r}r~rWrrr r-rVsetattrr)rjr7rrrrrr0r&rr@rArs"       z_build_case.._make_opNcSrr@rrMr@r@rArPrr)rr2rbrrarrrZstateless_caser rrr+rrr) rrrr&r0Z case_inputsZ stateful_opsrrrr@r:rAr3s&     #r3cCsttt|jd}|D] }t|jt|jdksJd|s#q.t|}||}q dd|D}t|tj t |dS)zSets the list of resource inputs which are read-only. This is used by AutomaticControlDependencies. Args: op: If or Case Operation. branch_graphs: List of branch FuncGraphs. rzshould never happencSsg|]}|dqS)rr@)rNrr@r@rArPrz7_set_read_only_resource_inputs_attr..N) rrrrjacdZ*get_read_only_resource_input_indices_graph intersectionr Zset_int_list_attrZREAD_ONLY_RESOURCE_INPUTS_ATTRsorted)rTrZread_only_indicesrZbranch_read_only_indicesr@r@rAr~s    r~)rr)r-N)NN)Gr*r"Ztensorflow.core.frameworkrZtensorflow.python.eagerrZtensorflow.python.frameworkrrr;rrrr r2r r rrrrrZtensorflow.python.opsrrrr+rrrrrrZtensorflow.python.utilrr7r2rBZRegisterGradientrqr8rUrrrrrXrhrr`r^rrZrirrrrrrrrbrxryr6r4rr-r8r3r~r@r@r@rAs                          . [ y'&4.3C , "s ) ` U