o ?eZ@s|dZddlZddlmZddlmZddlmZddlmZddl m Z ddl m Z dd l m Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZdZdZ e!dgZ"e!gdZ#ededdddd0ddZ$Gdddej%Z&d0d d!Z'd"d#Z(d$d%Z)d&d'Z*ej+d(d)Z,Gd*d+d+e-Z.d,d-Z/d.d/Z0dS)1z>xla is an experimental library that provides XLA support APIs.N)xla_ops) xla_ops_grad)attr_value_pb2)summary_op_util)context) def_function)ops) array_ops)control_flow_ops)variable_scope) tf_logging)compat)nest) tf_inspect)collections_abc) deprecated) tf_exportZ_xla_compile_idZ Placeholder) Z AudioSummaryZAudioSummaryV2ZHistogramSummaryZ ImageSummaryZ MergeSummaryZPrintZ ScalarSummaryZ TensorSummaryZTensorSummaryV2zxla.experimental.compilezXxla.experimental.compile is deprecated. Consider using `@tf.function(jit_compile=True)`.T)Z warn_oncecs,trtjfdd}|StS)aBuilds an operator that compiles and runs `computation` with XLA. NOTE: In eager mode, `computation` will have `@tf.function` semantics. Args: computation: A Python function that builds a computation to apply to the input. If the function takes n inputs, 'inputs' should be a list of n `Tensor`s. `computation` may return a list of `Tensor`s and `Operation`s. `Tensor`s must come before `Operation`s in the returned list. All `Operation`s returned from `computation` will be executed when evaluating any of the returned output tensors. inputs: A list of inputs or `None` (equivalent to an empty list). Each input can be a nested structure containing values that can be converted to `Tensor`s. Note that passing an N-dimension list of compatible values will result in an N-dimension list of scalar `Tensor`s rather than a single Rank-N `Tensor`. If you need a different behavior, convert parts of `inputs` to `Tensor`s with `tf.convert_to_tensor`. Returns: List of `Tensor`s corresponding to the `Tensor`s from the output of `computation` i.e. the same return value as if computation(*inputs) is called directly, with the following exceptions: * None output: a NoOp would be returned with a control dependency on `computation`. * Single value output: a tuple containing the value would be returned. * Operation-only outputs: a NoOp would be returned with a control dependency on `computation`. TODO(b/121383831): Investigate into removing these special cases. Raises: RuntimeError: When eager execution is enabled. Known issues: When a tf.random operation is built with XLA, the implementation doesn't pass the user provided seed to the XLA compiler. As such, the XLA compiler generates a random number and uses it as a seed when compiling the operation. This implementation causes a violation of the Tensorflow defined semantics in two aspects. First, changing the value of the user defined seed doesn't change the numbers generated by the operation. Second, when a seed is not specified, running the program multiple times will generate the same numbers. cs tSN)_compile_internal computationinputsrc/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/compiler/xla/xla.pyxla_compile_wrapperss z$compile..xla_compile_wrapper)rZexecuting_eagerlyrfunctionr)rrrrrrcompile>s 3 rcs`eZdZdZfddZddZddZdd Zd d Zd d Z e ddZ e ddZ Z S)XLACompileContextaA `ControlFlowContext` for nodes inside an XLA computation cluster. THIS IS ONLY FOR TENSORFLOW INTERNAL IMPLEMENTATION, DO NO USE DIRECTLY. The primary role of `XLACompileContext` is to mark operators inside a xla.compile() computation with attribute "_xla_compile_id=XYZ", where XYZ is a unique name. `ControlFlowContext` is used to perform the annotation since it integrates with Tensorflow constructs like ResourceVariables. For example, if a `ResourceVariable` is constructed inside a xla.compile() block, the `ResourceVariable` implementation can use `with ops.control_dependencies(None)` to build the variable's definition outside the compiled computation. cs0tt|||_t||_g|_||_dS)aBuilds a new XLACompileContext. Args: name: a unique name for the context, used to populate the `_xla_compile_id` attribute. pivot: a pivot node. Nodes in the XLACompileContext that do not have any inputs will have a control dependency on the pivot node. This ensures that nodes are correctly included in any enclosing control flow contexts. N) superr__init___namer as_bytes_name_as_bytes_unsupported_ops_pivot)selfnamepivot __class__rrr s    zXLACompileContext.__init__cCsh|jr0ddd|jdtD}tdt|j|t|jtkr2tdt|jtdSdSdS)N cSsg|] }d|j|jfqS)z %s (%s))typer').0oprrr szCXLACompileContext.report_unsupported_operations..z$%d unsupported operations found: %sz... and %d more)r$join_MAX_WARNING_LINESloggingwarninglen)r&Zop_strrrrreport_unsupported_operationss    z/XLACompileContext.report_unsupported_operationscCsxg}g}|jD]'}d}|}|dur!||krd}q!|j}|dus|r)||q||q|||||fS)z2Remove any external control dependency on this op.FNT)Zcontrol_inputsZ_get_control_flow_context_outer_contextappendZ_remove_all_control_inputs_add_control_inputs)r&r.internal_control_inputsexternal_control_inputsxZis_internal_opZctxtrrr_RemoveExternalControlEdgess"    z-XLACompileContext._RemoveExternalControlEdgesc Cs|jtvrtd|j|j|jtvr|j|tdd|j Dr*t d|jt |j j vr7td|j|t tj|jd|j||j|||\}}|j sa|s`||jntt|j D]}|j |}||}||ur~|||qh|rtd|dd |D}| Wdn1swY|!|d d |j"D}|}|dur|j#$||j%}|dus|j%r|j%&|dSdS) zFCreate op in XLACompileContext and notifies outer context recursively.zhOperation of type %s (%s) is not supported in XLA. Execution will fail if this op is used in the graph. css|]}|jjVqdSr)ZdtypeZ _is_ref_dtyper-r;rrr sz*XLACompileContext.AddOp..zTNon-resource Variables are not supported inside XLA computations (operator name: %s)z?XLA compiled computations cannot be nested, (operator name: %s))sNcSs$g|]}|jrt|jdjqS)r)outputsr identityr.r=rrrr/s z+XLACompileContext.AddOp..cSsg|]}|jqSrr'r=rrrr/s)'r,_DENYLISTED_OPSr2errorr'_UNSUPPORTED_OPSr$r7anyrNotImplementedError_XLA_COMPILE_ATTRZnode_defattr ValueErrorZ _set_attrrZ AttrValuer#graphZprevent_feedingZprevent_fetchingr<Z_add_control_inputr%ranger4AddValueZ _update_inputrcontrol_dependenciesEnterExitr8r@_valuesupdater6 AddInnerOp) r&r.r9r:indexr;Zreal_xZ output_namesrrrrAddOpsj              zXLACompileContext.AddOpcCsl|j|jvr|j|j}|dur|S|S|}|j|j|jr.|j|}|j|j||j|j<|S)zCAdd `val` to the current context and its outer context recursively.N)r'rQZ_external_valuesgetaddr6rM)r&valresultrrrrM s   zXLACompileContext.AddValuecCs$|||jr|j|dSdSr)rUr6rS)r&r.rrrrSs zXLACompileContext.AddInnerOpcCsdSrrr&rrr grad_state!szXLACompileContext.grad_statecCs|r |jSdS)z0Forwards to the enclosing while context, if any.F)ZGetWhileContext back_proprZrrrr\)s zXLACompileContext.back_prop)__name__ __module__ __qualname____doc__r r5r<rUrMrSpropertyr[r\ __classcell__rrr)rr|s  I rc Cs|durg}t|tjstdt|}dd|D}td}t j |dd}t ||d}z^| d dt |D}tj||d }t}|j}|d t ||} Wdn1sdwY||t| } | r{t| \} } nt| \} } || W||n ||w| st j| d dSd dt | D} t| ddt | D} Wdn1swY| stj| | d } | S)aEBuilds graph operators that compiles and symbolically executes computation. Args: computation: A Python function that builds the computation to compile and execute. inputs: A list of inputs or `None` (equivalent to an empty list). Each input can be a nested structure containing values that are convertible to tensors. Note that passing an N-dimension list of compatible values will result in a N-dimension list of scalar tensors rather than a single Rank-N tensors. If you need different behavior, convert part of inputs to tensors with `tf.convert_to_tensor`. Returns: Same data structure as if computation(*inputs) is called directly with some exceptions for correctness. Exceptions include: 1) None output 2) Single value output 3) Operation-only outputs Raises: ValueError: If any element in computation outputs is neither an operations or a value that can be converted to tensor. ValueError: If computation outputs is non-flat and contains any Operations. TypeError: If `inputs` is not a list or tuple. Nzinputs must be a listcSsg|]}t|qSr)rconvert_to_tensorr=rrrr/Qsz%_compile_internal..Zclusterz/pivotrB)r'r(cS$g|]\}}tj|d|dqS)zinput_{}rB)r rAformat)r-ir;rrrr/[)Z structureZ flat_sequenceTZoutput_0cSrd)zoutput{}rB)rZxla_cluster_outputrer-rforrrr/rgcSs"g|] \}}tj|d|dqS)z output_%drB)r rArhrrrr/s) isinstancerSequence TypeErrorrflattenrZget_default_graphZ unique_namer no_oprrO enumerateZpack_sequence_asr Zget_variable_scopeZ use_resourceZset_use_resource_disable_summary_contextis_flat_postprocess_flat_outputs_postprocess_non_flat_outputsZ ExitResultr5rPgrouprN) rrZ flat_inputsZ cluster_namer(rZcomputation_inputsZvscopeZsaved_use_resourcer@Zoutputs_is_flatoutput_tensorsZ control_depsrrrr1s`            rcCsdt|tjr |D]}t|tjst|tjst|jdrdSqt|tjr(dSt|jdr0dSdS)auChecks if outputs is a flat structure. Following structures and values are considered flat: 1) None 2) A single object 3) A list or tuple of Tensors/Operations The only structures that this function understands are sequences, dictionaries and types defined using the attrs library. E.g. this means that if outputs contains a single user-defined Object, it is considered to be flat. Errors are raised later on if that Object cannot be converted to a Tensor. Args: outputs: Output from `computation` inside `xla.compile`. Returns: A boolean indicates whether outputs is flat. Z__attrs_attrs__FT)rjrrkMappinghasattrr*)r@rirrrrqs      rqc Cs|durt}t|tjs|f}|tf7}z dd|D}Wnty4}ztdt|d}~wwdd|D}dd|D}|||krMtdg}|D]%}t |j r[|j nd| t |Wdn1sqwYqQ||fS) zValidates flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors and Operations extracted from outputs. NcSs&g|]}t|tjr |nt|qSr)rjr Operationrcr-rirrrr/sz-_postprocess_flat_outputs..oXLA computation function return values must all either be Operations or convertible to Tensors. Got error: "%s"cSsg|] }t|tjr|qSrrjrrxryrrrr/cSsg|] }t|tjs|qSrr{ryrrrr/r|zdXLA computation function must return zero or more Tensor values followed by zero or more Operations.)tuplerjrrkr rn ExceptionrJstrrdevicer7r rA)r@eZoutput_operationsrunew_output_tensorstrrrrrs<   rrc Csg}t|D]N}t|tjrtd|jzt|}Wnty1}ztdt |d}~wwt |j r:|j nd| t |Wdn1sPwYq|gfS)aValidates non-flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors extracted from outputs and an empty list because Operations are not allowed in non-flat outputs.. zxla.compile does not support Operation as return value in non-flat output structure. You can set returned Operations as control dependencies of returned Tensors so Operations are triggered when Tensors are evaluated. Operation found: "%s"rzNr})rrmrjrrxrJr'rcrrrr7r rA)r@rrirrrrrss.  rsccs.tj}ddt_z dVW|t_dS|t_w)aEnters a context where all summary ops are skipped. Summaries are not yet supported in xla.compile(). So we provide this context manager that can skip creating summary ops. This is a temporary workaround due to XLA not supporting summary ops. Yields: None. cSsdS)NTrrrrr%sz*_disable_summary_context..N)rZ skip_summary)Zoriginal_skip_summary_funcrrrrps  rpc@s(eZdZdZddZddZddZdS) _CapturedObjectz#A placeholder to capture an object.cCs d|_dSr_objectrZrrrr 0s z_CapturedObject.__init__cCs|jrtd||_dS)NzFInternalError: _CapturedObject can capture only once. Please file bug.)r RuntimeError)r&rirrrcapture3s  z_CapturedObject.capturecCs|jSrrrZrrrrV;sz_CapturedObject.getN)r]r^r_r`r rrVrrrrr-s  rcCs*|}|sdS|}|durtd|S)z3Retrieves the Scaffold from `captured_scaffold_fn`.Nz?TPUEstimatorSpec.scaffold_fn returns None, which is not allowed)rVrJ)Zcaptured_scaffold_fnZ scaffold_fnZscaffoldrrr _get_scaffold?src Csdd}|}|dur||j7}t|}t|j}|jdur!d}nt|j}||}||krA|dkr<|jdur<|d|S|d|S|jdurX||krX|dkrS|d|S|d|SdS)aValidate the number of input arguments to an XLA function. Args: func: the Python function that will be called to generate the body of an XLA computation graph. input_arity: the number of explicit arguments supplied by the caller. infeed_queue: if not None, the infeed queue that will supply additional arguments to the function. Returns: None if function can be called with the supplied number of arguments, or an error string if it cannot. cSsd|||dkr dfSdfS)Nz%s %d argument%sr}r?r)Z complaintZquantityrrr format_error\s  z3check_function_argument_count..format_errorNrZexactlyzat leastzat most)Znumber_of_tuple_elementsr getargspecr4argsdefaultsvarargs) funcZ input_arityZ infeed_queuerZnum_args_suppliedZarg_specZ num_func_argsZnum_func_defaultsZ min_func_argsrrrcheck_function_argument_countNs&         rr)1r` contextlibZtensorflow.compiler.jit.opsrrZtensorflow.core.frameworkrZtensorflow.python.distributerZtensorflow.python.eagerrrZtensorflow.python.frameworkrZtensorflow.python.opsr r r Ztensorflow.python.platformr r2Ztensorflow.python.utilr rrZtensorflow.python.util.compatrZ"tensorflow.python.util.deprecationrZ tensorflow.python.util.tf_exportrrHr1setrCrErZXLAControlFlowContextrrrqrrrscontextmanagerrpobjectrrrrrrrsT                   9 6g*4#