o ?ea@sdZddlZddlZddlmZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZd Zd Zd d ZddZ  dddZ  dddZedgddddZdS)zqLogging tensorflow::tfprof::OpLogProto. OpLogProto is used to add extra model information for offline analysis. N)tfprof_log_pb2)context)ops) tensor_shape)gfile)flops_registry) tf_exportZ_trainable_variablesZflopsc Cs|jjD]}|jD]~}|jsq z||j}Wnty)}zWYd}~q d}~wwt|jt|jkr5q t |jD]L\}}|j| rHq:|j j j}tdd|D} z|j||j| | Wq:ty}ztjd|j|fWYd}~q:d}~wwq q|S)zBFill Tensor shapes in 'graph' with run time shape from 'run_meta'.NcSsg|]}|jqS)size).0dr r i/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/profiler/tfprof_logger.py 5sz-_fill_missing_graph_shape..z!Node %s incompatible shapes: %s. )Z step_statsZ dev_statsZ node_statsoutputZget_operation_by_nameZ node_nameKeyErrorlenZoutputs enumerate get_shapeZis_fully_definedZtensor_descriptionshapedimrZ TensorShape set_shapeZ merge_with ValueErrorsysstderrwrite) graphrun_metaZdev_statZ node_statopeiZ node_stat_outZnode_stat_dimsZnode_stat_shaper r r _fill_missing_graph_shape"s@      r cCs(||d}|durt|}|||<|S)zMaps string to id.N)getr)sZ str_to_idnumr r r _str_id?s r$Tc Cs|rt||}d}i}i}t||d<|D]y}z t||jt}Wnty2|d7}d}Ynwt } |j | _ d} |rJ|j rJt |j | _ d} |r|jr|jD]4\} } } }| jj}| ret| |nd|_| rl| nd|_| rvt| |nd|_|rt||nd|_d|_qRd} | r| || j <q|r|tjjD]'}|jj |vrt } |jj | _ | jt| || j <q||jj jtq|dkr|stj d|||fS) aExtract trainable model parameters and FLOPs for ops from a Graph. Args: graph: tf.Graph. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: logged_ops: dict mapping from op_name to OpLogEntry. string_to_id: dict mapping from string to id. rnoneNFTz0%d ops no flops stats due to incomplete shapes. )!r rZget_operationsrZget_stats_for_node_defZnode_defREGISTERED_FLOP_STATSrrZ OpLogEntrynamevalueint float_ops tracebackcode_deftracesaddr$Zfile_idlinenoZ function_idZline_idZfunc_start_lineZget_collectionZ GraphKeysTRAINABLE_VARIABLESrtypesappendrrr)rr add_traceadd_trainable_varZop_missing_shape logged_ops string_to_idrstatsentry add_entryfilenamer1funcnamelinetracevr r r _get_logged_opsHs^             rAc Cs|s ts t}t}|s|St||||d\}}|s'|j| nUi}|jD]} | || j <q,| D];\} } | |vro|| j | j | j dkr[|| j dkr[| j || _ | jjrn|| jjsn|| j| jq8| || <q8|j| | D] \} } | |j| <q|S)aMerge the tfprof default extra info with caller's op_log. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. op_log: OpLogProto proto. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: tmp_op_log: Merged OpLogProto proto. )r5r6r)rexecuting_eagerlyrget_default_graphrZ OpLogProtorAZ log_entriesextendvaluesr)itemsr3r,r.r/Z MergeFromZ id_to_string) rop_logrr5r6Z tmp_op_logr7r8Zall_opsr:Zop_namer"rr r r merge_default_with_oplogs4       rHzprofiler.write_op_log)v1cCsl|s ts t}t||||}ttj |dd}| | WddS1s/wYdS)aLog provided 'op_log', and add additional model information below. The API also assigns ops in tf.compat.v1.trainable_variables() an op type called '_trainable_variables'. The API also logs 'flops' statistics for ops with op.RegisterStatistics() defined. flops calculation depends on Tensor shapes defined in 'graph', which might not be complete. 'run_meta', if provided, completes the shape information with best effort. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. log_dir: directory to write the log file. op_log: (Optional) OpLogProto proto to be written. If not provided, an new one is created. run_meta: (Optional) RunMetadata proto that helps flops computation using run time shape information. add_trace: Whether to add python code trace information. Used to support "code" view. Z tfprof_logwN) rrBrrCrHrZOpenospathjoinrZSerializeToString)rZlog_dirrGrr5logr r r write_op_logs "rO)NTT)NNTT)NNT)__doc__rKrZtensorflow.core.profilerrZtensorflow.python.eagerrZtensorflow.python.frameworkrrZtensorflow.python.platformrZ#tensorflow.python.profiler.internalrZ tensorflow.python.util.tf_exportrr2r(r r$rArHrOr r r r s,         E /