o ?el#@sdZddlmZddlmZddlmZddl m Z ddl m Z dZ Gdd d ZGd d d Ze d gd        dddZdS)z(Device function for replicated training.) node_def_pb2)device) tf_logging) server_lib) tf_export)VariableZ VariableV2ZAutoReloadVariableZMutableHashTableZMutableHashTableV2ZMutableHashTableOfTensorsZMutableHashTableOfTensorsV2ZMutableDenseHashTableZMutableDenseHashTableV2Z VarHandleOpZ$BoostedTreesEnsembleResourceHandleOpZ*BoostedTreesQuantileStreamResourceHandleOpZResourceConditionalAccumulatorZDecisionTreeResourcec@ eZdZdZddZddZdS)_RoundRobinStrategyzReturns the next ps task index for placement in round-robin order. This class is not to be used directly by users. See instead `replica_device_setter()` below. cCs||_d|_dS)zgCreate a new `_RoundRobinStrategy`. Args: num_tasks: Number of ps tasks to cycle among. rN) _num_tasks _next_task)selfZ num_tasksr i/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/training/device_setter.py__init__)s z_RoundRobinStrategy.__init__cCs|j}|jd|j|_|S)aChoose a ps task index for the given `Operation`. Args: unused_op: An `Operation` to be placed on ps. Returns: The next ps task index to use for the `Operation`. Returns the next index, in the range `[offset, offset + num_tasks)`. )r r )r Z unused_optaskr r r__call__2s z_RoundRobinStrategy.__call__N)__name__ __module__ __qualname____doc__rrr r r rr "s r c@r)_ReplicaDeviceChooserzClass to choose devices for Ops in a replicated training setup. This class is not to be used directly by users. See instead `replica_device_setter()` below. cCs(||_||_||_||_||_||_dS)a3Create a new `_ReplicaDeviceChooser`. Args: ps_tasks: Number of tasks in the `ps` job. ps_device: String. Name of the `ps` job. worker_device: String. Name of the `worker` job. merge_devices: Boolean. Set to True to allow merging of device specs. ps_ops: List of strings representing `Operation` types that need to be placed on `ps` devices. ps_strategy: A callable invoked for every ps `Operation` (i.e. matched by `ps_ops`), that takes the `Operation` and returns the ps task index to use. N) _ps_tasks _ps_device_worker_device_merge_devices_ps_ops _ps_strategy)r ps_tasks ps_device worker_device merge_devicesps_ops ps_strategyr r rrHs  z_ReplicaDeviceChooser.__init__cCs|js |jr |jStj|jpd}t|tjr|n|j}|j rQ|j rQ|j |j vrQtj|j }|j |j }}|rH|r?||krH|j||d}||}|Stj|jpXd}||}|S)z~Choose a device for `op`. Args: op: an `Operation`. Returns: The device to use for the `Operation`. )r)rrpydev DeviceSpec from_string isinstancerZNodeDefnode_defrroprjobreplacerZmake_merged_specZ to_stringr)r r*Zcurrent_devicer)rZ current_jobZps_jobr r r rdevice_function^s   z%_ReplicaDeviceChooser.device_functionN)rrrrrr-r r r rrAs rztrain.replica_device_setter)v1/job:ps /job:workerTNc Cs|dur/t|tjr|}nt|}tj|j}||vs'||dur)dSt||}|dkr5dS|dur=t t }|sDt d|durLt |}t|sTtdt||||||} | jS)a Return a `device function` to use when building a Graph for replicas. Device Functions are used in `with tf.device(device_function):` statement to automatically assign devices to `Operation` objects as they are constructed, Device constraints are added from the inner-most context first, working outwards. The merging behavior adds constraints to fields that are yet unset by a more inner context. Currently the fields are (job, task, cpu/gpu). If `cluster` is `None`, and `ps_tasks` is 0, the returned function is a no-op. Otherwise, the value of `ps_tasks` is derived from `cluster`. By default, only Variable ops are placed on ps tasks, and the placement strategy is round-robin over all ps tasks. A custom `ps_strategy` may be used to do more intelligent placement, such as `tf.contrib.training.GreedyLoadBalancingStrategy`. For example, ```python # To build a cluster with two ps jobs on hosts ps0 and ps1, and 3 worker # jobs on hosts worker0, worker1 and worker2. cluster_spec = { "ps": ["ps0:2222", "ps1:2222"], "worker": ["worker0:2222", "worker1:2222", "worker2:2222"]} with tf.compat.v1.device(tf.compat.v1.train.replica_device_setter(cluster=cluster_spec)): # Build your graph v1 = tf.Variable(...) # assigned to /job:ps/task:0 v2 = tf.Variable(...) # assigned to /job:ps/task:1 v3 = tf.Variable(...) # assigned to /job:ps/task:0 # Run compute ``` Args: ps_tasks: Number of tasks in the `ps` job. Ignored if `cluster` is provided. ps_device: String. Device of the `ps` job. If empty no `ps` job is used. Defaults to `ps`. worker_device: String. Device of the `worker` job. If empty no `worker` job is used. merge_devices: `Boolean`. If `True`, merges or only sets a device if the device constraint is completely unset. merges device specification rather than overriding them. cluster: `ClusterDef` proto or `ClusterSpec`. ps_ops: List of strings representing `Operation` types that need to be placed on `ps` devices. If `None`, defaults to `STANDARD_PS_OPS`. ps_strategy: A callable invoked for every ps `Operation` (i.e. matched by `ps_ops`), that takes the `Operation` and returns the ps task index to use. If `None`, defaults to a round-robin strategy across all `ps` devices. Returns: A function to pass to `tf.device()`. Raises: TypeError if `cluster` is not a dictionary or `ClusterDef` protocol buffer, or if `ps_strategy` is provided but not a callable. NrzQDEPRECATION: It is recommended to set merge_devices=true in replica_device_setterzps_strategy must be callable)r(rZ ClusterSpecas_dictr%r&r'r+lenlistSTANDARD_PS_OPSloggingwarningr callable TypeErrorrr-) rrr r!Zclusterr"r#Z cluster_specZ ps_job_nameZchooserr r rreplica_device_setters0B   r9)rr/r0TNNN)rZtensorflow.core.frameworkrZtensorflow.python.frameworkrr%Ztensorflow.python.platformrr5Ztensorflow.python.trainingrZ tensorflow.python.util.tf_exportrr4r rr9r r r rs$       A