o ?ePT @s(dZddlZddlZddlmZmZmZddlZddl m Z ddl m Z ddlmZgdggZdd Zed Gd d d ejZed GdddeZdedededeeeeeffddZdedededeeeeeffddZdddejfde deejdeejdededef dd ZdS)!z Library of TPU helper functions.N)ListOptionalTuple) tf_logging)Topology) tf_export)rrrrc Csi}t|jdD]8}t|jdD].}|||ddf}||}||vr*i||<|||vr7t|||<||||qq i}|D]\}} i} | D] \} } t| | | <qR| ||<qH|S)zDComputes a nested dict which maps task and logical core to replicas.rN)rangeshapeZtask_ordinal_at_coordinatessetadditemssorted) core_assignmenttopologyZtask_and_cores_to_replicasreplica logical_core coordinatestask_idZtask_to_sorted_replica_idtaskZcore_to_replicasZcore_to_sorted_replicascoreZreplicasrh/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/tpu/device_assignment.py#_compute_task_and_cores_to_replicass$    rz tpu.experimental.DeviceOrderModec@seZdZdZdZdZdZdS)DeviceOrderModezFThe way of determining device orders when computing device assignment.rrN)__name__ __module__ __qualname____doc__AUTORINGZMESHrrrrr8s rz!tpu.experimental.DeviceAssignmentc@s@eZdZdZdedejfddZedefddZ ede fd d Z ede fd d Z edejfd dZ de de defddZde de dee fddZd&de de de fddZ   d'de de deedefddZ   d'de de deedefddZedddejfded eejd!eejd"e d#eddf d$d%ZdS)(DeviceAssignmentzMapping from logical cores in a computation to the physical TPU topology. Prefer to use the `DeviceAssignment.build()` helper to construct a `DeviceAssignment`; it is easier if less flexible than constructing a `DeviceAssignment` directly. rrcCst|tstdt|tj|tjd}||_|j dkr&td|j |j d|_ |j d|_ |j d|j krFtd|j d |j ||_t|j||_d S) aConstructs a `DeviceAssignment` object. Args: topology: A `Topology` object that describes the physical TPU topology. core_assignment: A logical to physical core mapping, represented as a rank 3 numpy array. See the description of the `core_assignment` property for more details. Raises: ValueError: If `topology` is not `Topology` object. ValueError: If `core_assignment` is not a rank 3 numpy array. z*topology must be a Topology object, got {}Zdtypez8core_assignment must be a rank 3 numpy array, got shape rrzAcore_assignment.shape[-1] must have size equal to topology rank (z), got core_assignment.shape=N) isinstancer ValueErrorformattypenpasarrayint32 _topologyndimr _num_replicas_num_cores_per_replicaZ mesh_rank_core_assignmentr_task_and_cores_to_replicas)selfrrrrr__init__Ms0     zDeviceAssignment.__init__returncC|jS)z-A `Topology` that describes the TPU topology.)r-r3rrrrrzDeviceAssignment.topologycCr6)z The number of cores per replica.)r0r7rrrnum_cores_per_replicawr8z&DeviceAssignment.num_cores_per_replicacCr6)z*The number of replicas of the computation.)r/r7rrr num_replicas|r8zDeviceAssignment.num_replicascCr6)zThe logical to physical core mapping. Returns: An integer numpy array of rank 3, with shape `[num_replicas, num_cores_per_replica, topology_rank]`. Maps (replica, logical core) pairs to physical topology coordinates. )r1r7rrrrs z DeviceAssignment.core_assignmentrrcCst|j||ddfS)zReturns the name of the TPU device assigned to a logical core.rA)rr-Ztpu_device_name_at_coordinatesrBrrr tpu_devicerDzDeviceAssignment.tpu_devicercomputation_shapecomputation_strider:device_order_modecCst|||||dS)N)rrFrGr:rH)device_assignment)clsrrFrGr:rHrrrbuilds zDeviceAssignment.build)rr)rrN)rrrrrr*ndarrayr4propertyrintr9r:rrrrr=r>rstrrCrE classmethodrr rKrrrrr"Dsp%    r"x_sizey_sizez_coordr5cCsg}t|dD]+}td|D] }||d||fqt|dddD]}||d|d|fq$qt|dddD] }|d||fq<|S)aRing-order of a X by Y mesh, with a fixed Z coordinate. For example, in a 4x4 mesh, this returns the following order. 0 -- 1 -- 2 -- 3 | | | | 15-- 6 -- 5 -- 4 | | | | 14-- 7 -- 8 -- 9 | | | | 13-- 12-- 11-- 10 Note that chip 0 is not included in the output. Args: x_size: An integer represents the mesh size in the x-dimension. Must be larger than 1. y_size: An integer represents the mesh size in the y-dimension. Must be larger than 1. z_coord: An integer represents the z-coordinate to use for the chips in the ring. Returns: A list of (x,y,z) triples in ring order. rrrr%)r append)rQrRrSretijrrr _open_ring_2dsrXz_sizecs|dkr|dkrddt|DS|dkr"|dkr"ddt|DS|dkr3|dkr3ddt|DS|dkr=|ddksQ|dkrG|ddksQ|dkrw|ddkrwtdg}t|D]t|D]|fd d t|Dqbq\|Sd g}|dkr|t||d|S|dkrd g}|d d t||dD|S|dkrd g}|d d t||dD|Sd g}td|D]}t|||}|ddkr||q|t|qt|dddD] }|dd|fq|S)aRing-order of a X by Y by Z mesh. Constructs the 3d ring from 2d rings that are stacked in the Z dimension and joined in one corner. z == 0: 0 -- 1 -- 2 -- 3 | | | | 15 - 6 -- 5 -- 4 | | | | 14 - 7 -- 8 -- 9 | | | | 13 - 12 - 11 - 10 z == 1: 63 - 30 - 29 - 28 | | | | 16 - 25 - 26 - 27 | | | | 17 - 24 - 23 - 22 | | | | 18 - 19 - 20 - 21 z == 2: 62 - 31 - 32 - 33 | | | | 45 - 36 - 35 - 34 | | | | 44 - 37 - 38 - 39 | | | | 43 - 42 - 41 - 40 z == 3: 61 - 60 - 59 - 58 | | | | 46 - 55 - 56 - 57 | | | | 47 - 54 - 53 - 52 | | | | 48 - 49 - 50 - 51 Args: x_size: An integer represents the mesh size in the x-dimension. Must be larger than 1. y_size: An integer represents the mesh size in the y-dimension. Must be larger than 1. z_size: An integer represents the mesh size in the z-dimension. Must be larger than 1. For example, in a 4x4x4 mesh, this returns the following order. Returns: A list of (x,y,z) triples in ring order. rcSsg|]}dd|fqSrr.0rVrrr 'z_ring_3d..cSsg|]}d|dfqSrZrr[rrrr])r^cSsg|]}|ddfqSrZrr[rrrr]+r^rrz Odd dimensionc3s|]}|fVqdSNr)r\xyzrr 6sz_ring_3d..)rrrcss |] \}}}|||fVqdSr_r)r\r`rcrbrrrrdBcss |] \}}}|||fVqdSr_r)r\rbrcr`rrrrdFrer%)r loggingwarningextendrXreversedrT)rQrRrYrUrVrrrar_ring_3dsR6         rkrrrFrGr:rHc2 sBt|tr t|d}t|tstdt|t|j|j}dur0tjdgtj dntj tj ddurHtjdgtj dntj tj dj fkratddj j fkrrtddj t dkrtd t dkrtd }t ||krtd ||||}|}t|} || krtd || |d d} |jjdkrDdg} |dkr|} } tfddt|DD])\}}|d}tt| d| }t||| |<| | | |} | d8} q| dkr| dksJtj|gdtj d}dko:d|dko:tdko:|| k}|tjkrS|tjkrN|sNtd|tjk}|rg}tddd}t| d| d| d}t|D]N}||\}}}g}ttD]5}|||d\}}}|d|} |d|}!|d|}"||d}#|| |!|"|#gq||qtqt|D]F}|}$g}%t| dg| ddgD]}&|%|$|&|$|&}$qt|%dd|%dgg}'|'}(|'})|(||)||ddf<qɇfddtD}*tjddtj|*ddiDdd}*|* df}*|*|ddtj!ddf}nWtdksNJ|t|j"|j#ks^J|j$}+g}t},t|D].}-g}.t|,D]}|-|,|}/|/|j#}0|/|j#}1|.|+|0|1ddfqt||.qlt%||dS)aComputes a device_assignment of a computation across a TPU topology. Attempts to choose a compact grid of cores for locality. Returns a `DeviceAssignment` that describes the cores in the topology assigned to each core of each replica. `computation_shape` and `computation_stride` values should be powers of 2 for optimal packing. Args: topology: A `Topology` object that describes the TPU cluster topology. To obtain a TPU topology, evaluate the `Tensor` returned by `initialize_system` using `Session.run`. Either a serialized `TopologyProto` or a `Topology` object may be passed. Note: you must evaluate the `Tensor` first; you cannot pass an unevaluated `Tensor` here. computation_shape: A rank 1 int32 numpy array with size equal to the topology rank, describing the shape of the computation's block of cores. If None, the `computation_shape` is `[1] * topology_rank`. computation_stride: A rank 1 int32 numpy array of size `topology_rank`, describing the inter-core spacing of the `computation_shape` cores in the TPU topology. If None, the `computation_stride` is `[1] * topology_rank`. num_replicas: The number of computation replicas to run. The replicas will be packed into the free spaces of the topology. device_order_mode: An enum of `DeviceOrderMode` class which indicates whether to assign devices to form rings or meshes, or let the library to choose. Returns: A DeviceAssignment object, which describes the mapping between the logical cores in each computation replica and the physical cores in the TPU topology. Raises: ValueError: If `topology` is not a valid `Topology` object. ValueError: If `computation_shape` or `computation_stride` are not 1D int32 numpy arrays with shape [3] where all values are positive. ValueError: If computation's replicas cannot fit into the TPU topology. )Z serializedz)`topology` is not a Topology object; got Nrr#z#computation_shape must have shape [z]; got z$computation_stride must have shape [zcomputation_stride must be positive; got computation_stride={}z>computation footprint {} does not fit in TPU topology shape {}zprequested {} replicas but only {} replicas with shape {} and computation_stride {} fit in a TPU mesh of shape {}cSs||d|S)Nrr)nmrrr ceil_of_ratiosz(device_assignment..ceil_of_ratiorc3s$|] \}}||dfVqdS)rNr)r\rVr`) topology_rankrrrdsz$device_assignment..g?r%zdevice_order_mode=DeviceOrderMode.RING is not compatible with the 3D tiling current topology. Try setting device_order_mode=DeviceOrderMode.AUTOrcs*g|]}td|||qSrZ)r*Zaranger[)rFrGrrr]s z%device_assignment..cSsg|] }|dtjfqS).)r*newaxisr[rrrr]sZindexingZij)Zaxis)r)&r&bytesrr'r)len mesh_shaper*arrayr,r+r anyr(prodZmissing_devicessizer enumeraterNmathceilminfullrr r!rkr rTZ concatenateZmeshgridZreshaperqZ num_tasksZnum_tpus_per_taskdevice_coordinatesr")2rrFrGr:rHrtZcomputation_footprintZ block_countsZreplica_countsZ max_replicasrnZ replica_shapeZremaining_replicasZremaining_dimsr`nirVZ target_sizeZreplica_offsetsZenable_3d_tilingZ assignmentZ inner_ringZ outer_ringrZouter_xZouter_yZouter_zZper_replica_assignmentindexZinner_xZinner_yZinner_zZpxpyZpzpitposdimZ replica_posouterinnerindicesr~Zdevices_per_replicarindexZreplica_assignmentZ logical_idrZdevicer)rFrGrorrIVs 0                                 rI)renumrztypingrrrnumpyr*Ztensorflow.python.platformrrfZtensorflow.python.tpu.topologyrZ tensorflow.python.util.tf_exportrZSINGLE_CORE_ASSIGNMENTrIntEnumrobjectr"rNrXrkr rLrIrrrrsT       % h