o ?e@sdZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd l m Zdd d ZGd ddejZdS)z0The implementation of `tf.data.Dataset.rebatch`.N) dataset_ops)nest)dtypes)ops) tensor_shape) tensor_util)gen_experimental_dataset_opsFcCst||||SN)_RebatchDataset) input_datasetZ batch_sizedrop_remaindernamerf/home/www/facesmatcher.com/pyenv/lib/python3.10/site-packages/tensorflow/python/data/ops/rebatch_op.py_rebatchsrcsBeZdZdZ  d fdd ZddZdd Zed d ZZ S) r aeA `Dataset` that rebatches elements from its input into new batch sizes. `_RebatchDataset(input_dataset, batch_sizes)` is functionally equivalent to `input_dataset.unbatch().batch(N)`, where the value of N cycles through the `batch_sizes` input list. The elements produced by this dataset have the same rank as the elements of the input dataset. FNcs||_tj|tjdd|_tj|tjdd|_||_| t fddt ||_t |}tj|jf||d|j}t||dS)z"See `Dataset.rebatch` for details. batch_sizes)Zdtyper r cs|Sr )Z_unbatchZ_batch)ts new_batch_dimrr9sz*_RebatchDataset.__init__..)rr N)_input_datasetrZconvert_to_tensorrZint64 _batch_sizesbool_drop_remainder_name_compute_static_batch_dimrZ map_structurer get_structure _element_specZnormalize_to_denseged_opsZrebatch_dataset_v2Z_variant_tensorZ_flat_structuresuper__init__)selfr rr r Zvariant_tensor __class__rrr )s.  z_RebatchDataset.__init__cCst|j}|dur dSt|tjr;t|jdkr)t||dkr'|d}q;dSt|jdkr;t dt|jd| |rBdS|S)a0Computes the static batch dimension of a dataset if it can be determined. Given the RebatchDataset parameters, determines the batch dimension of this dataset statically. Returns None if this cannot be determined or is variable. Returns: An integer representing the batch dimension of the dataset. If it cannot be determined statically, returns None. Raises: ValueError: The batch_sizes parameter is malformed, input_dataset is not batched, or input_dataset batch sizes are incompatible with each other. NrziInvalid `batch_sizes`. Expected `batch_sizes` to be a scalar or a vector. Received `batch_sizes` of rank .) rconstant_valuer isinstancenpZndarraylenshapeall ValueError_may_form_partial_batches)r!rrrrrIs"    z)_RebatchDataset._compute_static_batch_dimcst|jrdSddfddtt|jD}dd|D}|s'dSt |}t ||dksBt d |dd |d |d|dkS) z6Returns whether this dataset may form partial batches.FcSs`z|}Wn tyYdSwt|tjsdS|jdur dSt|dkr*td|jdj S)Nr$zInvalid `batch_sizes`. Expected dataset with rank of >= 1 but found a dataset with scalar elements. Fix the issue by adding the `batch` transformation to the dataset.r) Z_to_legacy_output_shapesNotImplementedErrorr'rZ TensorShapeZrankr)r,dimsvalue)Z type_specr*rrr get_batch_dimss      z@_RebatchDataset._may_form_partial_batches..get_batch_dimcsg|]}|qSrr).0rr1rr sz=_RebatchDataset._may_form_partial_batches..cSsg|]}|dur|qSr r)r2drrrr4sTrz?Invalid `input_dataset.` The batch dimension of component 0 is z., while the batch dimension of component i is r%) rr&rrflattenrrrr(Zasarrayr+r,)r!Zdesired_batch_sizeZinput_batch_dimsZknown_input_batch_dimsrr3rr-ns&   z)_RebatchDataset._may_form_partial_batchescCs|jSr )r)r!rrr element_specsz_RebatchDataset.element_specFN) __name__ __module__ __qualname____doc__r rr-propertyr7 __classcell__rrr"rr s  %'r r8)r<numpyr(Ztensorflow.python.data.opsrZtensorflow.python.data.utilrZtensorflow.python.frameworkrrrrZtensorflow.python.opsrrrZ UnaryDatasetr rrrrs