o `js&@sddlZddlZddlZddlZdZdZdZzddlZWn ey'dZYnwzddl Z Wn ey9dZ Ynwz ddl m Z ddZWn eyQdZ YnwzddlZWn eycdZYnwzddlZWn eyudZYnwzddlZWn eydZYnweedkZdd Zd d Zd d ZddZd&ddZddZd'ddZddZddZe reZnerejZnerej ZndZddZ!e re j"Z#n ere!Z#nerej"Z#ndZ#ddZ$d d!Z%d"d#Z&d$d%Z'dS)(N) iSNAPrYrr bccccccBiiicKsZz|ddtj|fddi|WSty,|ddtj|fddi|YSw)N block_linkedF block_moder)poplz4compress TypeError)payloadkwargsrI/home/djax/ivt_ai_plugin/venv/lib/python3.10/site-packages/kafka/codec.py _lz4_compresss   rPyPycCsdS)NTrrrrrhas_gzip7srcCtduSN)snappyrrrr has_snappy;rcCrr)zstdrrrrhas_zstd?rrcCs(tdurdStdur dStdurdSdS)NTF)r lz4f lz4framedrrrrhas_lz4CsrcCsH|sd}t}tj|d|d}z||W||S|w)N w)fileobjmode compresslevel)ioBytesIOgzipGzipFilewriteclosegetvalue)rr$bufgzipperrrr gzip_encodeMs  r.cCs6t|}tj|dd}z |W|S|w)Nr)r"r#)r%r&r'r(readr*)rr,r-rrr gzip_decode^s r1Tc ststd|stSt}tttD]\}}| t d||qt r.ddnddfddt dtDD]}t|}t|}| t d || |qC|S) aEncodes the given data with snappy compression. If xerial_compatible is set then the stream is encoded in a fashion compatible with the xerial snappy library. The block size (xerial_blocksize) controls how frequent the blocking occurs 32k is the default in the xerial library. The format winds up being: +-------------+------------+--------------+------------+--------------+ | Header | Block1 len | Block1 data | Blockn len | Blockn data | +-------------+------------+--------------+------------+--------------+ | 16 bytes | BE int32 | snappy bytes | BE int32 | snappy bytes | +-------------+------------+--------------+------------+--------------+ It is important to note that the blocksize is the amount of uncompressed data presented to snappy at each block, whereas the blocklen is the number of bytes that will be present in the stream; so the length will always be <= blocksize. Snappy codec is not available!cSs||||Srrrisizerrrszsnappy_encode..cSst||||Sr) memoryviewtobytesr5rrrr8sc3s|] }|VqdSrr).0r6chunkerrxerial_blocksizerr sz snappy_encode..r!i)rNotImplementedErrorrr r%r&zip_XERIAL_V1_FORMAT_XERIAL_V1_HEADERr)structpackPYPYrangelenr+) rxerial_compatibler>outfmtdatchunkblock block_sizerr<r snappy_encodejs$    rQcCspt|dkr6tdtddt|dd}tdtddt|dd\}}|tddkr6dSdS)aDetects if the data given might have been encoded with the blocking mode of the xerial snappy library. This mode writes a magic header of the format: +--------+--------------+------------+---------+--------+ | Marker | Magic String | Null / Pad | Version | Compat | +--------+--------------+------------+---------+--------+ | byte | c-string | byte | int32 | int32 | +--------+--------------+------------+---------+--------+ | -126 | 'SNAPPY' | | | | +--------+--------------+------------+---------+--------+ The pad appears to be to ensure that SNAPPY is a valid cstring The version is the version of this format as written by xerial, in the wild this is currently 1 as such we only support v1. Compat is there to claim the minimum supported version that can read a xerial block stream, presently in the wild this is 1. r4NTF)rIrEunpackrCbytesrD)rmagicversioncompatrrr_detect_xerial_streams $(rYcCststdt|rNt}|dd}t|}d}||krEtd||dd}|d7}||}|t ||||}||ks| d| St |S)Nr3rRrr@) rrArYr%r&rIrE unpack_fromr)r decompressseekr0)rrKbytlengthcursorrPendrrr snappy_decodes"   rbcCs:t}t||}t||ddkrtd|dS)zFDecode payload using interoperable LZ4 framing. Requires Kafka >= 0.10nextrz&lz4f unable to decompress full payloaddecomp)rcreateDecompContextdecompressFramefreeDecompContext RuntimeError)rctxdatarrr lz4f_decodes    rkcCstdurtdt|}d}|d}t|tst|}|d?d@}|r<|d8}t|}||d<t|}||dd}n||d}t|d|d d d }d |d|d||gS) zLEncode payload for 0.8/0.9 brokers -- requires an incorrect header checksum.N8pip install xxhash for lz4 encoding with 0.8/0.9 brokersrZrrSr) xxhashrh lz4_encode isinstanceintord bytearrayrUxxh32digestjoin)rrj header_sizeflgcontent_size_bithcrrrlz4_encode_old_kafkas*   "rcCstdurtdd}t|dtr|d}nt|d}|d?d@}|r(|d7}t|d|ddd}d |d |d|||dg}t|S) NrlrmrZrnrrSrorprqr) rrrhrtrurvrxryrz lz4_decode)rr{r|r}r~munged_payloadrrrlz4_decode_old_kafka#s    " rcCststdt|S)NZstd codec is not available)rrAZstdCompressorr )rrrr zstd_encode;srcCsPtstdtjt|dd }|WdS1s!wYdS)NrT)read_across_frames)rrAZstdDecompressor stream_readerr%r&r0)rreaderrrr zstd_decodeAs $rr)Tr2)(r'r%platformrErDrCZSTD_MAX_OUTPUT_SIZEr ImportError zstandardr lz4.frameframer rrrrrboolpython_implementationrGrrrrr.r1rQrYrbrs compressFramer rkr\rrrrrrrrrs               ;#