a Raa~@sddlmZmZmZddlmZddlmZmZddl Z ddl Z ddl m Z m Z ddlmZddlmZmZmZmZdd lmZdd lmZed d eDZed d eDZedd eDZeeddgBZdZejreddkreddksJe edde!ddZ"n e eZ"hdZ#e dZ$iZ%Gddde&Z'ddZ(Gddde&Z)Gddde)Z*Gdd d e+Z,Gd!d"d"e&Z-Gd#d$d$e&Z.d%d&Z/dS)')absolute_importdivisionunicode_literals) text_type) http_clienturllibN)BytesIOStringIO) webencodings)EOFspaceCharacters asciiLettersasciiUppercase)_ReparseException)_utilscCsg|]}|dqSasciiencode.0itemrR/opt/alt/python39/lib/python3.9/site-packages/pip/_vendor/html5lib/_inputstream.py rcCsg|]}|dqSrrrrrrrrcCsg|]}|dqSrrrrrrrr> i iii iiiiiiiiii iii ii i iiiiii i i ii iiz[ - -/:-@\[-`{-~]c@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS)BufferedStreamzBuffering for streams that do not have buffering of their own The buffer is implemented as a list of chunks on the assumption that joining many strings will be slow since it is O(n**2) cCs||_g|_ddg|_dS)Nrr)streambufferposition)selfr"rrr__init__:szBufferedStream.__init__cCs<d}|jd|jdD]}|t|7}q||jd7}|SNrr )r#r$len)r%poschunkrrrtell?s zBufferedStream.tellcCsT||ksJ|}d}t|j||krF|t|j|8}|d7}q||g|_dSr')_bufferedBytesr(r#r$)r%r)offsetirrrseekFs zBufferedStream.seekcCsT|js||S|jdt|jkrF|jdt|jdkrF||S||SdS)Nrr r)r# _readStreamr$r(_readFromBufferr%bytesrrrreadOs  zBufferedStream.readcCstdd|jDS)NcSsg|] }t|qSr)r(rrrrrYrz1BufferedStream._bufferedBytes..)sumr#r%rrrr,XszBufferedStream._bufferedBytescCs<|j|}|j||jdd7<t||jd<|Sr')r"r4r#appendr$r()r%r3datarrrr0[s   zBufferedStream._readStreamcCs|}g}|jd}|jd}|t|jkr|dkr|dks>J|j|}|t||krl|}|||g|_n"t||}|t|g|_|d7}|||||||8}d}q|r|||d|S)Nrr r)r$r(r#r7r0join)r%r3ZremainingBytesrvZ bufferIndexZ bufferOffsetZ bufferedDataZ bytesToReadrrrr1bs&     zBufferedStream._readFromBufferN) __name__ __module__ __qualname____doc__r&r+r/r4r,r0r1rrrrr!3s  r!cKst|tjs(t|tjjr.t|jtjr.d}n&t|drJt|dt }n t|t }|rdd|D}|rvt d|t |fi|St |fi|SdS)NFr4rcSsg|]}|dr|qS)Z _encoding)endswith)rxrrrrrz#HTMLInputStream..z3Cannot set an encoding with a unicode input, set %r) isinstancerZ HTTPResponserZresponseZaddbasefphasattrr4r TypeErrorHTMLUnicodeInputStreamHTMLBinaryInputStream)sourcekwargsZ isUnicodeZ encodingsrrrHTMLInputStream}s      rIc@speZdZdZdZddZddZddZd d Zd d Z d dZ dddZ ddZ ddZ dddZddZdS)rEProvides a unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. i(cCsZtjsd|_ntddkr$|j|_n|j|_dg|_tddf|_| ||_ | dS)Initialises the HTMLInputStream. HTMLInputStream(source, [encoding]) -> Normalized stream from source for use by html5lib. source can be either a file-object, local filename or a string. The optional encoding parameter must be a string that indicates the encoding. If specified, that encoding will be used, regardless of any BOM or later declaration (such as in a meta element) Nu􏿿r rutf-8certain) rsupports_lone_surrogatesreportCharacterErrorsr(characterErrorsUCS4characterErrorsUCS2ZnewLineslookupEncoding charEncoding openStream dataStreamreset)r%rGrrrr&s   zHTMLUnicodeInputStream.__init__cCs.d|_d|_d|_g|_d|_d|_d|_dS)Nr)r* chunkSize chunkOffseterrors prevNumLines prevNumCols_bufferedCharacterr6rrrrVszHTMLUnicodeInputStream.resetcCst|dr|}nt|}|SzvProduces a file object from source. source can be either a file object, local filename or a string. r4)rCr r%rGr"rrrrTs z!HTMLUnicodeInputStream.openStreamcCsT|j}|dd|}|j|}|dd|}|dkr@|j|}n ||d}||fS)N rrr )r*countr[rfindr\)r%r-r*ZnLinesZ positionLineZ lastLinePosZpositionColumnrrr _positions   z HTMLUnicodeInputStream._positioncCs||j\}}|d|fS)z:Returns (line, col) of the current position in the stream.r )rcrY)r%linecolrrrr$szHTMLUnicodeInputStream.positioncCs6|j|jkr|stS|j}|j|}|d|_|S)zo Read one character from the stream or queue if available. Return EOF when EOF is reached. r )rYrX readChunkr r*)r%rYcharrrrrgs   zHTMLUnicodeInputStream.charNcCs|dur|j}||j\|_|_d|_d|_d|_|j|}|j rX|j |}d|_ n|s`dSt |dkrt |d}|dksd|krdkrnn|d|_ |dd}|j r| || d d }| d d }||_t ||_d S) NrWrFr r iz r` T)_defaultChunkSizercrXr[r\r*rYrUr4r]r(ordrOreplace)r%rXr8Zlastvrrrrfs0           z HTMLUnicodeInputStream.readChunkcCs(ttt|D]}|jdqdS)Ninvalid-codepoint)ranger(invalid_unicode_refindallrZr7)r%r8_rrrrPsz*HTMLUnicodeInputStream.characterErrorsUCS4cCsd}t|D]}|rqt|}|}t|||drrt|||d}|tvrl|j dd}q|dkr|dkr|t |dkr|j dqd}|j dqdS)NFrnTriir ) rpfinditerrlgroupstartrZisSurrogatePairZsurrogatePairToCodepointnon_bmp_invalid_codepointsrZr7r()r%r8skipmatchZ codepointr)Zchar_valrrrrQ#s"  z*HTMLUnicodeInputStream.characterErrorsUCS2Fc Cszt||f}Wnftyv|D]}t|dks"Jq"ddd|D}|sXd|}td|}t||f<Yn0g}||j|j}|dur|j|j krqn0| }||j kr| |j|j|||_q| |j|jd| s|qq|d|} | S)z Returns a string of characters from the stream up to but not including any character in 'characters' or EOF. 'characters' must be a container that supports the 'in' method and iteration over its characters. rWcSsg|]}dt|qS)z\x%02x)rl)rcrrrrHrz5HTMLUnicodeInputStream.charsUntil..z^%sz[%s]+N) charsUntilRegExKeyErrorrlr9recompileryr*rYrXendr7rf) r%Z charactersZoppositecharsr{Zregexr:mrrrrr charsUntil:s0     z!HTMLUnicodeInputStream.charsUntilcCsT|turP|jdkr.||j|_|jd7_n"|jd8_|j|j|ksPJdSr')r rYr*rX)r%rgrrrungetis   zHTMLUnicodeInputStream.unget)N)F)r;r<r=r>rkr&rVrTrcr$rgrfrPrQrrrrrrrEs   & /rEc@sLeZdZdZdddZddZd d Zdd d Zd dZddZ ddZ dS)rFrJN windows-1252TcCsn|||_t||jd|_d|_||_||_||_||_ ||_ | ||_ |j ddusbJ| dS)rKidrN)rT rawStreamrEr& numBytesMetanumBytesChardetoverride_encodingtransport_encodingsame_origin_parent_encodinglikely_encodingdefault_encodingdetermineEncodingrSrV)r%rGrrrrrZ useChardetrrrr&s  zHTMLBinaryInputStream.__init__cCs&|jdj|jd|_t|dS)Nrrm)rSZ codec_info streamreaderrrUrErVr6rrrrVszHTMLBinaryInputStream.resetcCsJt|dr|}nt|}z||WntyDt|}Yn0|Sr^)rCrr/r+ Exceptionr!r_rrrrTs  z HTMLBinaryInputStream.openStreamcCs|df}|ddur|St|jdf}|ddur:|St|jdf}|ddurX|S|df}|ddurt|St|jdf}|ddur|djds|St|jdf}|ddur|S|rnzddl m }Wnt yYn0g}|}|j s:|j |j}t|tsJ|s$q:||||q|t|jd}|j d|durn|dfSt|jdf}|ddur|StddfS)NrMrZ tentativezutf-16)UniversalDetectorencodingr) detectBOMrRrrdetectEncodingMetarname startswithrZ%pip._vendor.chardet.universaldetectorr ImportErrorZdonerr4rrAr3r7Zfeedcloseresultr/r)r%ZchardetrSrZbuffersZdetectorr#rrrrrsR            z'HTMLBinaryInputStream.determineEncodingcCs|jddksJt|}|dur&dS|jdvrFtd}|dusJnT||jdkrf|jddf|_n4|jd|df|_|td|jd|fdS)Nr rMutf-16beutf-16lerLrzEncoding changed from %s to %s)rSrRrrr/rVr)r%Z newEncodingrrrchangeEncodings   z$HTMLBinaryInputStream.changeEncodingc Cstjdtjdtjdtjdtjdi}|jd}t|t sr&rVrTrrrrrrrrrFzs * >"rFc@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ e e e Z ddZe eZefddZddZddZddZdS) EncodingByteszString-like object with an associated position and various extra methods If the position is ever greater than the string length then an exception is raisedcCst|tsJt||SN)rAr3__new__lowerr%valuerrrrFszEncodingBytes.__new__cCs d|_dS)Nr)rcrrrrr&JszEncodingBytes.__init__cCs|Srrr6rrr__iter__NszEncodingBytes.__iter__cCs>|jd}|_|t|kr"tn |dkr.t|||dS)Nr rrcr( StopIterationrDr%prrr__next__Qs  zEncodingBytes.__next__cCs|Sr)rr6rrrnextYszEncodingBytes.nextcCsB|j}|t|krtn |dkr$t|d|_}|||dSr'rrrrrprevious]s zEncodingBytes.previouscCs|jt|krt||_dSrrcr(r)r%r$rrr setPositionfszEncodingBytes.setPositioncCs*|jt|krt|jdkr"|jSdSdS)Nrrr6rrr getPositionks  zEncodingBytes.getPositioncCs||j|jdSNr )r$r6rrrgetCurrentByteuszEncodingBytes.getCurrentBytecCsH|j}|t|kr>|||d}||vr4||_|S|d7}q||_dS)zSkip past a list of charactersr Nr$r(rcr%rrr{rrrrxzs  zEncodingBytes.skipcCsH|j}|t|kr>|||d}||vr4||_|S|d7}q||_dSrrrrrr skipUntils  zEncodingBytes.skipUntilcCs(|||j}|r$|jt|7_|S)zLook for a sequence of bytes at the start of a string. If the bytes are found return True and advance the position to the byte after the match. Otherwise return False and leave the position alone)rr$r()r%r3r:rrr matchBytesszEncodingBytes.matchBytescCs<z |||jt|d|_Wnty6tYn0dS)zLook for the next sequence of bytes matching a given sequence. If a match is found advance the position to the last byte of the matchr T)indexr$r(rc ValueErrorrr2rrrjumpTos    zEncodingBytes.jumpToN)r;r<r=r>rr&rrrrrrpropertyr$r currentBytespaceCharactersBytesrxrrrrrrrrBs      rc@sXeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ dS)rz?Mini parser for detecting character encoding from meta elementscCst||_d|_dS)z3string - the data to work on for encoding detectionN)rr8rr%r8rrrr&s zEncodingParser.__init__c Csd|jvrdSd|jfd|jfd|jfd|jfd|jfd|jff}|jD]}d}z|jdWntyxYqYn0|D]B\}}|j|r~z|}WqWq~tyd}YqYq~0q~|sHqqH|j S) Nsr8rr6rrrrszEncodingParser.handleCommentcCs|jjtvrdSd}d}|}|dur,dS|ddkr\|ddk}|r|dur||_dSq|ddkr|d}t|}|dur||_dSq|ddkrtt|d}|}|durt|}|dur|r||_dS|}qdS) NTFrs http-equivr s content-typecharsetscontent) r8rr getAttributerrRContentAttrParserrparse)r%Z hasPragmaZpendingEncodingattrZtentativeEncodingcodecZ contentParserrrrrs8      zEncodingParser.handleMetacCs |dS)NF)handlePossibleTagr6rrrrsz%EncodingParser.handlePossibleStartTagcCst|j|dS)NT)rr8rr6rrrrs z#EncodingParser.handlePossibleEndTagcCsb|j}|jtvr(|r$||dS|t}|dkrD|n|}|dur^|}qLdS)NTr)r8rasciiLettersBytesrrrspacesAngleBracketsr)r%ZendTagr8r{rrrrrs    z EncodingParser.handlePossibleTagcCs |jdS)Nrrr6rrrrszEncodingParser.handleOthercCs|j}|ttdgB}|dus2t|dks2J|dvr>dSg}g}|dkrV|rVqnX|tvrj|}qnD|dvrd|dfS|tvr||n|durdS||t |}qF|dkr| d|dfSt ||}|dvrJ|}t |}||kr"t |d|d|fS|tvr<||q||qnJ|d krbd|dfS|tvr|||n|durdS||t |}|t vrd|d|fS|tvr||n|durdS||qdS) z_Return a name,value pair for the next attribute in the stream, if one is found, or None/Nr )rN=)rrr)'"r) r8rxr frozensetr(r9asciiUppercaseBytesr7rrrr)r%r8r{ZattrNameZ attrValueZ quoteCharrrrrsb             zEncodingParser.getAttributeN) r;r<r=r>r&rrrrrrrrrrrrrs$rc@seZdZddZddZdS)rcCst|tsJ||_dSr)rAr3r8rrrrr&aszContentAttrParser.__init__cCsz|jd|jjd7_|j|jjdkssH      JgIb='