Team Ai
Datasetpublic

codekingpro/portable-devtools

sourceHugging Faceupdated 5mo agoView on Hugging Face
1likes14kdownloads
lexer.cpython-313.pyc330 linesDownload Raw Back to __pycache__
1�

2�jR���F�SrSSKrSSKrSSKrSSKJrJr SSKJr SSK	J3r4JrJrJ
r
Jr SSKJrJrJrJrJrJr SSKJr /SQr\R2"S	5r/S5Qr\"S5r"SS
\5r"SS\S9r "SS\ 5r!"SS\"5r#"SS5r$\$"5r%"SS\&5r'"SS5r(Sr)"SS5r*\*"5r+Sr,"SS 5r-"S!S"\5r."S#S$\5r/"S%S&\ \/S9r0"S'S(5r1"S)S*\05r2S+r3"S,S-\/5r4"S.S/\0\4S9r5g)0z�6pygments.lexer7~~~~~~~~~~~~~~8 9Base lexer classes.10 11:copyright: Copyright 2006-present by the Pygments team, see AUTHORS.12:license: BSD, see LICENSE for details.13�N)�
apply_filters�Filter)�get_filter_by_name)�Error�Text�Other�14Whitespace�15_TokenType)�get_bool_opt�get_int_opt�get_list_opt�make_analysator�Future�guess_decode)�	regex_opt)
�Lexer�16RegexLexer�ExtendedRegexLexer�DelegatingLexer�LexerContext�include�inherit�bygroups�using�this�default�words�line_rez.*?17))s�utf-8)s��zutf-32)s��zutf-32be)s��zutf-16)s��zutf-16bec��g)N��)�xs �TD:\code\apps\devtools\python\user_packages\Python313\site-packages\pygments/lexer.py�<lambda>r%"s��#�c��\rSrSrSrSrSrg)�	LexerMeta�%zv18This metaclass automagically converts ``analyse_text`` methods into19static methods which always return float values.20c�\�SU;a[US5US'[RXX#5$)N�analyse_text)r�type�__new__)�mcs�name�bases�ds    r$r-�LexerMeta.__new__+s/���Q�� /��.�0A� B�A�n���|�|�C�u�0�0r&r"N)�__name__�21__module__�__qualname__�__firstlineno__�__doc__r-�__static_attributes__r"r&r$r(r(%s���221r&r(c�j�\rSrSrSrSr/r/r/r/r	Sr23SrSrSr
SrSrSrSrS	rS
S24jrSrSrg)r�1an25Lexer for a specific language.26 27See also :doc:`lexerdevelopment`, a high-level guide to writing28lexers.29 30Lexer classes have attributes used for choosing the most appropriate31lexer based on various criteria.32 33.. autoattribute:: name34   :no-value:35.. autoattribute:: aliases36   :no-value:37.. autoattribute:: filenames38   :no-value:39.. autoattribute:: alias_filenames40.. autoattribute:: mimetypes41   :no-value:42.. autoattribute:: priority43 44Lexers included in Pygments should have two additional attributes:45 46.. autoattribute:: url47   :no-value:48.. autoattribute:: version_added49   :no-value:50 51Lexers included in Pygments may have additional attributes:52 53.. autoattribute:: _example54   :no-value:55 56You can pass options to the constructor. The basic options recognized57by all lexers and processed by the base `Lexer` class are:58 59``stripnl``60    Strip leading and trailing newlines from the input (default: True).61``stripall``62    Strip all leading and trailing whitespace from the input63    (default: False).64``ensurenl``65    Make sure that the input ends with a newline (default: True).  This66    is required for some lexers that consume input linewise.67 68    .. versionadded:: 1.369 70``tabsize``71    If given and greater than 0, expand tabs in the input (default: 0).72``encoding``73    If given, must be an encoding name. This encoding will be used to74    convert the input string to Unicode, if it is not already a Unicode75    string (default: ``'guess'``, which uses a simple UTF-8 / Locale /76    Latin1 detection.  Can also be ``'chardet'`` to use the chardet77    library, if it is installed.78``inencoding``79    Overrides the ``encoding`` if given.80Nrc�x�Xl[USS5Ul[USS5Ul[USS5Ul[USS5UlURSS	5UlURS815=(d URUl/Ul	[USS5HnURU5 M g
)a)82This constructor takes arbitrary options as keyword arguments.83Every subclass must first process its own options and then call84the `Lexer` constructor, since it processes the basic85options like `stripnl`.86 87An example looks like this:88 89.. sourcecode:: python90 91   def __init__(self, **options):92       self.compress = options.get('compress', '')93       Lexer.__init__(self, **options)94 95As these options must all be specifiable as strings (due to the96command line usage), there are various utility functions97available to help with that, see `Utilities`_.98�stripnlT�stripallF�ensurenl�tabsizer�encoding�guess�99inencoding�filtersr"N)�optionsrr<r=r>rr?�getr@rCr
�100add_filter)�selfrD�filter_s   r$�__init__�Lexer.__init__�s���&��#�G�Y��=���$�W�j�%�@��
�$�W�j�$�?��
�"�7�I�q�9������J��8��
����L�1�B�T�]�]��
����#�G�Y��;�G��O�O�G�$�<r&c��UR(a(SURRSUR<S3$SURRS3$)Nz<pygments.lexers.z with �>)rD�	__class__r3�rGs r$�__repr__�Lexer.__repr__�sI���<�<�&�t�~�~�'>�'>�&?�v�d�l�l�EU�UV�W�W�&�t�~�~�'>�'>�&?�q�A�Ar&c�|�[U[5(d[U40UD6nURR	U5 g)z(101Add a new stream filter to this lexer.102N)�103isinstancerrrC�append)rGrHrDs   r$rF�Lexer.add_filter�s2���'�6�*�*�(��<�G�<�G������G�$r&c��g)aa104A static method which is called for lexer guessing.105 106It should analyse the text and return a float in the range107from ``0.0`` to ``1.0``.  If it returns ``0.0``, the lexer108will not be selected as the most probable one, if it returns109``1.0``, it will be selected immediately.  This is used by110`guess_lexer`.111 112The `LexerMeta` metaclass automatically wraps this function so113that it works like a static method (no ``self`` or ``cls``114parameter) and the return value is automatically converted to115`float`. If the return value is an object that is boolean `False`116it's the same as if the return values was ``0.0``.117Nr")�texts r$r+�Lexer.analyse_text�s�r&c�0�[U[5(d�URS:Xa[U5upGOURS:Xa�SSKnSn[H;upgURU5(dMU[U5SRUS5n O Uc>URUSS5nURURS5=(d S	S5nUnOdURUR5nURS1185(aU[S1195SnO$URS1205(aU[S1215SnURSS5nURS
S5nUR(aUR5nO"UR(aURS5nUR S:�aUR#UR 5nUR$(aUR'S5(dUS-
nU$![122an[S5UeSnAff=f)zVApply preprocessing such as decoding the input, removing BOM and normalizing newlines.rA�chardetrNzkTo enable chardet encoding guessing, please install the chardet library from http://chardet.feedparser.org/�replaceir@ruz
123�124�
)rR�strr@rrY�ImportError�
_encoding_map�125startswith�len�decode�detectrErZr=�stripr<r?�126expandtabsr>�endswith)	rGrV�_rY�e�decoded�bomr@�encs	         r$�_preprocess_lexer_input�Lexer._preprocess_lexer_input�s����$��$�$��}�}��'�&�t�,���a����)�+�T�"���%2�M�C����s�+�+�"&�s�3�x�y�/�"8�"8��9�"M���&3�127�?�!�.�.��e�t��5�C�"�k�k�#�'�'�*�*=�*H��*3�5�G����{�{�4�=�=�1���?�?�8�,�,���H�
��/�D�����x�(�(��C��M�N�+���|�|�F�D�)���|�|�D�$�'���=�=��:�:�<�D�
�\�\��:�:�d�#�D��<�<�!���?�?�4�<�<�0�D��=�=����t�!4�!4��D�L�D����I#�T�%�'L�M�RS�T��T�s�G:�:128H�H�Hc�^^�TRT5mUU4SjnU"5nU(d[UTRT5nU$)a129This method is the basic interface of a lexer. It is called by130the `highlight()` function. It must process the text and return an131iterable of ``(tokentype, value)`` pairs from `text`.132 133Normally, you don't need to override this method. The default134implementation processes the options recognized by all lexers135(`stripnl`, `stripall` and so on), and then yields all tokens136from `get_tokens_unprocessed()`, with the ``index`` dropped.137 138If `unfiltered` is set to `True`, the filtering mechanism is139bypassed even if filters are defined.140c3�N># �TRT5HupnX4v� M
 g7f�N)�get_tokens_unprocessed)rg�t�vrGrVs   ��r$�streamer�"Lexer.get_tokens.<locals>.streamer
s&�����6�6�t�<���a��d�141�=�s�"%)rlrrC)rGrV�142unfilteredrt�streams``   r$�143get_tokens�Lexer.get_tokens�s=����+�+�D�1��	�����"�6�4�<�<��>�F��
r&c��[e)a#144This method should process the text and return an iterable of145``(index, tokentype, value)`` tuples where ``index`` is the starting146position of the token within the input text.147 148It must be overridden by subclasses. It is recommended to149implement it as a generator to maximize effectiveness.150)�NotImplementedError)rGrVs  r$rq�Lexer.get_tokens_unprocesseds151��"�!r&)r@r>rCrDr=r<r?)F)r3r4r5r6r7r/�aliases�	filenames�alias_filenames�	mimetypes�priority�url�
version_added�_examplerIrOrFr+rlrxrqr8r"r&r$rr1sl��8�v�D��G�152�I��O��I��H��C��M��H�%�<B�%��"-�^�0	"r&r)�	metaclassc�*�\rSrSrSr\4SjrSrSrg)ri!a153This lexer takes two lexer as arguments. A root lexer and154a language lexer. First everything is scanned using the language155lexer, afterwards all ``Other`` tokens are lexed using the root156lexer.157 158The lexers from the ``template`` lexer package use this base lexer.159c�r�U"S0UD6UlU"S0UD6UlX0l[R"U40UD6 g�Nr")�160root_lexer�language_lexer�needlerrI)rG�_root_lexer�_language_lexer�_needlerDs     r$rI�DelegatingLexer.__init__+s7��%�0��0���-�8��8�����
���t�'�w�'r&c��Sn/n/nURRU5HRupVnX`RLa+U(aUR[	U5U45 /nX'-
nM?URXVU45 MT U(aUR[	U5U45 [UURRU55$)N�)r�rqr�rSra�
do_insertionsr�)rGrV�buffered�161insertions�162lng_buffer�irrrss        r$rq�&DelegatingLexer.get_tokens_unprocessed1s������163��164��*�*�A�A�$�G�G�A�!��K�K����%�%�s�8�}�j�&A�B�!#�J��
���!�!�1��)�,�H�����s�8�}�j�9�:��Z�!�_�_�C�C�H�M�O�	Or&)r�r�r�N)	r3r4r5r6r7rrIrqr8r"r&r$rr!s���>C�(�Or&rc��\rSrSrSrSrg)riHzA165Indicates that a state should include rules from another state.166r"N�r3r4r5r6r7r8r"r&r$rrHs���	r&rc��\rSrSrSrSrSrg)�_inheritiOz;167Indicates the a state should inherit from its superclass.168c��g)Nrr"rNs r$rO�_inherit.__repr__Ss��r&r"N)r3r4r5r6r7rOr8r"r&r$r�r�Os���r&r�c�$�\rSrSrSrSrSrSrg)�combinediYz2169Indicates a state combined from multiple states.170c�,�[RX5$rp)�tupler-)�cls�argss  r$r-�combined.__new__^s���}�}�S�'�'r&c��grpr")rGr�s  r$rI�combined.__init__as��r&r"N)r3r4r5r6r7r-rIr8r"r&r$r�r�Ys���(�
r&r�c�H�\rSrSrSrSrSSjrSSjrSSjrSr	S	r171S172rg)�_PseudoMatchifz2173A pseudo match object constructed from a string.174c��X lXlgrp)�_text�_start)rG�startrVs   r$rI�_PseudoMatch.__init__ks���175��r&Nc��UR$rp)r��rG�args  r$r��_PseudoMatch.startos���{�{�r&c�F�UR[UR5-$rp)r�rar�r�s  r$�end�_PseudoMatch.endrs���{�{�S����_�,�,r&c�>�U(a[S5eUR$)Nz
No such group)�176IndexErrorr�r�s  r$�group�_PseudoMatch.groupus����_�-�-��z�z�r&c��UR4$rp)r�rNs r$�groups�_PseudoMatch.groupszs���177�178�}�r&c��0$rpr"rNs r$�	groupdict�_PseudoMatch.groupdict}s���	r&)r�r�rp)r3r4r5r6r7rIr�r�r�r�r�r8r"r&r$r�r�fs%�����-��179�r&r�c�^�SU4SjjnU$)zD180Callback that yields multiple actions for each group in the match.181c1823��># �[T5H�up4UcM183[U5[La7URUS-5nU(aUR	US-5XE4v� MQMSURUS-5nUcMlU(aUR	US-5UlU"U[
UR	US-5U5U5HnU(dMUv� M M� U(aUR5Ulgg7f)N�)�	enumerater,r184r�r��posr�r�)�lexer�match�ctxr��action�data�itemr�s       �r$�callback�bygroups.<locals>.callback�s�����"�4��I�A��~���f���+��{�{�1�q�5�)����+�+�a�!�e�,�f�:�:���{�{�1�q�5�)���#��"'�+�+�a�!�e�"4��� &�u�'3�E�K�K��A��4F��'M�s�!T���4�"&�J�!T�)� ��i�i�k�C�G��s�A4C7�;AC7�
*C7rpr")r�r�s` r$rr�s���"�&�Or&c��\rSrSrSrSrg)�_Thisi�zL185Special singleton used for indicating the caller class.186Used by ``using``.187r"Nr�r"r&r$r�r��s��r&r�c��^^^�0mST;a9TRS5n[U[[45(aUTS'OSU4TS'T[LaSUU4SjjnU$SUUU4SjjnU$)a�188Callback that processes the match with a different lexer.189 190The keyword arguments are forwarded to the lexer, except `state` which191is handled separately.192 193`state` specifies the state that the new lexer will start in, and can194be an enumerable such as ('root', 'inline', 'string') or a simple195string which is assumed to be on top of the root state.196 197Note: For that to work, `_other` must not be an `ExtendedRegexLexer`.198�state�stack�rootc3�R># �T199(a9[UR5nURT2005 UR"S0UD6nOUnUR	5nUR201"UR
540T	D6HupgnXe-Xx4v� M U(aUR5Ulgg7fr�)	�dictrD�updaterMr�rqr�r�r�)r�r�r�r1�lx�sr�rrrs�	gt_kwargs�kwargss         ��r$r��using.<locals>.callback�s����������'������ ��_�_�)�q�)�������
�A��4�4�U�[�[�]�P�i�P���a��e�Q�k�!�Q���)�)�+����s�B$B'c3�*># �[UR5nURT5 T	"S0UD6nUR5nUR"UR540T202D6HupgnXe-Xx4v� M U(aUR
5Ulgg7fr�)r�rDr�r�rqr�r�r�)r�r�r�r1r�r�r�rrrs�_otherr�r�s         ���r$r�r��s|�����U�]�]�#�A�
�H�H�V����!��B����
�A��4�4�U�[�[�]�P�i�P���a��e�Q�k�!�Q���)�)�+����s�BBrp)�poprR�listr�r)r�r�r�r�r�s``  @r$rr�sr����I��&���J�J�w����a�$���'�'�!"�I�g��"(�!��I�g��
��~�	&�	&�6�O�203	&�204	&��Or&c��\rSrSrSrSrSrg)ri�z�205Indicates a state or state action (e.g. #pop) to apply.206For example default('#pop') is equivalent to ('', Token, '#pop')207Note that state tuples may be used as well.208 209.. versionadded:: 2.0210c��Xlgrp�r�)rGr�s  r$rI�default.__init__�s���211r&r�N)r3r4r5r6r7rIr8r"r&r$rr�s���r&rc�(�\rSrSrSrSSjrSrSrg)ri�z�212Indicates a list of literal words that is transformed into an optimized213regex that matches any of the words.214 215.. versionadded:: 2.0216c�(�XlX lX0lgrp)r�prefix�suffix)rGrr�r�s    r$rI�words.__init__�s���217����r&c�T�[URURURS9$)N�r�r�)rrr�r�rNs r$rE�	words.get�s������D�K�K����L�Lr&)r�r�rN)r�r�)r3r4r5r6r7rIrEr8r"r&r$rr�s����218Mr&rc�F�\rSrSrSrSrSrSrSrSSjr	S	r219S220rSrg)
�RegexLexerMetai�zk221Metaclass for RegexLexer, creates the self._tokens attribute from222self.tokens on the first instantiation.223c��[U[5(aUR5n[R"X5R224$)zBPreprocess the regular expression component of a token definition.)rRrrE�re�compiler�)r��regex�rflagsr�s    r$�_process_regex�RegexLexerMeta._process_regex�s/���e�V�$�$��I�I�K�E��z�z�%�(�.�.�.r&c�`�[U5[Ld[U5(dSU<35eU$)z5Preprocess the token component of a token definition.z0token type must be simple type or callable, not )r,r225�callable)r��tokens  r$�_process_token�RegexLexerMeta._process_tokens4���E�{�j�(�H�U�O�O�	I�>�u�i�H�	I�;��r&c��[U[5(a:US:XagX;aU4$US:XaU$USSS:Xa[USS5*$SU<35e[U[5(agSUR-nU=RS	-
sl/nUH4nXa:wdS226U<35eURUR
UX655 M6 XSU'U4$[U[5(a"UHnXb;aM227US;aMSU-5e U$SU<35e)
z=Preprocess the state transition action of a token definition.�#pop������#pushN�z#pop:zunknown new state z_tmp_%dr�zcircular state ref )r�r�zunknown new state def )rRr]�intr��_tmpname�extend�_process_stater�)r��	new_state�unprocessed�	processed�	tmp_state�itokens�istates       r$�_process_new_state�!RegexLexerMeta._process_new_statesD���i��%�%��F�"���)�!�|�#��g�%� � ��2�A��'�)��I�a�b�M�*�*�*�@� 2�9�-�@�@�u�
�	�8�
,�
,�!�C�L�L�0�I��L�L�A��L��G�#���*�L�.A�&��,L�L�*����s�1�1�+�2;� E�F�$�$+�i� ��<��
�	�5�
)�
)�#���-��"3�3�2�(�6�1�2�4�$���@�2�9�-�@�@�5r&c��[U[5(dSU<35eUSS:wdSU<35eX2;aX#$/=oBU'URnXGHRn[U[5(a<Xc:wdSU<35eUR	URX[U555 MU[U[5(aMl[U[5(aPURURX5nUR[R"S5RSU45 M�[U5[LdSU<35eUR!USXS5nUR'US
5n228[)U5S:XaSnOURUSX5nURX�U45 GMU U$!["a#n	[%S	US<S229U<SU<SU	35U	eSn	A	ff=f)z%Preprocess a single state definition.zwrong state name r�#zinvalid state name zcircular state reference r�Nzwrong rule def zuncompilable regex z230 in state z of z: r��)rRr]�flagsrr�rr�rrr�rSr�r�r�r,r�r��	Exception�231ValueErrorr�ra)r�rrr��tokensr��tdefr�rex�errr�s           r$r�RegexLexerMeta._process_state)s����%��%�%�D�):�5�)�'D�D�%��Q�x�3��?�"5�e�Y� ?�?�����#�#�$&�&��5�!������&�D��$��(�(��}�K�(A�%��&K�K�}��
�
�c�0�0��14�T��<�=���$��)�)���$��(�(��2�2�4�:�:�{�V�	��
�
�r�z�z�"�~�3�3�T�9�E�F����:��&�B�/�$��(B�B�&�
r��(�(��a��&�@���&�&�t�A�w�/�E��4�y�A�~� �	��2�2�4��7�3>�K�	�
�M�M�3�y�1�2�A'�B�
���
r� �#6�t�A�w�k��E�9�TX�Y\�X_�_a�be�af�!g�h�nq�q��
r�s�F-�-232G�7G�GNc��0=o0RU'U=(d URUn[U5HnURX#U5 M U$)z-Preprocess a dictionary of token definitions.)�_all_tokensrr�r)r�r/�	tokendefsrr�s     r$�process_tokendef�RegexLexerMeta.process_tokendefTsJ��,.�.�	�O�O�D�)��1����D�!1�	��)�_�E����y�U�;�%��r&c��0n0nURH�nURRS05nUR5HrupVURU5nUc XaU'UR	[2335nX�U'M9URUS5nUcMPXgX�S-&UR	[2345n	X�-X%'Mt M� U$![a M�f=f![a M�f=f)a�235Merge tokens from superclasses in MRO order, returning a single tokendef236dictionary.237 238Any state that is not defined by a subclass will be inherited239automatically.  States that *are* defined by subclasses will, by240default, override that state in the superclass.  If a subclass wishes to241inherit definitions from a superclass, it can use the special value242"inherit", which will cause the superclass' state definition to be243included at that point in the state.244rNr�)�__mro__�__dict__rE�items�indexrrr�)245r�r�inheritable�c�toksr�r�curitems�inherit_ndx�new_inh_ndxs246          r$�
get_tokendefs�RegexLexerMeta.get_tokendefs\s����������A��:�:�>�>�(�B�/�D� $�247�248����!�:�:�e�,���#�249%*�5�M�!�&+�k�k�'�&:��*5��&��)�o�o�e�T�:���&��7<���]�3�C�#(�+�+�g�"6�K�*5�)B�K�&�9!-��B�
��)&�!� �!��"����s$�B<�C
�<250C251�	C252�
253C�Cc��SUR;aV0UlSUl[US5(aUR(aO%URSUR
55Ul[R"U/UQ70UD6$)z:Instantiate cls after preprocessing its token definitions.�_tokensr�token_variantsr�)254rrr��hasattrr(rr$r'r,�__call__)r�r��kwdss   r$r*�RegexLexerMeta.__call__�sk���C�L�L�(� �C�O��C�L��s�,�-�-�#�2D�2D��!�2�2�2�s�7H�7H�7J�K����}�}�S�0�4�0�4�0�0r&r"rp)
r3r4r5r6r7r�r�rrrr$r*r8r"r&r$r�r��s.���255/��!A�F)�V�/�b1r&r�c�>�\rSrSrSr\Rr0rSSjr	Sr256g)ri�z�257Base for simple stateful regular expression-based lexers.258Simplifies the lexing process so that you need only259provide a list of states and regular expressions.260c#�n# �SnURn[U5nXESnUGH>upxn	U"X5n261U262(dMUb7[U5[LaX8U263R	54v� OU"X2645Shv�N U265R5nU	b�[
U	[5(a`U	HYnUS:Xa#[U5S:�aUR5 M*M,US:XaURUS5 MHURU5 M[ O^[
U	[5(a#[U	5[U5:�aUSS2	O+XYS2	O&U	S:XaURUS5 OSU	<35eXESn O8 XS:XaS	/nUS	nU[S4v� US-
nGMjU[X4v� US-
nGM~GN/![a gf=f7f)266zf267Split ``text`` into (tokentype, text) pairs.268 269``stack`` is the initial stack (default: ``['root']``)270rr�r�Nr�r��wrong state def: r[r�)r'r�r,r271r�r�rRr�rar�rSr��absr	rr�)rGrVr�r�r�272statestack�statetokens�rexmatchr�r�mr�s            r$rq�!RegexLexer.get_tokens_unprocessed�s�������L�L�	��%�[�273��2��/���/:�+��)��T�'���1��)���<�:�5�"%�q�w�w�y�"8�8�'-�d��6�6��%�%�'�C� �,�%�i��7�7�)2��#(�F�?�'*�:���':�(2���(8�(;�%*�g�%5�$.�$5�$5�j��n�$E�$.�$5�$5�e�$<�*3�(�	�3�7�7� #�9�~��Z��@�$.�q�r�N�$.�z�$:�&�'�1�&�-�-�j��n�=�K�,=�i�]�*K�K�5�&/�2��&?���C0;�J��y�D�(�&,�X�274�&/��&7��!�:�t�3�3��q��� ��u�d�i�/�/��1�H�C�_�7��P"����sG�8F5�5F5�3F"�4C5F5�* F%�275F5�
F%�F5�%276F2�/F5�1F2�2F5r"N�)r�)r3r4r5r6r7r��	MULTILINErrrqr8r"r&r$rr�s���
�L�L�E�0�F�;r&rc�(�\rSrSrSrSSjrSrSrg)ri�z1277A helper object that holds lexer position data.278Nc�p�XlX lU=(d [U5UlU=(d S/Ulg)Nr�)rVr�rar�r�)rGrVr�r�r�s     r$rI�LexerContext.__init__s*���	����#�#�d�)����&�v�h��279r&c�\�SUR<SUR<SUR<S3$)Nz
LexerContext(z, �))rVr�r�rNs r$rO�LexerContext.__repr__s)���t�y�y�m�2�d�h�h�\��D�J�J�>��K�Kr&)r�r�r�rV�NN)r3r4r5r6r7rIrOr8r"r&r$rr�s���'�Lr&rc�"�\rSrSrSrSSjrSrg)riz=280A RegexLexer that uses a context object to store its state.281Nc#��# �URnU(d[US5nUSnOUnX4RSnURnUGH�upgnU"XRUR2825n	U	(dM-Ubo[
U5[La4URXyR54v� U	R5UlO)U"X	U5Shv�N U(dX4RSnUGbI[U[5(a�UH�n283U284S:Xa7[UR5S:�aURR5 M>M@U285S:Xa*URRURS5 MpURRU2865 M� O�[U[5(aB[U5[UR5:�aURSS2	OJURUS2	O:US:Xa)URRURS5 OSU<35eX4RSn O� URUR287:�agXRS	:Xa:S/UlUSnUR[ S	4v� U=RS-
slGM[UR["XR4v� U=RS-
slGM�GN![$a gf=f7f)288zg289Split ``text`` into (tokentype, text) pairs.290If ``context`` is given, use this lexer context instead.291rr�r�r�Nr�r�r/r[)r'rr�rVr�r�r,r292r�rRr�rar�rSr�r0rrr�)rGrV�contextrr�r2r3r�rr4r�s           r$rq�)ExtendedRegexLexer.get_tokens_unprocessedsg���293�L�L�	���t�Q�'�C�#�F�+�K��C�#�I�I�b�M�2�K��8�8�D��/:�+��)��T�7�7�C�G�G�4���1��)���<�:�5�"%�'�'�6�7�7�9�"<�<�&'�e�e�g�C�G�'-�d�s�';�;�;�#,�.7�	�	�"�
�.F�� �,�%�i��7�7�)2��#(�F�?�'*�3�9�9�~��'9�(+�	�	�
�
��(:�%*�g�%5�$'�I�I�$4�$4�S�Y�Y�r�]�$C�$'�I�I�$4�$4�U�$;�*3�(�	�3�7�7�"�9�~��S�Y�Y��?�$'�I�I�a�b�M�$'�I�I�i�j�$9�&�'�1��I�I�,�,�S�Y�Y�r�]�;�K�,=�i�]�*K�K�5�&/�	�	�"�
�&>���G0;�J
��w�w�#�'�'�)���G�G�}��,�%+�H��	�&/��&7��!�g�g�t�T�1�1����1��� ��'�'�5�$�w�w�-�7�7��G�G�q�L�G�c�<��R"����sW�A1K.�7AK.�K�
E-K.�;K�K.�A	K�K.�"6K�K.�294K+�(K.�*K+�+K.r"r>)r3r4r5r6r7rqr8r"r&r$rrs
���@r&rc#��# �[U5n[U5up#SnSnUH�upgnUcUnSn	U(a�U[U5-U:�anX�X&-295n296U297(aXGU2984v� U[U2995-
nUHup�n
XLU
4v� U[U
5-
nM X&-300n	[U5up#U(aU[U5-U:�aMnU	[U5:dM�XGX�S4v� U[U5U	-301-
nM� U(aCU=(d SnUHup�nXGU4v� U[U5-
nM [U5up#U(aMBgg![a UShv�N gf=f![a SnM�f=f![a Sngf=f7f)aC302Helper for lexers which must combine the results of several303sublexers.304 305``insertions`` is a list of ``(index, itokens)`` pairs.306Each ``itokens`` iterable should be inserted at position307``index`` into the token stream given by the ``tokens``308argument.309 310The result is a combined token stream.311 312TODO: clean up the code here.313NTrF)�iter�next�
StopIterationra)r�rrr�realpos�insleftr�rrrs�oldi�tmpval�it_index�it_token�it_value�ps               r$r�r�Ss�����j�!�J���j�)����G��G����a��?��G����!�c�!�f�*��-��E�I�&�F���&�(�(��3�v�;�&��07�,��H���1�1��3�x�=�(��18��9�D�
�!%�j�!1����!�c�!�f�*��-��#�a�&�=��a��h�&�&��s�1�v��}�$�G�+�0��,�Q���G�A�!��a�-���s�1�v��G��	�!�*�-�N�E��'��E��������4!�
����
�� �	��G��	�s��E/�
D/�A1E/�
E314�E/�6
E/�AE/�
E�$E/�-E/�/E�>E�?E�E/�E�E/�315E�E/�E�E/�E,�)E/�+E,�,E/c��\rSrSrSrSrSrg)�ProfilingRegexLexerMetai�z>Metaclass for ProfilingRegexLexer, collects regex timing info.c��^^^^�[U[5(a*[URURURS9mOUm[316R"TU5m[R4UUUU4SjjnU$)Nr�c��>�TRSRT317T	4SS/5n[R"5nTRXU5n[R"5nUS==S-
ss'US==Xd-318-
ss'U$)Nr�rr!r�)�319_prof_data�320setdefault�timer�)rVr��endpos�info�t0�res�t1r��compiledrr�s       ����r$�321match_func�:ProfilingRegexLexerMeta._process_regex.<locals>.match_func�sn����>�>�"�%�0�0�%����3�x�H�D�����B��.�.��F�3�C�����B���G�q�L�G���G�r�w��G��Jr&)	rRrrr�r�r�r��sys�maxsize)r�r�r�r�r\r[rs`  ` @@r$r��&ProfilingRegexLexerMeta._process_regex�s]����e�U�#�#��E�K�K����#(�<�<�1�C��C��:�:�c�6�*��),���	�	��r&r"N)r3r4r5r6r7r�r8r"r&r$rPrP�s322��H�r&rPc�*�\rSrSrSr/rSrSSjrSrg)�ProfilingRegexLexeri�zFDrop-in replacement for RegexLexer that does profiling of its regexes.�c#�H^# �TRRR05 [R	TX5Shv�N TRRR5n[
SUR55U4SjSS9n[SU55n[5 [STRR[U5U4-5 [S5 [SS	-5 [S3235 UHn[SU-5 M [S5 gN�7f)Nc3�# �UHGuupup4U[U5RS5RSS5SSUSU-SU-U-4v� MI g7f)zu'z\\�\N�Ai�)�reprrdrZ)�.0r��r�nrrs     r$�	<genexpr>�=ProfilingRegexLexer.get_tokens_unprocessed.<locals>.<genexpr>�s\���@�/>�+�F�Q�F�Q��4��7�=�=��/�7�7���E�c�r�J��4�!�8�T�A�X��\�3�/>�s�AAc�">�UTR$rp)�_prof_sort_index)r#rGs �r$r%�<ProfilingRegexLexer.get_tokens_unprocessed.<locals>.<lambda>�s���A�d�&;�&;�$<r&T)�key�reversec3�*# �UH	oSv� M g7f)�Nr")rir#s  r$rlrm�s���+�d��!��d�s�z2Profiling result for %s lexing %d chars in %.3f mszn==============================================================================================================z$%-20s %-64s ncalls  tottime  percall)r�r�zn--------------------------------------------------------------------------------------------------------------z%-20s %-65s %5d %8.4f %8.4f)rMrSrSrrqr��sortedr�sum�printr3ra)rGrVr��rawdatar��	sum_totalr1s`      r$rq�*ProfilingRegexLexer.get_tokens_unprocessed�s��������!�!�(�(��,��4�4�T�4�G�G�G��.�.�+�+�/�/�1���@�/6�}�}��@�=�"�	$��324�+�d�+�+�	�
��
�B��~�~�&�&��D�	�9�=�>�	?�
�i��
�4�7I�I�J�
�i���A��/�!�3�4��
�i��#	H�s�?D"�D �CD"r"Nr6)	r3r4r5r6r7rSrorqr8r"r&r$rbrb�s��P��J���r&rb)6r7r�r^rU�pygments.filterrr�pygments.filtersr�pygments.tokenrrrr	r325�
pygments.utilrrr
rrr�pygments.regexoptr�__all__r�rr_�staticmethod�_default_analyser,r(rrr]rr�rr�r�r�rr�rrrrr�rrrr�rPrbr"r&r$�<module>r�sJ���326�327��1�/�E�E�*�*�'�*��
�*�*�W�328��,�
� �
�.��	1��	1�m"�i�m"�`O�e�O�N	�c�	����*��329
�u�330
���6�4��
�w��1�h	�	�
M�F�
M� d1�Y�d1�N^��.�^�BL�L�E��E�P=�@�n��,�*�0G�r&
codekingpro/portable-devtools · Team Ai