next_tag( 'option' ) ) {
* $processor->set_attribute( '', 'selected', 'yes' );
* }
*
* ### Finding tags
*
* The `next_tag()` function moves the internal cursor through
* your input XML document until it finds a tag meeting any of
* the supplied restrictions in the optional query argument. If
* no argument is provided then it will find the next XML tag,
* regardless of what kind it is.
*
* If you want to _find whatever the next tag is_:
*
* $processor->next_tag();
*
* | Goal | Query |
* |----------------------------------------------------------|----------------------------------------------------------|
* | Find any tag. | `$processor->next_tag();` |
* | Find next image tag. | `$processor->next_tag( array( 'tag_name' => 'image' ) );`|
* | Find next image tag (shorthand). | `$processor->next_tag( 'image' );` |
* | Find next image tag in the "wp.org" namespace. | `$processor->next_tag( array( 'wp.org', 'image' ) );` |
*
* #### Namespace Examples
*
* To work with namespaces, you can use the `breadcrumbs` query format, where each breadcrumb is a tuple of (namespace prefix, local name):
*
* $xml = '';
* $processor = XMLProcessor::create_from_string( $xml );
* // Find the tag
* if ( $processor->next_tag( array( 'http://wordpress.org/export/1.2/', 'image' ) ) ) {
* // Get the namespace URI of the matched tag
* $ns = $processor->get_tag_namespace(); // 'http://wordpress.org/export/1.2/'
* // Get the value of the 'src' attribute
* $src = $processor->get_attribute( $ns, 'src' );
* // Set a new attribute in the same namespace
* $processor->set_attribute( $ns, 'alt', 'A cat' );
* }
*
* If a tag was found meeting your criteria then `next_tag()`
* will return `true` and you can proceed to modify it. If it
* returns `false`, it failed to find the tag and moved the cursor to the end of the file.
*
* Once the cursor reaches the end of the file the processor
* is done and if you want to reach an earlier tag you will
* need to recreate the processor and start over, as it's
* unable to back up or move in reverse (except via bookmarks).
*
* #### Custom queries
*
* Sometimes it's necessary to further inspect an XML tag than
* the query syntax here permits. In these cases one may further
* inspect the search results using the read-only functions
* provided by the processor or external state or variables.
*
* Example:
*
* // Paint up to the first five `musician` or `actor` tags marked with the "jazzy" style.
* $remaining_count = 5;
* while ( $remaining_count > 0 && $processor->next_tag() ) {
* $tag = $processor->get_tag_local_name();
* if (
* ( 'musician' === $tag || 'actor' === $tag ) &&
* 'jazzy' === $processor->get_attribute( '', 'data-style' )
* ) {
* $processor->set_attribute( '', 'theme-style', 'theme-style-everest-jazz' );
* $remaining_count--;
* }
* }
*
* `get_attribute()` will return `null` if the attribute wasn't present
* on the tag when it was called. It may return `""` (the empty string)
* in cases where the attribute was present but its value was empty.
* For boolean attributes, those whose name is present but no value is
* given, it will return `true` (the only way to set `false` for an
* attribute is to remove it).
*
* #### When matching fails
*
* When `next_tag()` returns `false` it could mean different things:
*
* - The requested tag wasn't found in the input document.
* - The input document ended in the middle of an XML syntax element.
*
* When a document ends in the middle of a syntax element it will pause
* the processor. This is to make it possible in the future to extend the
* input document and proceed - an important requirement for chunked
* streaming parsing of a document.
*
* Example:
*
* $processor = XMLProcessor::create_from_string( 'This ' );
*
* $ns = 'http://wordpress.org/export/1.2/';
* if ( $processor->next_tag( 'image' ) ) {
* $processor->set_attribute( $ns, 'src', 'cat.jpg' );
* }
*
* echo $processor->get_modifiable_text();
* //
*
* ### Bookmarks
*
* While scanning through the input XML document it's possible to set
* a named bookmark when a particular tag is found. Later on, after
* continuing to scan other tags, it's possible to `seek` to one of
* the set bookmarks and then proceed again from that point forward.
*
* Because bookmarks create processing overhead one should avoid
* creating too many of them. As a rule, create only bookmarks
* of known string literal names; avoid creating "mark_{$index}"
* and so on. It's fine from a performance standpoint to create a
* bookmark and update it frequently, such as within a loop.
*
* $total_todos = 0;
* while ( $processor->next_tag( array( 'tag_name' => 'todo-list' ) ) ) {
* $processor->set_bookmark( 'list-start' );
* while ( $processor->next_tag() ) {
* if ( 'todo' === $processor->get_tag_local_name() && $processor->is_tag_closer() ) {
* $processor->set_bookmark( 'list-end' );
* $processor->seek( 'list-start' );
* $processor->set_attribute( '', 'data-contained-todos', (string) $total_todos );
* $total_todos = 0;
* $processor->seek( 'list-end' );
* break;
* }
* if ( 'todo-item' === $processor->get_tag_local_name() && ! $processor->is_tag_closer() ) {
* $total_todos++;
* }
* }
* }
*
* ## Tokens and finer-grained processing
*
* It's possible to scan through every lexical token in the
* XML document using the `next_token()` function. This
* alternative form takes no argument and provides no built-in
* query syntax.
*
* Example:
*
* $title = '(untitled)';
* $text = '';
* while ( $processor->next_token() ) {
* switch ( $processor->get_token_name() ) {
* case '#text':
* $text .= $processor->get_modifiable_text();
* break;
* case 'new-line':
* $text .= "\n";
* break;
* case 'title':
* $title = $processor->get_modifiable_text();
* break;
* }
* }
* return trim( "# {$title}\n\n{$text}" );
*
* ### Tokens and _modifiable text_
*
* There are also non-elements which are void/self-closing in nature and contain
* modifiable text that is part of that individual syntax token itself.
*
* - `#text` nodes, whose entire token _is_ the modifiable text.
* - XML comments and tokens that become comments due to some syntax error. The
* text for these tokens is the portion of the comment inside of the syntax.
* E.g. for `` the text is `" comment "` (note the spaces are included).
* - `CDATA` sections, whose text is the content inside of the section itself. E.g. for
* `` the text is `"some content"`.
* - XML Processing instruction nodes like `` (with restrictions [1]).
*
* [1]: XML requires "xml" as a processing instruction name. The Tag Processor captures the entire
* processing instruction as a single token up to the closing `?>`.
*
* ## Design and limitations
*
* The XMLProcessor is designed to linearly scan XML documents and tokenize
* XML tags and their attributes. It's designed to do this as efficiently as
* possible without compromising parsing integrity. Therefore it will be
* slower than some methods of modifying XML, such as those incorporating
* over-simplified PCRE patterns, but will not introduce the defects and
* failures that those methods bring in, which lead to broken page renders
* and often to security vulnerabilities. On the other hand, it will be faster
* than full-blown XML parsers such as DOMDocument and use considerably
* less memory. It requires a negligible memory overhead, enough to consider
* it a zero-overhead system.
*
* The performance characteristics are maintained by avoiding tree construction.
*
* The XMLProcessor checks the most important aspects of XML integrity as it scans
* through the document. It verifies that a single root element exists, that there are
* no unclosed tags, and that each opener tag has a corresponding closer. It also
* ensures no duplicate attributes exist on a single tag.
*
* At the same time, the XMLProcessor also skips expensive validation of XML entities
* in the document. The processor will initially pass through invalid entity references
* and only fail when the developer attempts to read their value. If that doesn't happen,
* the invalid values will be left untouched in the final document.
*
* Most operations within the XMLProcessor are designed to minimize the difference
* between an input and output document for any given change. For example, the
* `set_attribute` and `remove_attribute` methods preserve whitespace and the attribute
* ordering within the element definition. An exception to this rule is that all attribute
* updates store their values as double-quoted strings, meaning that attributes on input with
* single-quoted or unquoted values will appear in the output with double-quotes.
*
* ### Text Encoding
*
* The XMLProcessor assumes that the input XML document is encoded with a
* UTF-8 encoding and will refuse to process documents that declare other encodings.
*
* ### Namespaces
*
* Namespaces are first-class citizens in the XMLProcessor. Methods such as `set_attribute()` and `remove_attribute()`
* require the full namespace URI, not just the local name. The XML specification treats the local
* name as a mere syntax sugar. The actual matching is always done on the fully qualified namespace name.
*
* Example:
*
* $processor = XMLProcessor::from_string( '' );
* $processor->next_tag( 'image' );
* $local_name = $processor->get_tag_local_name(); // 'image'
* $ns = $processor->get_tag_namespace(); // 'http://wordpress.org/export/1.2/'
* echo $processor->get_tag_namespace_and_local_name(); // '{http://wordpress.org/export/1.2/}image'
*
* #### Internal representation of names
*
* Internally, the XMLProcessor stores names using the following format:
*
* {namespace}local_name
*
* It's safe, because the "{" and "}" bytes cannot be used in tag names or attribute names:
*
* [4] NameStartChar ::= ":" | [A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
* [4a] NameChar ::= NameStartChar | "-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]
*
* @since WP_VERSION
*/
class XMLProcessor {
/**
* The maximum number of bookmarks allowed to exist at
* any given time.
*
* @since WP_VERSION
* @var int
*
* @see XMLProcessor::set_bookmark()
*/
const MAX_BOOKMARKS = 10;
/**
* Maximum number of times seek() can be called.
* Prevents accidental infinite loops.
*
* @since WP_VERSION
* @var int
*
* @see XMLProcessor::seek()
*/
const MAX_SEEK_OPS = 1000;
/**
* The XML document to parse.
*
* @since WP_VERSION
* @var string
*/
public $xml;
/**
* Specifies mode of operation of the parser at any given time.
*
* | State | Meaning |
* | ------------------|----------------------------------------------------------------------|
* | *Ready* | The parser is ready to run. |
* | *Complete* | There is nothing left to parse. |
* | *Incomplete* | The XML ended in the middle of a token; nothing more can be parsed. |
* | *Matched tag* | Found an XML tag; it's possible to modify its attributes. |
* | *Text node* | Found a #text node; this is plaintext and modifiable. |
* | *CDATA node* | Found a CDATA section; this is modifiable. |
* | *PI node* | Found a processing instruction; this is modifiable. |
* | *XML declaration* | Found an XML declaration; this is modifiable. |
* | *Comment* | Found a comment or bogus comment; this is modifiable. |
*
* @since WP_VERSION
*
* @see XMLProcessor::STATE_READY
* @see XMLProcessor::STATE_COMPLETE
* @see XMLProcessor::STATE_INCOMPLETE_INPUT
* @see XMLProcessor::STATE_MATCHED_TAG
* @see XMLProcessor::STATE_TEXT_NODE
* @see XMLProcessor::STATE_CDATA_NODE
* @see XMLProcessor::STATE_PI_NODE
* @see XMLProcessor::STATE_XML_DECLARATION
* @see XMLProcessor::STATE_COMMENT
*
* @var string
*/
protected $parser_state = self::STATE_READY;
/**
* Whether the input has been finished.
*
* @var bool
*/
protected $expecting_more_input = true;
/**
* How many bytes from the current XML chunk have been read and parsed.
*
* This value points to the latest byte offset in the input document which
* has been already parsed. It is the internal cursor for the Tag Processor
* and updates while scanning through the XML tokens.
*
* @since WP_VERSION
* @var int
*/
public $bytes_already_parsed = 0;
/**
* How many XML bytes from the original stream have already been removed
* from the memory.
*
* @since WP_VERSION
* @var int
*/
public $upstream_bytes_forgotten = 0;
/**
* Byte offset in the current `$xml` string where the currently recognized token starts.
* `null` if no token is currently active.
*
* Example:
*
* ...
* ^-- token_starts_at = 0
*
* @since WP_VERSION
*
* @var int|null
*/
protected $token_starts_at;
/**
* Byte length of current token.
*
* Example:
*
* ...
* 012345678901234
* - token length is 14 - 0 = 14
*
* a is a token.
* 0123456789 123456789 123456789
* - token length is 17 - 2 = 15
*
* @since WP_VERSION
*
* @var int|null
*/
private $token_length;
/**
* Currently matched XML element object.
*
* @var XMLElement|null
*/
private $element;
/**
* Byte offset in input document where current tag name starts.
*
* Example:
*
* ...
* 01234
* - tag name starts at 1
*
* @since WP_VERSION
*
* @var int|null
*/
private $tag_name_starts_at;
/**
* Byte length of current tag name.
*
* Example:
*
* ...
* 01234
* --- tag name length is 3
*
* @since WP_VERSION
*
* @var int|null
*/
private $tag_name_length;
/**
* Byte offset into input document where current modifiable text starts.
*
* @since WP_VERSION
*
* @var int
*/
private $text_starts_at;
/**
* Byte length of modifiable text.
*
* @since WP_VERSION
*
* @var string
*/
private $text_length;
/**
* Whether the current tag is an opening tag, e.g. , or a closing tag, e.g. .
*
* @var bool
*/
private $is_closing_tag;
/**
* Stores the error for why something failed, if it did.
*
* @see self::get_last_error
*
* @since WP_VERSION
*
* @var string|null
*/
protected $last_error = null;
/**
* Stores context for why the parser bailed on unsupported XML, if it did.
*
* @see self::get_exception
*
* @var XMLUnsupportedException|null
*/
private $exception = null;
/**
* Temporary index of attributes found within an XML tag, keyed by the qualified
* attribute name. It is only used during the initial attributes parsing phase and
* discarded once all the attributes have been parsed.
*
* @since WP_VERSION
* @var XMLAttributeToken[]
*/
private $qualified_attributes = array();
/**
* Stores the attributes found within an XML tag, keyed by their namespace
* and local name combination.
*
* Example:
*
* // Supposing the parser just finished parsing the wp:content tag:
* //
* //
* //
* //
* // Then, the attributes array would be:
* $this->attributes = array(
* '{http://wordpress.org/export/1.2/}id' => new XMLAttributeToken( 9, 6, 5, 14, 'wp', 'id' ),
* 'class' => new XMLAttributeToken( 23, 7, 17, 13, '', 'class', '' )
* );
*
* @since WP_VERSION
* @var XMLAttributeToken[]
*/
private $attributes = array();
/**
* Tracks a semantic location in the original XML which
* shifts with updates as they are applied to the document.
*
* @since WP_VERSION
* @var WP_HTML_Span[]
*/
protected $bookmarks = array();
/**
* Lexical replacements to apply to input XML document.
*
* "Lexical" in this class refers to the part of this class which
* operates on pure text _as text_ and not as XML. There's a line
* between the public interface, with XML-semantic methods like
* `set_attribute` and `add_class`, and an internal state that tracks
* text offsets in the input document.
*
* When higher-level XML methods are called, those have to transform their
* operations (such as setting an attribute's value) into text diffing
* operations (such as replacing the sub-string from indices A to B with
* some given new string). These text-diffing operations are the lexical
* updates.
*
* As new higher-level methods are added they need to collapse their
* operations into these lower-level lexical updates since that's the
* Tag Processor's internal language of change. Any code which creates
* these lexical updates must ensure that they do not cross XML syntax
* boundaries, however, so these should never be exposed outside of this
* class or any classes which intentionally expand its functionality.
*
* These are enqueued while editing the document instead of being immediately
* applied to avoid processing overhead, string allocations, and string
* copies when applying many updates to a single document.
*
* Example:
*
* // Replace an attribute stored with a new value, indices
* // sourced from the lazily-parsed XML recognizer.
* $start = $attributes['src']->start;
* $length = $attributes['src']->length;
* $modifications[] = new WP_HTML_Text_Replacement( $start, $length, $new_value );
*
* // Correspondingly, something like this will appear in this array.
* $lexical_updates = array(
* WP_HTML_Text_Replacement( 14, 28, 'https://my-site.my-domain/wp-content/uploads/2014/08/kittens.jpg' )
* );
*
* @since WP_VERSION
* @var WP_HTML_Text_Replacement[]
*/
protected $lexical_updates = array();
/**
* The Name from the DOCTYPE declaration.
*
* ```
* doctypedecl ::= ''
* ^^^^
* ```
*
* @since WP_VERSION
* @var WP_HTML_Span|null
*/
protected $doctype_name = null;
/**
* The system literal value from the DOCTYPE declaration.
*
* ```
* doctypedecl ::= ''
* ExternalID ::= 'SYSTEM' S SystemLiteral | 'PUBLIC' S PubidLiteral
* ^^^^^^^^^^^^^
* ```
*
* Example:
*
*
*
* In this example, the system_literal would be:
* "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"
*
* @since WP_VERSION
* @var WP_HTML_Span|null
*/
protected $system_literal = null;
/**
* The public identifier value from the DOCTYPE declaration.
*
* ```
* doctypedecl ::= ''
* ExternalID ::= 'SYSTEM' S SystemLiteral | 'PUBLIC' S PubidLiteral
* ```
* ^^^^^^^^^^^^
* Example:
*
* ```
*
* ```
*
* In this example, the publid_literal would be:
* "-//W3C//DTD XHTML 1.0 Strict//EN"
*
* @since WP_VERSION
* @var WP_HTML_Span|null
*/
protected $pubid_literal = null;
/**
* Memory budget for the processed XML.
*
* `append_bytes()` will flush the processed bytes whenever the XML buffer
* exceeds this budget. The lexical updates will be applied and the bookmarks
* will be reset.
*
* @var int
*/
protected $memory_budget = 1024 * 1024 * 1024;
/**
* Tracks and limits `seek()` calls to prevent accidental infinite loops.
*
* @since WP_VERSION
* @var int
*
* @see XMLProcessor::seek()
*/
protected $seek_count = 0;
/**
* Indicates the current parsing stage.
*
* A well-formed XML document has the following structure:
*
* document ::= prolog element Misc*
* prolog ::= XMLDecl? Misc* (doctypedecl Misc*)?
* Misc ::= Comment | PI | S
*
* There is exactly one element, called the root. No elements or text nodes may
* precede or follow it.
*
* See https://www.w3.org/TR/xml/#NT-document.
*
* | Stage | Meaning |
* | ----------------|---------------------------------------------------------------------|
* | *Prolog* | The parser is parsing the prolog. |
* | *Element* | The parser is parsing the root element. |
* | *Misc* | The parser is parsing miscellaneous content. |
*
* @see XMLProcessor::IN_PROLOG_CONTEXT
* @see XMLProcessor::IN_ELEMENT_CONTEXT
* @see XMLProcessor::IN_MISC_CONTEXT
*
* @since WP_VERSION
* @var bool
*/
protected $parser_context = self::IN_PROLOG_CONTEXT;
/**
* Top-level namespaces for the currently parsed document.
*
* @var array
*/
private $document_namespaces;
/**
* Tracks open elements and their namespaces while scanning XML.
*
* @var array
*/
private $stack_of_open_elements = array();
public static function create_from_string( $xml, $cursor = null, $known_definite_encoding = 'UTF-8', $document_namespaces = array() ) {
$processor = static::create_for_streaming( $xml, $cursor, $known_definite_encoding, $document_namespaces );
if ( null === $processor ) {
return false;
}
$processor->input_finished();
return $processor;
}
public static function create_for_streaming( $xml = '', $cursor = null, $known_definite_encoding = 'UTF-8', $document_namespaces = array() ) {
if ( 'UTF-8' !== $known_definite_encoding ) {
return false;
}
$processor = new XMLProcessor( $xml, $document_namespaces, self::CONSTRUCTOR_UNLOCK_CODE );
if ( null !== $cursor && true !== $processor->initialize_from_cursor( $cursor ) ) {
return false;
}
return $processor;
}
/**
* Returns a re-entrancy cursor – it's a string that can instruct a new XML
* Processor instance to continue parsing from the current location in the
* document.
*
* The only stable part of this API is the return type of string. The consumer
* of this method MUST NOT assume any specific structure of the returned
* string. It will change without a warning between WordPress releases.
*
* This is not a tell() API. No XML Processor method will accept the cursor
* to move to another location. The only way to use this cursor is creating
* a new XML Processor instance. If you need to move around the document, use
* `set_bookmark()` and `seek()`.
*/
public function get_reentrancy_cursor() {
$stack_of_open_elements = array();
foreach ( $this->stack_of_open_elements as $element ) {
$stack_of_open_elements[] = $element->to_array();
}
return base64_encode( // phpcs:ignore WordPress.PHP.DiscouragedPHPFunctions.obfuscation_base64_encode
json_encode(
array(
'is_finished' => $this->is_finished(),
'upstream_bytes_forgotten' => $this->upstream_bytes_forgotten,
'parser_context' => $this->parser_context,
'stack_of_open_elements' => $stack_of_open_elements,
'expecting_more_input' => $this->expecting_more_input,
'document_namespaces' => $this->document_namespaces,
)
)
);
}
/**
* Returns the byte offset in the input stream where the current token starts.
*
* You should probably not use this method.
*
* It's only exists to allow resuming the input stream at the same offset where
* the XML parsing was finished. It will never expose any attribute's byte
* offset and no method in the XML processor API will ever accept the byte offset
* to move to another location. If you need to move around the document, use
* `set_bookmark()` and `seek()` instead.
*/
public function get_token_byte_offset_in_the_input_stream() {
return $this->token_starts_at + $this->upstream_bytes_forgotten;
}
protected function initialize_from_cursor( $cursor ) {
if ( ! is_string( $cursor ) ) {
_doing_it_wrong( __METHOD__, 'Cursor must be a JSON-encoded string.', '1.0.0' );
return false;
}
$cursor = base64_decode( $cursor ); // phpcs:ignore WordPress.PHP.DiscouragedPHPFunctions.obfuscation_base64_decode
if ( false === $cursor ) {
_doing_it_wrong( __METHOD__, 'Invalid cursor provided to initialize_from_cursor().', '1.0.0' );
return false;
}
$cursor = json_decode( $cursor, true );
if ( false === $cursor ) {
_doing_it_wrong( __METHOD__, 'Invalid cursor provided to initialize_from_cursor().', '1.0.0' );
return false;
}
if ( $cursor['is_finished'] ) {
$this->parser_state = self::STATE_COMPLETE;
}
// Assume the input stream will start from the last known byte offset.
$this->bytes_already_parsed = 0;
$this->upstream_bytes_forgotten = $cursor['upstream_bytes_forgotten'];
$this->stack_of_open_elements = array();
foreach ( $cursor['stack_of_open_elements'] as $element ) {
array_push( $this->stack_of_open_elements, XMLElement::from_array( $element ) );
}
$this->document_namespaces = $cursor['document_namespaces'];
$this->parser_context = $cursor['parser_context'];
$this->expecting_more_input = $cursor['expecting_more_input'];
return true;
}
/**
* Constructor.
*
* Do not use this method. Use the static creator methods instead.
*
* @access private
*
* @param string $xml XML to process.
* @param array $document_namespaces Document namespaces.
* @param string|null $use_the_static_create_methods_instead This constructor should not be called manually.
*
* @see XMLProcessor::create_stream()
*
* @since 6.4.0
*
* @see XMLProcessor::create_fragment()
*/
protected function __construct( $xml, $document_namespaces = array(), $use_the_static_create_methods_instead = null ) {
if ( self::CONSTRUCTOR_UNLOCK_CODE !== $use_the_static_create_methods_instead ) {
_doing_it_wrong(
__METHOD__,
sprintf(
/* translators: %s: XMLProcessor::create_fragment(). */
__( 'Call %s to create an XML Processor instead of calling the constructor directly.' ),
'XMLProcessor::create_fragment()'
),
'6.4.0'
);
}
$this->xml = isset( $xml ) ? $xml : '';
$this->document_namespaces = array_merge(
$document_namespaces,
// These initial namespaces cannot be overridden.
array(
'xml' => 'http://www.w3.org/XML/1998/namespace', // Predefined, cannot be unbound or changed.
'xmlns' => 'http://www.w3.org/2000/xmlns/', // Reserved for xmlns attributes, not a real namespace for elements/attributes.
'' => '', // Default namespace is initially empty (no namespace).
)
);
}
/**
* Wipes out the processed XML and appends the next chunk of XML to
* any remaining unprocessed XML.
*
* @param string $next_chunk XML to append.
*/
public function append_bytes( $next_chunk ) {
if ( ! $this->expecting_more_input ) {
_doing_it_wrong(
__METHOD__,
__( 'Cannot append bytes after the last input chunk was provided and input_finished() was called.' ),
'WP_VERSION'
);
return false;
}
$this->xml .= $next_chunk;
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
$this->parser_state = self::STATE_READY;
}
// Periodically flush the processed bytes to avoid high memory usage.
if (
null !== $this->memory_budget &&
strlen( $this->xml ) > $this->memory_budget
) {
$this->flush_processed_xml();
}
return true;
}
/**
* Forgets the XML bytes that have been processed and are no longer needed to
* avoid high memory usage.
*
* @return string The flushed bytes.
*/
private function flush_processed_xml() {
// Flush updates.
$this->get_updated_xml();
$unreferenced_bytes = $this->bytes_already_parsed;
if ( null !== $this->token_starts_at ) {
$unreferenced_bytes = min( $unreferenced_bytes, $this->token_starts_at );
}
$flushed_bytes = substr( $this->xml, 0, $unreferenced_bytes );
$this->xml = substr( $this->xml, $unreferenced_bytes );
$this->bookmarks = array();
$this->lexical_updates = array();
$this->seek_count = 0;
$this->bytes_already_parsed -= $unreferenced_bytes;
if ( null !== $this->token_starts_at ) {
$this->token_starts_at -= $unreferenced_bytes;
}
if ( null !== $this->tag_name_starts_at ) {
$this->tag_name_starts_at -= $unreferenced_bytes;
}
if ( null !== $this->text_starts_at ) {
$this->text_starts_at -= $unreferenced_bytes;
}
$this->upstream_bytes_forgotten += $unreferenced_bytes;
return $flushed_bytes;
}
/**
* Indicates that all the XML document bytes have been provided.
*
* After calling this method, the processor will emit errors where
* previously it would have entered the STATE_INCOMPLETE_INPUT state.
*/
public function input_finished() {
$this->expecting_more_input = false;
$this->parser_state = self::STATE_READY;
}
/**
* Indicates if the processor is expecting more data bytes.
* If not, the processor will expect the remaining XML bytes to form
* a valid document and will not stop on incomplete input.
*
* @return bool Whether the processor is expecting more data bytes.
*/
public function is_expecting_more_input() {
return $this->expecting_more_input;
}
/**
* Internal method which finds the next token in the XML document.
*
* This method is a protected internal function which implements the logic for
* finding the next token in a document. It exists so that the parser can update
* its state without affecting the location of the cursor in the document and
* without triggering subclass methods for things like `next_token()`, e.g. when
* applying patches before searching for the next token.
*
* @return bool Whether a token was parsed.
* @since 6.5.0
*
* @access private
*/
protected function parse_next_token() {
$was_at = $this->bytes_already_parsed;
$this->after_tag();
// Don't proceed if there's nothing more to scan.
if (
self::STATE_COMPLETE === $this->parser_state ||
self::STATE_INCOMPLETE_INPUT === $this->parser_state ||
null !== $this->last_error
) {
return false;
}
/*
* The next step in the parsing loop determines the parsing state;
* clear it so that state doesn't linger from the previous step.
*/
$this->parser_state = self::STATE_READY;
if ( $this->bytes_already_parsed >= strlen( $this->xml ) ) {
if ( $this->expecting_more_input ) {
$this->parser_state = self::STATE_INCOMPLETE_INPUT;
} else {
$this->parser_state = self::STATE_COMPLETE;
}
return false;
}
// Find the next tag if it exists.
if ( false === $this->parse_next_tag() ) {
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
$this->bytes_already_parsed = $was_at;
}
return false;
}
if ( null !== $this->last_error ) {
return false;
}
/*
* For legacy reasons the rest of this function handles tags and their
* attributes. If the processor has reached the end of the document
* or if it matched any other token then it should return here to avoid
* attempting to process tag-specific syntax.
*/
if (
self::STATE_INCOMPLETE_INPUT !== $this->parser_state &&
self::STATE_COMPLETE !== $this->parser_state &&
self::STATE_MATCHED_TAG !== $this->parser_state
) {
return true;
}
if ( $this->is_closing_tag ) {
$this->skip_whitespace();
} else {
// Parse all of its attributes.
while ( $this->parse_next_attribute() ) {
continue;
}
}
if ( null !== $this->last_error ) {
return false;
}
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
$this->bytes_already_parsed = $was_at;
return false;
}
// Ensure that the tag closes before the end of the document.
if ( $this->bytes_already_parsed >= strlen( $this->xml ) ) {
// Does this appropriately clear state (parsed attributes)?
$this->mark_incomplete_input( 'Tag attributes were not closed before the end of the document.' );
$this->bytes_already_parsed = $was_at;
return false;
}
$tag_ends_at = strpos( $this->xml, '>', $this->bytes_already_parsed );
if ( false === $tag_ends_at ) {
$this->mark_incomplete_input( 'No > found at the end of a tag.' );
$this->bytes_already_parsed = $was_at;
return false;
}
if ( $this->is_closing_tag && $tag_ends_at !== $this->bytes_already_parsed ) {
$this->bail(
'Invalid closing tag encountered.',
self::ERROR_SYNTAX
);
return false;
}
$this->parser_state = self::STATE_MATCHED_TAG;
$this->bytes_already_parsed = $tag_ends_at + 1;
$this->token_length = $this->bytes_already_parsed - $this->token_starts_at;
/**
* Resolve the namespaces defined in opening tags.
*/
if ( ! $this->is_closing_tag ) {
/**
* By default, inherit all namespaces from the parent element.
*/
$namespaces = $this->get_tag_namespaces_in_scope();
foreach ( $this->qualified_attributes as $attribute ) {
/**
* `xmlns` attribute is the default namespace
* `xmlns:` declares a namespace prefix scoped to the current element and its descendants
*
* @see https://www.w3.org/TR/2006/REC-xml-names11-20060816/#ns-decl
*/
if ( 'xmlns' === $attribute->qualified_name ) {
$value = $this->get_qualified_attribute( $attribute->qualified_name );
// Update the default namespace.
$namespaces[''] = $value;
continue;
}
if ( 'xmlns' === $attribute->namespace_prefix ) {
$value = $this->get_qualified_attribute( $attribute->qualified_name );
/**
* @see https://www.w3.org/TR/2006/REC-xml-names11-20060816/#xmlReserved
*/
if ( 'xml' === $attribute->local_name && 'http://www.w3.org/XML/1998/namespace' !== $value ) {
$this->bail(
'The `xml` namespace prefix is by definition bound to the namespace name http://www.w3.org/XML/1998/namespace and must not be overridden.',
self::ERROR_SYNTAX
);
return false;
}
/**
* @see https://www.w3.org/TR/2006/REC-xml-names11-20060816/#xmlReserved
*/
if ( 'xmlns' === $attribute->local_name ) {
$this->bail( 'The `xmlns` namespace prefix must not be overridden.', self::ERROR_SYNTAX );
return false;
}
/**
* The attribute value in a namespace declaration for a prefix MAY be empty.
* This has the effect, within the scope of the declaration, of removing any
* association of the prefix with a namespace name. Further declarations MAY
* re-declare the prefix again.
*/
if ( '' === $value ) {
unset( $namespaces[ $attribute->namespace_prefix ] );
continue;
}
$namespaces[ $attribute->local_name ] = $value;
continue;
}
}
/**
* Confirm the tag name is valid with respect to XML namespaces.
*
* @see https://www.w3.org/TR/2006/REC-xml-names11-20060816/#Conformance
*/
$tag_name = $this->get_tag_name_qualified();
if ( false === $this->validate_qualified_name( $tag_name ) ) {
return false;
}
list( $tag_namespace_prefix, $tag_local_name ) = $this->parse_qualified_name( $tag_name );
/**
* Validate the element namespace.
*/
if ( ! array_key_exists( $tag_namespace_prefix, $namespaces ) ) {
$this->bail(
sprintf(
'Namespace prefix "%s" does not resolve to any namespace in the current element\'s scope.',
$tag_namespace_prefix
),
self::ERROR_SYNTAX
);
}
/**
* Compute fully qualified attributes and assert:
*
* – All attributes have valid namespaces.
* – No two attributes have the same (local name, namespace) pair.
*
* @see https://www.w3.org/TR/2006/REC-xml-names11-20060816/#uniqAttrs
*/
$namespaced_attributes = array();
foreach ( $this->qualified_attributes as $attribute ) {
list( $attribute_namespace_prefix, $attribute_local_name ) = $this->parse_qualified_name( $attribute->qualified_name );
if ( ! array_key_exists( $attribute_namespace_prefix, $namespaces ) ) {
$this->bail(
sprintf(
'Attribute "%s" has an invalid namespace prefix "%s".',
$attribute->qualified_name,
$attribute_namespace_prefix
),
self::ERROR_SYNTAX
);
return false;
}
$namespace_reference = $attribute_namespace_prefix ? $namespaces[ $attribute_namespace_prefix ] : '';
/**
* It looks supicious but it's safe – $local_name is guaranteed to not contain
* curly braces at this point.
*/
$attribute_full_name = $namespace_reference ? '{' . $namespace_reference . '}' . $attribute_local_name : $attribute_local_name;
if ( isset( $namespaced_attributes[ $attribute_full_name ] ) ) {
$this->bail(
sprintf(
'Duplicate attribute "%s" with namespace "%s" found in the same element.',
$attribute_local_name,
$namespace_reference
),
self::ERROR_SYNTAX
);
return false;
}
$namespaced_attributes[ $attribute_full_name ] = $attribute;
$attribute->namespace = $namespace_reference;
}
// Store attributes with their namespaces and discard the temporary
// qualified attributes array.
$this->attributes = $namespaced_attributes;
$this->qualified_attributes = array();
$this->element = new XMLElement( $tag_local_name, $tag_namespace_prefix, $namespaces[ $tag_namespace_prefix ], $namespaces );
// Closers assume $this->element is the element created for the opener.
// @see step_in_element.
}
/*
* Preserve the opening tag pointers, as these will be overwritten
* when finding the closing tag. They will be reset after finding
* the closing to tag to point to the opening of the special atomic
* tag sequence.
*/
$tag_name_starts_at = $this->tag_name_starts_at;
$tag_name_length = $this->tag_name_length;
$tag_ends_at = $this->token_starts_at + $this->token_length;
$attributes = $this->qualified_attributes;
/*
* The values here look like they reference the opening tag but they reference
* the closing tag instead. This is why the opening tag values were stored
* above in a variable. It reads confusingly here, but that's because the
* functions that skip the contents have moved all the internal cursors past
* the inner content of the tag.
*/
$this->token_starts_at = $was_at;
$this->token_length = $this->bytes_already_parsed - $this->token_starts_at;
$this->text_starts_at = $tag_ends_at;
$this->text_length = $this->tag_name_starts_at - $this->text_starts_at;
$this->tag_name_starts_at = $tag_name_starts_at;
$this->tag_name_length = $tag_name_length;
$this->qualified_attributes = $attributes;
return true;
}
private function get_tag_namespaces_in_scope() {
$top = $this->top_element();
if ( null === $top ) {
// Namespaces defined by default in every XML document.
return $this->document_namespaces;
}
return $top->namespaces_in_scope;
}
/**
* Returns the namespace prefix of the matched tag or, when the $namespace
* argument is given, the prefix of the requested fully-qualified namespace .
*
* Examples:
*
* $p = new XMLProcessor( 'Test' );
* $p->next_tag() === true;
* $p->get_tag_namespace_prefix() === 'xhtml';
*
* $p = new XMLProcessor( '
*
* Test
*
* ' );
* $p->next_tag() === true;
* $p->get_tag_namespace_prefix('http://wordpress.org/export/1.2/') === 'wp';
*
* @internal
* @param string|null $xml_namespace Fully-qualified namespace to return the prefix for.
* @return string|null The namespace prefix of the matched tag, or null if not available.
*/
private function get_tag_namespace_prefix( $xml_namespace = null ) {
if ( null === $xml_namespace ) {
if ( self::STATE_MATCHED_TAG !== $this->parser_state ) {
return null;
}
return $this->element->namespace_prefix;
} else {
$namespaces_in_scope = $this->get_tag_namespaces_in_scope();
foreach ( $namespaces_in_scope as $prefix => $uri ) {
if ( $uri === $xml_namespace ) {
return $prefix;
}
}
return false;
}
}
/**
* Returns the top XMLElement on the stack without removing it.
*
* @return XMLElement|null Returns the top element, or null if stack is empty.
*/
private function top_element() {
if ( empty( $this->stack_of_open_elements ) ) {
return null;
}
return $this->stack_of_open_elements[ count( $this->stack_of_open_elements ) - 1 ];
}
/**
* Whether the processor paused because the input XML document ended
* in the middle of a syntax element, such as in the middle of a tag.
*
* Example:
*
* $processor = new XMLProcessor( '
Surprising fact you may no…
* ^ ^
* \-|-- it shifts with edits
*
* Bookmarks provide the ability to seek to a previously-scanned
* place in the XML document. This avoids the need to re-scan
* the entire document.
*
* Example:
*
*
) and false for empty elements () and tag closers (
).
*
* This method exists to provide a consistent interface with WP_HTML_Processor.
*
* @return bool Whether the tag is expected to be closed.
*/
public function expects_closer() {
if ( self::STATE_MATCHED_TAG !== $this->parser_state ) {
return false;
}
return $this->is_tag_opener() && ! $this->is_empty_element();
}
/**
* Indicates if the currently matched tag is an empty element tag.
*
* XML tags ending with a solidus ("/") are parsed as empty elements. They have no
* content and no matching closer is expected.
*
* @return bool Whether the currently matched tag is an empty element tag.
* @since WP_VERSION
*/
public function is_empty_element() {
if ( self::STATE_MATCHED_TAG !== $this->parser_state ) {
return false;
}
/*
* An empty element tag is defined by the solidus at the _end_ of the tag, not the beginning.
*
* Example:
*
*
* ^ this appears one character before the end of the closing ">".
*/
return '/' === $this->xml[ $this->token_starts_at + $this->token_length - 2 ];
}
/**
* Indicates if the current tag token is a tag closer.
*
* Example:
*
* $p = new XMLProcessor( '' );
* $p->next_tag( array( 'tag_name' => 'content', 'tag_closers' => 'visit' ) );
* $p->is_tag_closer() === false;
*
* $p->next_tag( array( 'tag_name' => 'content', 'tag_closers' => 'visit' ) );
* $p->is_tag_closer() === true;
*
* @return bool Whether the current tag is a tag closer.
* @since WP_VERSION
*/
public function is_tag_closer() {
return (
self::STATE_MATCHED_TAG === $this->parser_state &&
$this->is_closing_tag
);
}
/**
* Indicates if the current tag token is a tag opener.
*
* Example:
*
* $p = new XMLProcessor( '' );
* $p->next_token();
* $p->is_tag_opener() === true;
*
* $p->next_token();
* $p->is_tag_opener() === false;
*
* @return bool Whether the current tag is a tag closer.
* @since WP_VERSION
*/
public function is_tag_opener() {
return (
self::STATE_MATCHED_TAG === $this->parser_state &&
! $this->is_closing_tag &&
! $this->is_empty_element()
);
}
/**
* Indicates the kind of matched token, if any.
*
* This differs from `get_token_name()` in that it always
* returns a static string indicating the type, whereas
* `get_token_name()` may return values derived from the
* token itself, such as a tag name or processing
* instruction tag.
*
* Possible values:
* - `#tag` when matched on a tag.
* - `#text` when matched on a text node.
* - `#cdata-section` when matched on a CDATA node.
* - `#comment` when matched on a comment.
* - `#presumptuous-tag` when matched on an empty tag closer.
*
* @return string|null What kind of token is matched, or null.
* @since WP_VERSION
*/
public function get_token_type() {
switch ( $this->parser_state ) {
case self::STATE_MATCHED_TAG:
return '#tag';
default:
return $this->get_token_name();
}
}
/**
* Returns the node name represented by the token.
*
* This matches the DOM API value `nodeName`. Some values
* are static, such as `#text` for a text node, while others
* are dynamically generated from the token itself.
*
* Dynamic names:
* - Uppercase tag name for tag matches.
*
* Note that if the Tag Processor is not matched on a token
* then this function will return `null`, either because it
* hasn't yet found a token or because it reached the end
* of the document without matching a token.
*
* @return string|null Name of the matched token.
* @since WP_VERSION
*/
public function get_token_name() {
switch ( $this->parser_state ) {
case self::STATE_MATCHED_TAG:
return $this->get_tag_local_name();
case self::STATE_TEXT_NODE:
return '#text';
case self::STATE_CDATA_NODE:
return '#cdata-section';
case self::STATE_DOCTYPE_NODE:
return '#doctype';
case self::STATE_XML_DECLARATION:
return '#xml-declaration';
case self::STATE_PI_NODE:
return '#processing-instructions';
case self::STATE_COMMENT:
return '#comment';
case self::STATE_COMPLETE:
return '#complete';
case self::STATE_INVALID_DOCUMENT:
return '#error';
default:
return '#none';
}
}
/**
* Returns the modifiable text for a matched token, or an empty string.
*
* Modifiable text is text content that may be read and changed without
* changing the XML structure of the document around it. This includes
* the contents of `#text` and `#cdata-section` nodes in the XML as well
* as the inner contents of XML comments, Processing Instructions, and others.
*
* If a token has no modifiable text then an empty string is returned to
* avoid needless crashing or type errors. An empty string does not mean
* that a token has modifiable text, and a token with modifiable text may
* have an empty string (e.g. a comment with no contents).
*
* @return string
* @since WP_VERSION
*/
public function get_modifiable_text() {
if ( null === $this->text_starts_at ) {
return '';
}
$text = substr( $this->xml, $this->text_starts_at, $this->text_length );
/*
* > the XML processor must behave as if it normalized all line breaks in external parsed
* > entities (including the document entity) on input, before parsing, by translating both
* > the two-character sequence #xD #xA and any #xD that is not followed by #xA to a single
* > #xA character.
*
* See https://www.w3.org/TR/xml/#sec-line-ends
*/
$text = str_replace( array( "\r\n", "\r" ), "\n", $text );
// Comment data and CDATA sections contents are not decoded any further.
if (
self::STATE_CDATA_NODE === $this->parser_state ||
self::STATE_COMMENT === $this->parser_state
) {
return $text;
}
$decoded = XMLDecoder::decode( $text );
if ( ! isset( $decoded ) ) {
/**
* If the attribute contained an invalid value, it's
* a fatal error.
*
* @see WP_XML_Decoder::decode()
*/
$this->last_error = self::ERROR_SYNTAX;
_doing_it_wrong(
__METHOD__,
__( 'Invalid text content encountered.' ),
'WP_VERSION'
);
return false;
}
return $decoded;
}
/**
* Updates the modifiable text for a matched token.
*
* Modifiable text is text content that may be read and changed without
* changing the XML structure of the document around it. This includes
* the contents of `#text` and `#cdata-section` nodes in the XML as well
* as the inner contents of XML comments, Processing Instructions, and others.
*
* If a token has no modifiable text then false is returned to avoid needless
* crashing or type errors.
*
* @param $new_value New modifiable text for the current node.
* @return string
*/
public function set_modifiable_text( $new_value ) {
switch ( $this->parser_state ) {
case self::STATE_TEXT_NODE:
case self::STATE_COMMENT:
$this->lexical_updates[] = new WP_HTML_Text_Replacement(
$this->text_starts_at,
$this->text_length,
// @TODO: Audit this in details. Is this too naive? Or is it actually safe?
htmlspecialchars( $new_value, ENT_XML1, 'UTF-8' )
);
return true;
case self::STATE_CDATA_NODE:
$this->lexical_updates[] = new WP_HTML_Text_Replacement(
$this->text_starts_at,
$this->text_length,
// @TODO: Audit this in details. Is this too naive? Or is it actually safe?
str_replace( ']]>', ']]>', $new_value )
);
return true;
default:
_doing_it_wrong(
__METHOD__,
__( 'Cannot set text content on a non-text node.' ),
'WP_VERSION'
);
return false;
}
}
/**
* Updates or creates a new attribute on the currently matched tag with the passed value.
*
* For boolean attributes special handling is provided:
* - When `true` is passed as the value, then only the attribute name is added to the tag.
* - When `false` is passed, the attribute gets removed if it existed before.
*
* For string attributes, the value is escaped using the `esc_attr` function.
*
* @param string $xml_namespace The attribute's namespace.
* @param string $local_name The attribute name to target.
* @param string|bool $value The new attribute value.
*
* @return bool Whether an attribute value was set.
* @since WP_VERSION
*/
public function set_attribute( $xml_namespace, $local_name, $value ) {
if ( ! is_string( $value ) ) {
_doing_it_wrong(
__METHOD__,
__( 'Non-string attribute values cannot be passed to set_attribute().' ),
'WP_VERSION'
);
return false;
}
if ( 'xmlns' === $xml_namespace ) {
$this->bail(
__( 'Setting attributes in the xmlns namespace is not yet supported by set_attribute().' ),
$xml_namespace
);
return false;
}
if (
self::STATE_MATCHED_TAG !== $this->parser_state ||
$this->is_closing_tag
) {
return false;
}
$value = htmlspecialchars( $value, ENT_XML1, 'UTF-8' );
if ( '' !== $xml_namespace ) {
$prefix = $this->get_tag_namespace_prefix( $xml_namespace );
if ( false === $prefix ) {
$this->bail(
// Translators: 1: The XML namespace.
__( 'The namespace "%1$s" is not in the current element\'s scope.' ),
$xml_namespace
);
return false;
}
$name = $prefix . ':' . $local_name;
} else {
$name = $local_name;
}
$updated_attribute = "{$name}=\"{$value}\"";
/*
* > An attribute name must not appear more than once
* > in the same start-tag or empty-element tag.
* - XML 1.0 spec
*
* @see https://www.w3.org/TR/xml/#sec-starttags
*/
if ( isset( $this->attributes[ $name ] ) ) {
/*
* Update an existing attribute.
*
* Example – set attribute id to "new" in :
*
*
* ^-------------^
* start end
* replacement: `id="new"`
*
* Result:
*/
$existing_attribute = $this->attributes[ $name ];
$this->lexical_updates[ $name ] = new WP_HTML_Text_Replacement(
$existing_attribute->start,
$existing_attribute->length,
$updated_attribute
);
} else {
/*
* Create a new attribute at the tag's name end.
*
* Example – add attribute id="new" to :
*
*
* ^
* start and end
* replacement: ` id="new"`
*
* Result:
*/
$this->lexical_updates[ $name ] = new WP_HTML_Text_Replacement(
$this->tag_name_starts_at + $this->tag_name_length,
0,
' ' . $updated_attribute
);
}
return true;
}
/**
* Remove an attribute from the currently-matched tag.
*
* @param string $xml_namespace The attribute's namespace.
* @param string $local_name The attribute name to remove.
*
* @return bool Whether an attribute was removed.
* @since WP_VERSION
*/
public function remove_attribute( $xml_namespace, $local_name ) {
if (
self::STATE_MATCHED_TAG !== $this->parser_state ||
$this->is_closing_tag
) {
return false;
}
$name = $xml_namespace ? '{' . $xml_namespace . '}' . $local_name : $local_name;
/*
* If updating an attribute that didn't exist in the input
* document, then remove the enqueued update and move on.
*
* For example, this might occur when calling `remove_attribute()`
* after calling `set_attribute()` for the same attribute
* and when that attribute wasn't originally present.
*/
if ( ! isset( $this->attributes[ $name ] ) ) {
if ( isset( $this->lexical_updates[ $name ] ) ) {
unset( $this->lexical_updates[ $name ] );
}
return false;
}
/*
* Removes an existing tag attribute.
*
* Example – remove the attribute id from :
*
* ^-------------^
* start end
* replacement: ``
*
* Result:
*/
$this->lexical_updates[ $name ] = new WP_HTML_Text_Replacement(
$this->attributes[ $name ]->start,
$this->attributes[ $name ]->length,
''
);
return true;
}
/**
* Returns the string representation of the XML Tag Processor.
*
* @return string The processed XML.
* @see XMLProcessor::get_updated_xml()
*
* @since WP_VERSION
*/
public function __toString() {
return $this->get_updated_xml();
}
/**
* Returns the string representation of the XML Tag Processor.
*
* @return string The processed XML.
* @since WP_VERSION
*/
public function get_updated_xml() {
$requires_no_updating = 0 === count( $this->lexical_updates );
/*
* When there is nothing more to update and nothing has already been
* updated, return the original document and avoid a string copy.
*/
if ( $requires_no_updating ) {
return $this->xml;
}
/*
* Keep track of the position right before the current token. This will
* be necessary for reparsing the current token after updating the XML.
*/
$before_current_token = isset( $this->token_starts_at ) ? $this->token_starts_at : 0;
/*
* 1. Apply the enqueued edits and update all the pointers to reflect those changes.
*/
$before_current_token += $this->apply_lexical_updates( $before_current_token );
/*
* 2. Rewind to before the current tag and reparse to get updated attributes.
*
* At this point the internal cursor points to the end of the tag name.
* Rewind before the tag name starts so that it's as if the cursor didn't
* move; a call to `next_tag()` will reparse the recently-updated attributes
* and additional calls to modify the attributes will apply at this same
* location, but in order to avoid issues with subclasses that might add
* behaviors to `next_tag()`, the internal methods should be called here
* instead.
*
* It's important to note that in this specific place there will be no change
* because the processor was already at a tag when this was called and it's
* rewinding only to the beginning of this very tag before reprocessing it
* and its attributes.
*
*
Previous XMLMore XML
* ↑ │ back up by the length of the tag name plus the opening <
* └←─┘ back up by strlen("em") + 1 ==> 3
*/
$this->bytes_already_parsed = $before_current_token;
$this->parse_next_token();
return $this->xml;
}
/**
* Finds the next token in the XML document.
*
* An XML document can be viewed as a stream of tokens,
* where tokens are things like XML tags, XML comments,
* text nodes, etc. This method finds the next token in
* the XML document and returns whether it found one.
*
* If it starts parsing a token and reaches the end of the
* document then it will seek to the start of the last
* token and pause, returning `false` to indicate that it
* failed to find a complete token.
*
* Possible token types, based on the XML specification:
*
* - an XML tag
* - a text node - the plaintext inside tags.
* - a CData section
* - an XML comment.
* - a DOCTYPE declaration.
* - a processing instruction, e.g. ``.
*
* @return bool Whether a token was parsed.
*/
public function next_token() {
return $this->step();
}
/**
* Moves the internal cursor to the next token in the XML document
* according to the XML specification.
*
* It considers the current XML context (prolog, element, or misc)
* and only expects the nodes that are allowed in that context.
*
* @param int $node_to_process Whether to process the next node or
* reprocess the current node, e.g. using another parser context.
*
* @return bool Whether a token was parsed.
* @since WP_VERSION
*
* @access private
*/
private function step( $node_to_process = self::PROCESS_NEXT_NODE ) {
// Refuse to proceed if there was a previous error.
if ( null !== $this->last_error ) {
return false;
}
// Finish stepping when there are no more tokens in the document.
if (
self::STATE_INCOMPLETE_INPUT === $this->parser_state ||
self::STATE_COMPLETE === $this->parser_state
) {
return false;
}
if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
if ( $this->is_empty_element() ) {
array_pop( $this->stack_of_open_elements );
}
}
try {
switch ( $this->parser_context ) {
case self::IN_PROLOG_CONTEXT:
return $this->step_in_prolog( $node_to_process );
case self::IN_ELEMENT_CONTEXT:
return $this->step_in_element( $node_to_process );
case self::IN_MISC_CONTEXT:
return $this->step_in_misc( $node_to_process );
default:
$this->last_error = self::ERROR_UNSUPPORTED;
return false;
}
} catch ( XMLUnsupportedException $e ) {
/*
* Exceptions are used in this class to escape deep call stacks that
* otherwise might involve messier calling and return conventions.
*/
return false;
}
}
/**
* Parses the next node in the 'prolog' part of the XML document.
*
* @return bool Whether a node was found.
* @see https://www.w3.org/TR/xml/#NT-document.
* @see XMLProcessor::step
*
* @since WP_VERSION
*/
private function step_in_prolog( $node_to_process = self::PROCESS_NEXT_NODE ) {
if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
$has_next_node = $this->parse_next_token();
if (
false === $has_next_node &&
! $this->expecting_more_input
) {
$this->bail( 'The root element was not found.', self::ERROR_SYNTAX );
}
}
// XML requires a root element. If we've reached the end of data in the prolog stage,
// before finding a root element, then the document is incomplete.
if ( self::STATE_COMPLETE === $this->parser_state ) {
$this->mark_incomplete_input();
return false;
}
// Do not step if we paused due to an incomplete input.
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
return false;
}
switch ( $this->get_token_type() ) {
case '#text':
$text = $this->get_modifiable_text();
$whitespaces = strspn( $text, " \t\n\r" );
if ( strlen( $text ) !== $whitespaces ) {
// @TODO: Only look for this in the 2 initial bytes of the document:.
if ( "\xFF\xFE" === substr( $text, 0, 2 ) ) {
$this->bail( 'Unexpected UTF-16 BOM byte sequence (0xFFFE) in the document. XMLProcessor only supports UTF-8.', self::ERROR_SYNTAX );
}
$this->bail( 'Unexpected non-whitespace text token in prolog stage.', self::ERROR_SYNTAX );
}
return $this->step();
// @TODO: Fail if there's more than one or if was found before the XML declaration token.
case '#doctype':
case '#comment':
case '#xml-declaration':
case '#processing-instructions':
return true;
case '#tag':
$this->parser_context = self::IN_ELEMENT_CONTEXT;
return $this->step( self::PROCESS_CURRENT_NODE );
default:
$this->bail( 'Unexpected token type in prolog stage.', self::ERROR_SYNTAX );
}
}
/**
* Parses the next node in the 'element' part of the XML document.
*
* @return bool Whether a node was found.
* @see https://www.w3.org/TR/xml/#NT-document.
* @see XMLProcessor::step
*
* @since WP_VERSION
*/
private function step_in_element( $node_to_process = self::PROCESS_NEXT_NODE ) {
if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
$has_next_node = $this->parse_next_token();
if (
false === $has_next_node &&
! $this->expecting_more_input
) {
$this->bail( 'A tag was not closed.', self::ERROR_SYNTAX );
}
}
// Do not step if we paused due to an incomplete input.
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
return false;
}
switch ( $this->get_token_type() ) {
case '#text':
case '#cdata-section':
case '#comment':
case '#processing-instructions':
return true;
case '#tag':
// Update the stack of open elements.
$tag_qname = $this->get_tag_name_qualified();
if ( $this->is_tag_closer() ) {
if ( ! count( $this->stack_of_open_elements ) ) {
$this->bail(
// Translators: 1: The closing tag name. 2: The opening tag name.
__( 'The closing tag "%1$s" did not match the opening tag "%2$s".' ),
$tag_qname,
$tag_qname
);
return false;
}
$this->element = array_pop( $this->stack_of_open_elements );
$popped_qname = $this->element->qualified_name;
if ( $popped_qname !== $tag_qname ) {
$this->bail(
sprintf(
// translators: %1$s is the name of the closing HTML tag, %2$s is the name of the opening HTML tag.
__( 'The closing tag "%1$s" did not match the opening tag "%2$s".' ),
$tag_qname,
$popped_qname
),
self::ERROR_SYNTAX
);
}
if ( 0 === count( $this->stack_of_open_elements ) ) {
$this->parser_context = self::IN_MISC_CONTEXT;
}
} else {
array_push( $this->stack_of_open_elements, $this->element );
$this->element = $this->top_element();
}
return true;
default:
$this->bail(
sprintf(
// translators: %1$s is the unexpected token type.
__( 'Unexpected token type "%1$s" in element stage.', 'data-liberation' ),
$this->get_token_type()
),
self::ERROR_SYNTAX
);
}
}
/**
* Parses the next node in the 'misc' part of the XML document.
*
* @return bool Whether a node was found.
* @see https://www.w3.org/TR/xml/#NT-document.
* @see XMLProcessor::step
*
* @since WP_VERSION
*/
private function step_in_misc( $node_to_process = self::PROCESS_NEXT_NODE ) {
if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
$has_next_node = $this->parse_next_token();
if (
false === $has_next_node &&
! $this->expecting_more_input
) {
// Parsing is complete.
$this->parser_state = self::STATE_COMPLETE;
return true;
}
}
// Do not step if we paused due to an incomplete input.
if ( self::STATE_INCOMPLETE_INPUT === $this->parser_state ) {
return false;
}
if ( self::STATE_COMPLETE === $this->parser_state ) {
return true;
}
switch ( $this->get_token_type() ) {
case '#comment':
case '#processing-instructions':
return true;
case '#text':
$text = $this->get_modifiable_text();
$whitespaces = strspn( $text, " \t\n\r" );
if ( strlen( $text ) !== $whitespaces ) {
$this->bail( 'Unexpected token type "' . $this->get_token_type() . '" in misc stage.', self::ERROR_SYNTAX );
}
return $this->step();
default:
$this->bail( 'Unexpected token type "' . $this->get_token_type() . '" in misc stage.', self::ERROR_SYNTAX );
}
}
/**
* Computes the XML breadcrumbs for the currently-matched element, if matched.
*
* Breadcrumbs start at the outermost parent and descend toward the matched element.
* They always include the entire path from the root XML node to the matched element.
* Example
*
* $processor = XMLProcessor::create_fragment( '
' );
* $processor->next_tag( 'img' );
* $processor->get_breadcrumbs() === array( 'p', 'strong', 'em', 'img' );
*
* @return string[]|null Array of tag names representing path to matched node, if matched, otherwise NULL.
* @since WP_VERSION
*/
public function get_breadcrumbs() {
return array_map(
function ( $element ) {
return array( $element->namespace, $element->local_name );
},
$this->stack_of_open_elements
);
}
/**
* Indicates if the currently-matched tag matches the given breadcrumbs.
*
* A "*" represents a single tag wildcard, where any tag matches, but not no tags.
*
* At some point this function _may_ support a `**` syntax for matching any number
* of unspecified tags in the breadcrumb stack. This has been intentionally left
* out, however, to keep this function simple and to avoid introducing backtracking,
* which could open up surprising performance breakdowns.
*
* Example:
*
* $processor = new XMLProcessor( '' );
* $processor->next_tag( 'img' );
* true === $processor->matches_breadcrumbs( array( 'content', 'image' ) );
* true === $processor->matches_breadcrumbs( array( 'post', 'content', 'image' ) );
* false === $processor->matches_breadcrumbs( array( 'post', 'image' ) );
* true === $processor->matches_breadcrumbs( array( 'post', '*', 'image' ) );
*
* @param string[] $breadcrumbs DOM sub-path at which element is found, e.g. `array( 'content', 'image' )`.
* May also contain the wildcard `*` which matches a single element, e.g. `array( 'post', '*' )`.
*
* @return bool Whether the currently-matched tag is found at the given nested structure.
* @since WP_VERSION
*/
public function matches_breadcrumbs( $breadcrumbs ) {
// Everything matches when there are zero constraints.
if ( 0 === count( $breadcrumbs ) ) {
return true;
}
// Start at the last crumb.
$crumb = end( $breadcrumbs );
if ( '#tag' !== $this->get_token_type() ) {
return false;
}
$open_elements = $this->stack_of_open_elements;
$crumb_count = count( $breadcrumbs );
$elem_count = count( $open_elements );
// Walk backwards through both arrays, matching each crumb to the corresponding open element.
for ( $j = 1; $j <= $crumb_count; $j++ ) {
$crumb = $breadcrumbs[ $crumb_count - $j ];
$element = isset( $open_elements[ $elem_count - $j ] ) ? $open_elements[ $elem_count - $j ] : null;
if ( ! $element ) {
return false;
}
// Normalize crumb to [namespace, local_name].
if ( ! is_array( $crumb ) ) {
if ( '*' === $crumb ) {
$crumb = array( '*', '*' );
} else {
$crumb = array( '*', $crumb );
}
}
list( $namespace, $local_name ) = $crumb;
// Match local name, respecting wildcard '*'.
if ( '*' !== $local_name && $local_name !== $element->local_name ) {
return false;
}
// Match namespace, respecting wildcard '*'.
if ( '*' !== $namespace && $namespace !== $element->namespace ) {
return false;
}
}
return true;
}
/**
* Returns the nesting depth of the current location in the document.
*
* Example:
*
* $processor = new XMLProcessor( '' );
* 0 === $processor->get_current_depth();
*
* // Opening the root element increases the depth.
* $processor->next_tag();
* 1 === $processor->get_current_depth();
*
* // Opening the text element increases the depth.
* $processor->next_tag();
* 2 === $processor->get_current_depth();
*
* // The text element is closed during `next_token()` so the depth is decreased to reflect that.
* $processor->next_token();
* 1 === $processor->get_current_depth();
*
* @return int Nesting-depth of current location in the document.
* @since WP_VERSION
*/
public function get_current_depth() {
return count( $this->stack_of_open_elements );
}
/**
* Parses a qualified name into a namespace prefix and local name.
*
* Example:
*
* $this->parse_qualified_name( 'wp:post' ); // Returns array( 'wp.org', 'post' )
* $this->parse_qualified_name( 'image' ); // Returns array( '', 'image' )
*
* @param string $qualified_name The qualified name to parse.
*
* @return array The namespace prefix and local name.
*/
private function parse_qualified_name( $qualified_name ) {
$namespace_prefix = '';
$local_name = $qualified_name;
$prefix_length = strcspn( $qualified_name, ':' );
if ( null !== $prefix_length && strlen( $qualified_name ) !== $prefix_length ) {
$namespace_prefix = substr( $qualified_name, 0, $prefix_length );
$local_name = substr( $qualified_name, $prefix_length + 1 );
}
return array( $namespace_prefix, $local_name );
}
/**
* Asserts a qualified tag name is syntactically valid according to the
* XML specification.
*
* @param string $qualified_name The qualified name to validate.
* @return bool Whether the qualified name is syntactically valid.
*/
private function validate_qualified_name( $qualified_name ) {
if ( substr_count( $qualified_name, ':' ) > 1 ) {
$this->bail(
sprintf(
'Invalid identifier "%s" – more than one ":" in tag name. Every tag name must contain either zero or one colon.',
$qualified_name
),
self::ERROR_SYNTAX
);
return false;
}
$prefix_length = strcspn( $qualified_name, ':' );
if ( 0 === $prefix_length && strlen( $qualified_name ) > 0 ) {
$this->bail(
sprintf( 'Invalid identifier "%s" – namespace qualifier must not have zero length.', $qualified_name ),
self::ERROR_SYNTAX
);
return false;
}
return true;
}
private function mark_incomplete_input(
$error_message = 'Unexpected syntax encountered.'
) {
if ( $this->expecting_more_input ) {
$this->parser_state = self::STATE_INCOMPLETE_INPUT;
return;
}
$this->parser_state = self::STATE_INVALID_DOCUMENT;
$this->last_error = self::ERROR_SYNTAX;
_doing_it_wrong( __METHOD__, $error_message, 'WP_VERSION' );
}
/**
* Returns context for why the parser aborted due to unsupported XML, if it did.
*
* This is meant for debugging purposes, not for production use.
*
* @return XMLUnsupportedException|null
*/
public function get_exception() {
return $this->exception;
}
/**
* Stops the parser and terminates its execution when encountering unsupported markup.
*
* @param string $message Explains support is missing in order to parse the current node.
*
* @throws XMLUnsupportedException Halts execution of the parser.
*/
private function bail( $message, $reason = self::ERROR_UNSUPPORTED ) {
$starts_at = isset( $this->token_starts_at ) ? $this->token_starts_at : strlen( $this->xml );
$length = isset( $this->token_length ) ? $this->token_length : 0;
$token = substr( $this->xml, $starts_at, $length );
$this->last_error = $reason;
$this->exception = new XMLUnsupportedException(
$message,
$this->get_token_type(),
$starts_at,
$token,
$this->get_breadcrumbs()
);
throw $this->exception;
}
/**
* Parser Ready State.
*
* Indicates that the parser is ready to run and waiting for a state transition.
* It may not have started yet, or it may have just finished parsing a token and
* is ready to find the next one.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_READY = 'STATE_READY';
/**
* Parser Complete State.
*
* Indicates that the parser has reached the end of the document and there is
* nothing left to scan. It finished parsing the last token completely.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_COMPLETE = 'STATE_COMPLETE';
/**
* Parser Incomplete Input State.
*
* Indicates that the parser has reached the end of the document before finishing
* a token. It started parsing a token but there is a possibility that the input
* XML document was truncated in the middle of a token.
*
* The parser is reset at the start of the incomplete token and has paused. There
* is nothing more than can be scanned unless provided a more complete document.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_INCOMPLETE_INPUT = 'STATE_INCOMPLETE_INPUT';
/**
* Parser Invalid Input State.
*
* Indicates that the parsed xml document contains malformed input and cannot be parsed.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_INVALID_DOCUMENT = 'STATE_INVALID_DOCUMENT';
/**
* Parser Matched Tag State.
*
* Indicates that the parser has found an XML tag and it's possible to get
* the tag name and read or modify its attributes (if it's not a closing tag).
*
* @since WP_VERSION
*
* @access private
*/
const STATE_MATCHED_TAG = 'STATE_MATCHED_TAG';
/**
* Parser Text Node State.
*
* Indicates that the parser has found a text node and it's possible
* to read and modify that text.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_TEXT_NODE = 'STATE_TEXT_NODE';
/**
* Parser CDATA Node State.
*
* Indicates that the parser has found a CDATA node and it's possible
* to read and modify its modifiable text. Note that in XML there are
* no CDATA nodes outside of foreign content (SVG and MathML). Outside
* of foreign content, they are treated as XML comments.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_CDATA_NODE = 'STATE_CDATA_NODE';
/**
* Parser DOCTYPE Node State.
*
* Indicates that the parser has found a DOCTYPE declaration and it's possible
* to read and modify its modifiable text.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_DOCTYPE_NODE = 'STATE_DOCTYPE_NODE';
/**
* Indicates that the parser has found an XML processing instruction.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_PI_NODE = 'STATE_PI_NODE';
/**
* Indicates that the parser has found an XML declaration
*
* @since WP_VERSION
*
* @access private
*/
const STATE_XML_DECLARATION = 'STATE_XML_DECLARATION';
/**
* Indicates that the parser has found an XML comment and it's
* possible to read and modify its modifiable text.
*
* @since WP_VERSION
*
* @access private
*/
const STATE_COMMENT = 'STATE_COMMENT';
/**
* Indicates that the parser encountered unsupported syntax and has bailed.
*
* @since WP_VERSION
*
* @var string
*/
const ERROR_SYNTAX = 'syntax';
/**
* Indicates that the provided XML document contains a declaration that is
* unsupported by the parser.
*
* @since WP_VERSION
*
* @var string
*/
const ERROR_UNSUPPORTED = 'unsupported';
/**
* Indicates that the parser encountered more XML tokens than it
* was able to process and has bailed.
*
* @since WP_VERSION
*
* @var string
*/
const ERROR_EXCEEDED_MAX_BOOKMARKS = 'exceeded-max-bookmarks';
/**
* Indicates that we're parsing the `prolog` part of the XML
* document.
*
* @since WP_VERSION
*
* @access private
*/
const IN_PROLOG_CONTEXT = 'prolog';
/**
* Indicates that we're parsing the `element` part of the XML
* document.
*
* @since WP_VERSION
*
* @access private
*/
const IN_ELEMENT_CONTEXT = 'element';
/**
* Indicates that we're parsing the `misc` part of the XML
* document.
*
* @since WP_VERSION
*
* @access private
*/
const IN_MISC_CONTEXT = 'misc';
/**
* Indicates that the next HTML token should be parsed and processed.
*
* @since WP_VERSION
*
* @var string
*/
const PROCESS_NEXT_NODE = 'process-next-node';
/**
* Indicates that the current HTML token should be processed without advancing the parser.
*
* @since WP_VERSION
*
* @var string
*/
const PROCESS_CURRENT_NODE = 'process-current-node';
/**
* Unlock code that must be passed into the constructor to create this class.
*
* This class extends the WP_HTML_Tag_Processor, which has a public class
* constructor. Therefore, it's not possible to have a private constructor here.
*
* This unlock code is used to ensure that anyone calling the constructor is
* doing so with a full understanding that it's intended to be a private API.
*
* @access private
*/
const CONSTRUCTOR_UNLOCK_CODE = 'Use WP_HTML_Processor::create_fragment() instead of calling the class constructor directly.';
}