p1: /v2/scrape, /v1/fetch, auth, CLI, contract tests

Single Fetcher.Fetch on the Guard that holds DenyPrefixes. skipTlsVerification
is ignored. No 502. keys add writes the secret only to --out. Release binary
has no -test-allow-prefix.

vendor/github.com/JohannesKaufmann/dom/.gitignore

@@ -0,0 +1,22 @@
+# If you prefer the allow list template instead of the deny list, see community template:
+# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
+#
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Go workspace file
+go.work
+go.work.sum

vendor/github.com/JohannesKaufmann/dom/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/JohannesKaufmann/dom/README.md

@@ -0,0 +1,277 @@
+# dom
+
+[![Go Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/dom.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/dom)
+
+Helper functions for "net/html" that make it easier to interact with `*html.Node`.
+
+🚀 [Getting Started](#getting-started) - 📚 [Documentation](#documentation) - 🧑‍💻 [Examples](/examples/)
+
+## Installation
+
+```bash
+go get -u github.com/JohannesKaufmann/dom
+```
+
+> [!NOTE]
+> This "dom" libary was developed for the needs of the [html-to-markdown](https://github.com/JohannesKaufmann/html-to-markdown) library.
+> That beeing said, please submit any functions that you need.
+
+## Getting Started
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func main() {
+	input := `
+	<ul>
+		<li><a href="github.com/JohannesKaufmann/dom">dom</a></li>
+		<li><a href="github.com/JohannesKaufmann/html-to-markdown">html-to-markdown</a></li>
+	</ul>
+	`
+
+	doc, err := html.Parse(strings.NewReader(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	// - - - //
+
+	firstLink := dom.FindFirstNode(doc, func(node *html.Node) bool {
+		return dom.NodeName(node) == "a"
+	})
+
+	fmt.Println("href:", dom.GetAttributeOr(firstLink, "href", ""))
+}
+```
+
+## Node vs Element
+
+The naming scheme in this library is:
+
+- "Node" means `*html.Node{}`
+  - This means _any_ node in the tree of nodes.
+- "Element" means `*html.Node{Type: html.ElementNode}`
+  - This means _only_ nodes with the type of `ElementNode`. For example `<p>`, `<span>`, `<a>`, ... but not `#text`, `<!--comment-->`, ...
+
+For most functions, there are two versions. For example:
+
+- `FirstChildNode()` and `FirstChildElement()`
+- `AllChildNodes()` and `AllChildElements()`
+- ...
+
+## Documentation
+
+[![Go Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/dom.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/dom)
+
+### Attributes & Content
+
+You can get the attributes of a node using `GetAttribute`, `GetAttributeOr` or the more specialized `GetClasses` that returns a slice of strings.
+
+For matching nodes, `HasID` and `HasClass` can be used.
+
+If you want to collect the #text of all the child nodes, you can call `CollectText`.
+
+```go
+name := dom.NodeName(node)
+// "h2"
+
+href := dom.GetAttributeOr(node, "href", "")
+// "github.com"
+
+isHeading := dom.HasClass(node, "repo__name")
+// `true`
+
+content := dom.CollectText(node)
+// "Lorem ipsum"
+```
+
+---
+
+### Children & Siblings
+
+You can already use `node.FirstChild` to get the first child _node_. For the convenience we added `FirstChildNode()` and `FirstChildElement()` which returns `*html.Node`.
+
+To get all direct children, use `AllChildNodes` and `AllChildElements` which returns `[]*html.Node`.
+
+- `PrevSiblingNode` and `PrevSiblingElement`
+
+- `NextSiblingNode` and `NextSiblingElement`
+
+### Find Nodes
+
+Searching for nodes deep in the tree is made easier with:
+
+```go
+firstParagraph := dom.FindFirstNode(doc, func(node *html.Node) bool {
+    return dom.NodeName(node) == "p"
+})
+// *html.Node
+
+
+allParagraphs := dom.FindAllNodes(doc, func(node *html.Node) bool {
+    return dom.NodeName(node) == "p"
+})
+// []*html.Node
+```
+
+- 🧑‍💻 [Example code, find](/examples/find/main.go)
+- 🧑‍💻 [Example code, selectors](/examples/selectors/main.go)
+
+---
+
+### Get next/previous neighbors
+
+What is special about this? The order!
+
+If you are somewhere in the DOM, you can call `GetNextNeighborNode` to get the next node, even if it is _further up_ the tree. The order is the same as you would see the elements in the DOM.
+
+```go
+node := startNode
+for node != nil {
+    fmt.Println(dom.NodeName(node))
+
+    node = dom.GetNextNeighborNode(node)
+}
+```
+
+If we start the `for` loop at the `<button>` and repeatedly call `GetNextNeighborNode` this would be the _order_ that the nodes are _visited_.
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─nav
+│ │ │ ├─p
+│ │ │ │ ├─#text "up"
+│ │ ├─main
+│ │ │ ├─button   *️⃣
+│ │ │ │ ├─span  0️⃣
+│ │ │ │ │ ├─#text "start"  1️⃣
+│ │ │ ├─div  2️⃣
+│ │ │ │ ├─h3  3️⃣
+│ │ │ │ │ ├─#text "heading"  4️⃣
+│ │ │ │ ├─p  5️⃣
+│ │ │ │ │ ├─#text "description"  6️⃣
+│ │ ├─footer  7️⃣
+│ │ │ ├─p  8️⃣
+│ │ │ │ ├─#text "down"  9️⃣
+```
+
+If you only want to visit the ElementNode's (and skip the `#text` Nodes) you can use `GetNextNeighborElement` instead.
+
+If you want to skip the children you can use `GetNextNeighborNodeExcludingOwnChild`. In the example above, when starting at the `<button>` the next node would be the `<div>`.
+
+The same functions also exist for the previous nodes, e.g. `GetPrevNeighborNode`.
+
+- 🧑‍💻 [Example code, next basics](/examples/next_basics/main.go)
+- 🧑‍💻 [Example code, next inside a loop](/examples/next_loop/main.go)
+
+---
+
+### Remove & Replace Node
+
+```go
+if dom.HasClass(node, "lang__old") {
+	newNode := &html.Node{
+		Type: html.TextNode,
+		Data: "🪦",
+	}
+	dom.ReplaceNode(node, newNode)
+}
+
+
+for _, node := range emptyTextNodes {
+	dom.RemoveNode(node)
+}
+```
+
+- 🧑‍💻 [Example code, remove and replace](/examples/remove_replace/main.go)
+
+### Unwrap Node
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─article   *️⃣
+│ │ │ ├─h3
+│ │ │ │ ├─#text "Heading"
+│ │ │ ├─p
+│ │ │ │ ├─#text "short description"
+```
+
+If we take the input above and run `UnwrapNode(articleNode)` we can "unwrap" the `<article>`. That means removing the `<article>` while _keeping_ the children (`<h3>` and `<p>`).
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─h3
+│ │ │ ├─#text "Heading"
+│ │ ├─p
+│ │ │ ├─#text "short description"
+```
+
+For the reverse you can use `WrapNode(existingNode, newNode)`.
+
+---
+
+### RenderRepresentation
+
+```go
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func main() {
+	input := `<a href="/about">Read More</a>`
+
+	doc, err := html.Parse(strings.NewReader(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	fmt.Println(dom.RenderRepresentation(doc))
+}
+```
+
+The tree representation helps to visualize the tree-structure of the DOM.
+And the `#text` nodes stand out.
+
+> [!TIP]
+> This function could be useful for debugging & testcases.
+> For example in [neighbors_test.go](/neighbors_test.go)
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─a (href=/about)
+│ │ │ ├─#text "Read More"
+```
+
+While the normal "net/html" [`Render()`](https://pkg.go.dev/golang.org/x/net/html#Render) function would have produced this:
+
+```
+<html><head></head><body><a href="/about">Read More</a></body></html>
+```
+
+- 🧑‍💻 [Example code, dom representation](/examples/dom_representation/main.go)

vendor/github.com/JohannesKaufmann/dom/attr.go

@@ -0,0 +1,68 @@
+package dom
+
+import (
+	"bytes"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+func GetAttribute(node *html.Node, key string) (string, bool) {
+	for _, attr := range node.Attr {
+		if attr.Key == key {
+			return attr.Val, true
+		}
+	}
+	return "", false
+}
+func GetAttributeOr(node *html.Node, key string, fallback string) string {
+	for _, attr := range node.Attr {
+		if attr.Key == key {
+			return attr.Val
+		}
+	}
+	return fallback
+}
+
+func GetClasses(node *html.Node) []string {
+	val, found := GetAttribute(node, "class")
+	if !found {
+		return nil
+	}
+
+	return strings.Fields(val)
+}
+
+func HasID(node *html.Node, expectedID string) bool {
+	val, found := GetAttribute(node, "id")
+	if !found {
+		return false
+	}
+	return strings.TrimSpace(val) == expectedID
+}
+func HasClass(node *html.Node, expectedClass string) bool {
+	classes := GetClasses(node)
+	for _, class := range classes {
+		if class == expectedClass {
+			return true
+		}
+	}
+	return false
+}
+
+// - - - - //
+
+func collectText(n *html.Node, buf *bytes.Buffer) {
+	if n.Type == html.TextNode {
+		buf.WriteString(n.Data)
+	}
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		collectText(c, buf)
+	}
+}
+
+func CollectText(node *html.Node) string {
+	var buf bytes.Buffer
+	collectText(node, &buf)
+	return buf.String()
+}

vendor/github.com/JohannesKaufmann/dom/change.go

@@ -0,0 +1,49 @@
+package dom
+
+import "golang.org/x/net/html"
+
+func RemoveNode(node *html.Node) {
+	if node == nil || node.Parent == nil {
+		return
+	}
+
+	node.Parent.RemoveChild(node)
+}
+
+func ReplaceNode(node, newNode *html.Node) {
+	if node.Parent == nil || node == newNode {
+		return
+	}
+
+	node.Parent.InsertBefore(newNode, node)
+	node.Parent.RemoveChild(node)
+}
+
+func UnwrapNode(node *html.Node) {
+	if node == nil || node.Parent == nil {
+		return
+	}
+
+	// In each iteration, we once again grab the first child, since
+	// the previous first child was just removed.
+	for child := node.FirstChild; child != nil; child = node.FirstChild {
+		node.RemoveChild(child)
+		node.Parent.InsertBefore(child, node)
+	}
+
+	node.Parent.RemoveChild(node)
+}
+
+// WrapNode wraps the newNode around the existingNode.
+func WrapNode(existingNode, newNode *html.Node) *html.Node {
+	if existingNode == nil || existingNode.Parent == nil {
+		return existingNode
+	}
+
+	existingNode.Parent.InsertBefore(newNode, existingNode)
+	existingNode.Parent.RemoveChild(existingNode)
+
+	newNode.AppendChild(existingNode)
+
+	return newNode
+}

vendor/github.com/JohannesKaufmann/dom/dom.go

@@ -0,0 +1,88 @@
+// dom makes it easier to interact with the html document.
+//
+// Node = return all the nodes
+// Element = return all the nodes that are of type Element. This e.g. excludes #text nodes.
+package dom
+
+import "golang.org/x/net/html"
+
+// AllNodes recursively gets all the nodes in the tree.
+func AllNodes(startNode *html.Node) (allNodes []*html.Node) {
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		allNodes = append(allNodes, node)
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(startNode)
+
+	return allNodes
+}
+
+// - - - - - - - - - - - - - - - //
+
+func AllChildNodes(node *html.Node) (children []*html.Node) {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		children = append(children, child)
+	}
+	return children
+}
+
+// AllChildElements is similar to AllChildNodes but only returns
+// nodes of type `ElementNode`.
+func AllChildElements(node *html.Node) (children []*html.Node) {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			children = append(children, child)
+		}
+	}
+	return children
+}
+
+// - - - - - - - - - - - - - - - //
+
+func FirstChildNode(node *html.Node) *html.Node {
+	return node.FirstChild
+}
+func FirstChildElement(node *html.Node) *html.Node {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			return child
+		}
+	}
+	return nil
+}
+
+// - - - - - - - - - - - - - - - //
+
+func PrevSiblingNode(node *html.Node) *html.Node {
+	return node.PrevSibling
+}
+func PrevSiblingElement(node *html.Node) *html.Node {
+	for sibling := node.PrevSibling; sibling != nil; sibling = sibling.PrevSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}
+
+// - - - - - - - - - - - - - - - //
+
+func NextSiblingNode(node *html.Node) *html.Node {
+	return node.NextSibling
+}
+
+// NextSiblingElement returns the element immediately following the passed-in node or nil.
+// In contrast to `node.NextSibling` this only returns the next `ElementNode`.
+func NextSiblingElement(node *html.Node) *html.Node {
+	for sibling := node.NextSibling; sibling != nil; sibling = sibling.NextSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}

vendor/github.com/JohannesKaufmann/dom/dom_representation.go

@@ -0,0 +1,64 @@
+package dom
+
+import (
+	"bytes"
+	"fmt"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+func writePipeChar(buf *bytes.Buffer, index int) {
+	if index == 0 {
+		return
+	}
+	buf.WriteString(strings.Repeat("│ ", index-1))
+	buf.WriteString("├─")
+}
+func writeNode(buf *bytes.Buffer, node *html.Node) {
+	name := NodeName(node)
+	buf.WriteString(name)
+
+	if len(node.Attr) != 0 {
+		buf.WriteString(" (")
+	}
+	for i, attr := range node.Attr {
+		buf.WriteString(fmt.Sprintf("%s=%q", attr.Key, attr.Val))
+
+		if i < len(node.Attr)-1 {
+			buf.WriteString(" ")
+		}
+	}
+	if len(node.Attr) != 0 {
+		buf.WriteString(")")
+	}
+
+	if name == "#text" {
+		buf.WriteString(fmt.Sprintf(" %q", node.Data))
+	}
+
+}
+
+// RenderRepresentation is useful for debugging.
+// It renders out the *structure* of the dom.
+func RenderRepresentation(startNode *html.Node) string {
+	var buf bytes.Buffer
+	var finder func(*html.Node, int)
+	finder = func(node *html.Node, index int) {
+		writePipeChar(&buf, index)
+		writeNode(&buf, node)
+		buf.WriteRune('\n')
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child, index+1)
+		}
+	}
+
+	if startNode.Parent == nil {
+		finder(startNode, 0)
+	} else {
+		finder(startNode, 1)
+	}
+
+	return strings.TrimSpace(buf.String())
+}

vendor/github.com/JohannesKaufmann/dom/find.go

@@ -0,0 +1,49 @@
+package dom
+
+import "golang.org/x/net/html"
+
+func ContainsNode(startNode *html.Node, matchFn func(node *html.Node) bool) bool {
+	return FindFirstNode(startNode, matchFn) != nil
+}
+
+func FindFirstNode(startNode *html.Node, matchFn func(node *html.Node) bool) *html.Node {
+	nextFunc := UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		func(node *html.Node) bool {
+			// We should not get higher up than the startNode...
+			return node == startNode
+		},
+	)
+
+	child := startNode.FirstChild
+	for child != nil {
+		if matchFn(child) {
+			return child
+		}
+
+		child = nextFunc(child)
+	}
+	return nil
+}
+
+func FindAllNodes(startNode *html.Node, matchFn func(node *html.Node) bool) (foundNodes []*html.Node) {
+	nextFunc := UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		func(node *html.Node) bool {
+			// We should not get higher up than the startNode...
+			return node == startNode
+		},
+	)
+
+	child := startNode.FirstChild
+	for child != nil {
+		if matchFn(child) {
+			foundNodes = append(foundNodes, child)
+		}
+
+		child = nextFunc(child)
+	}
+	return foundNodes
+}

vendor/github.com/JohannesKaufmann/dom/neighbors.go

@@ -0,0 +1,108 @@
+package dom
+
+import "golang.org/x/net/html"
+
+// Warning: It is not meant to be called directly and may change signature from release to release!
+func UNSTABLE_initGetNeighbor(
+	firstChildFunc func(node *html.Node) *html.Node,
+	prevNextFunc func(node *html.Node) *html.Node,
+	goUpUntilFunc func(node *html.Node) bool,
+) func(*html.Node) *html.Node {
+
+	return func(node *html.Node) *html.Node {
+		// First look at the children
+		if child := firstChildFunc(node); child != nil {
+			return child
+		}
+
+		// Otherwise my prev/next sibling
+		if sibling := prevNextFunc(node); sibling != nil {
+			return sibling
+		}
+
+		for {
+			// Finally, continously go upwards until we find an element with a sibling
+			node = node.Parent
+			if node == nil {
+				// We reached the top
+				return nil
+			}
+
+			if goUpUntilFunc(node) {
+				// Don't go too far up...
+				return nil
+			}
+
+			sibling := prevNextFunc(node)
+			if sibling != nil {
+				return sibling
+			}
+		}
+	}
+}
+
+// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //
+// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //
+
+var goUpForever = func(node *html.Node) bool { return false }
+var skipFirstChild = func(node *html.Node) *html.Node { return nil }
+
+func GetPrevNeighborNode(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		PrevSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborElement(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildElement,
+		PrevSiblingElement,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborNodeExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		PrevSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborElementExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		PrevSiblingElement,
+		goUpForever,
+	)(node)
+}
+
+// - - - - - - - - //
+
+func GetNextNeighborNode(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborElement(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildElement,
+		NextSiblingElement,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborNodeExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		NextSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborElementExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		NextSiblingElement,
+		goUpForever,
+	)(node)
+}

vendor/github.com/JohannesKaufmann/dom/tags.go

@@ -0,0 +1,147 @@
+package dom
+
+import "golang.org/x/net/html"
+
+// In order to stay consistent with v1 of the library, this follows
+// the naming scheme of goquery.
+// E.g. "#text", "div", ...
+func NodeName(node *html.Node) string {
+	if node == nil {
+		return ""
+	}
+
+	switch node.Type {
+	case html.ErrorNode:
+		return "#error"
+	case html.TextNode:
+		return "#text"
+	case html.DocumentNode:
+		return "#document"
+	case html.CommentNode:
+		return "#comment"
+
+	case html.DoctypeNode:
+		// E.g. for `<!DOCTYPE html>` it would be "html"
+		return node.Data
+	case html.ElementNode:
+		// E.g. "div" or "p"
+		return node.Data
+	}
+
+	return ""
+}
+
+func NameIsInlineNode(name string) bool {
+	switch name {
+	case
+		"#text",
+		"a",
+		"abbr",
+		"acronym",
+		"audio",
+		"b",
+		"bdi",
+		"bdo",
+		"big",
+		"br",
+		"button",
+		"canvas",
+		"cite",
+		"code",
+		"data",
+		"datalist",
+		"del",
+		"dfn",
+		"em",
+		"embed",
+		"i",
+		"iframe",
+		"img",
+		"input",
+		"ins",
+		"kbd",
+		"label",
+		"map",
+		"mark",
+		"meter",
+		"noscript",
+		"object",
+		"output",
+		"picture",
+		"progress",
+		"q",
+		"ruby",
+		"s",
+		"samp",
+		"script",
+		"select",
+		"slot",
+		"small",
+		"span",
+		"strong",
+		"sub",
+		"sup",
+		"svg",
+		"template",
+		"textarea",
+		"time",
+		"u",
+		"tt",
+		"var",
+		"video",
+		"wbr":
+		return true
+
+	default:
+		return false
+	}
+}
+
+func NameIsBlockNode(name string) bool {
+	switch name {
+	case
+		"address",
+		"article",
+		"aside",
+		"blockquote",
+		"details",
+		"dialog",
+		"dd",
+		"div",
+		"dl",
+		"dt",
+		"fieldset",
+		"figcaption",
+		"figure",
+		"footer",
+		"form",
+		"h1", "h2", "h3", "h4", "h5", "h6",
+		"header",
+		"hgroup",
+		"hr",
+		"li",
+		"main",
+		"nav",
+		"ol",
+		"p",
+		"pre",
+		"section",
+		"table",
+		"ul":
+
+		return true
+
+	default:
+		return false
+	}
+}
+
+func NameIsHeading(name string) bool {
+	switch name {
+	case
+		"h1", "h2", "h3", "h4", "h5", "h6":
+		return true
+	default:
+		return false
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/.gitignore

@@ -0,0 +1,27 @@
+
+# - - - - - General - - - - - #
+
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, build with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+.DS_Store
+
+
+
+# - - - - - Project Specific - - - - - #
+
+NOTES.md
+.tmp
+
+
+dist/

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/.goreleaser.yaml

@@ -0,0 +1,67 @@
+
+# The lines below are called `modelines`. See `:help modeline`
+# Feel free to remove those if you don't want/need to use them.
+# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
+# vim: set ts=2 sw=2 tw=0 fo=cnqoj
+
+version: 2
+
+before:
+  hooks:
+    - go mod tidy
+
+builds:
+  - env:
+      - CGO_ENABLED=0
+    goos:
+      - linux
+      - windows
+      - darwin
+
+    # Note: We only use goreleaser for the CLI,
+    # so we have to go into the "cli" directory.
+    dir: cli/html2markdown
+    binary: html2markdown
+
+archives:
+  - format: tar.gz
+    # this name template makes the OS and Arch compatible with the results of `uname`.
+    name_template: >-
+      {{ .ProjectName }}_
+      {{- title .Os }}_
+      {{- if eq .Arch "amd64" }}x86_64
+      {{- else if eq .Arch "386" }}i386
+      {{- else }}{{ .Arch }}{{ end }}
+      {{- if .Arm }}v{{ .Arm }}{{ end }}
+    # use zip for windows archives
+    format_overrides:
+      - goos: windows
+        format: zip
+
+brews:
+  - name: html2markdown
+    repository:
+      owner: JohannesKaufmann
+      name: homebrew-tap
+      token: "{{ .Env.TAP_GITHUB_TOKEN }}"
+
+nfpms:
+  - id: default
+    package_name: html2markdown
+
+    vendor: JohannesKaufmann
+    homepage: https://github.com/JohannesKaufmann/html-to-markdown
+    maintainer: Johannes Kaufmann <johannes@joina.de>
+    description: |
+      Convert HTML to Markdown. Even works with entire websites.
+    license: MIT
+
+    formats:
+      - deb
+
+changelog:
+  sort: asc
+  filters:
+    exclude:
+      - "^docs:"
+      - "^test:"

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/ESCAPING.md

@@ -0,0 +1,63 @@
+## Escaping
+
+Some characters have a special meaning in markdown. For example, the character "\*" can be used for lists, emphasis and dividers. By placing a backlash before that character (e.g. "\\\*") you can "escape" it. Then the character will render as a raw "\*" without the _"markdown meaning"_ applied.
+
+But why is "escaping" even necessary?
+
+<!-- prettier-ignore -->
+```md
+Paragraph 1
+-
+Paragraph 2
+```
+
+The markdown above doesn't seem that problematic. But "Paragraph 1" (with only one hyphen below) will be recognized as a _setext heading_.
+
+```html
+<h2>Paragraph 1</h2>
+<p>Paragraph 2</p>
+```
+
+A well-placed backslash character would prevent that...
+
+<!-- prettier-ignore -->
+```md
+Paragraph 1
+\-
+Paragraph 2
+```
+
+---
+
+How to configure escaping? Depending on the `EscapeMode` option, the markdown output is going to be different.
+
+```go
+conv := converter.NewConverter(
+    converter.WithEscapeMode("smart"), // default
+    converter.WithPlugins(
+        // ...
+    )
+)
+```
+
+Lets try it out with this HTML input:
+
+|          |                                                       |
+| -------- | ----------------------------------------------------- |
+| input    | `<p>fake **bold** and real <strong>bold</strong></p>` |
+|          |                                                       |
+|          | **With EscapeMode "smart"**                           |
+| output   | `fake \*\*bold\*\* and real **bold**`                 |
+| rendered | fake \*\*bold\*\* and real **bold**                   |
+|          |                                                       |
+|          | **With EscapeMode "disabled"**                        |
+| output   | `fake **bold** and real **bold**`                     |
+| rendered | fake **bold** and real **bold**                       |
+
+With **smart** escaping, we get some escape characters (the backlash "\\") but it renders correctly.
+
+With escaping **disabled**, the fake and real bold can't be distinguished in the markdown. That means it is both going to render as bold.
+
+---
+
+So now you know the purpose of escaping. However, if you encounter some content where the escaping breaks, you can manually disable it. But please also open an issue!

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2018 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/README.md

@@ -0,0 +1,399 @@
+# html-to-markdown
+
+A robust html-to-markdown converter that transforms HTML (even entire websites) into clean, readable Markdown. It supports complex formatting, customizable options, and plugins for full control over the conversion process.
+
+Use the fully extendable [Golang library](#golang-library) or a quick [CLI command](#cli---using-it-on-the-command-line). Alternatively, try the [Online Demo](https://html-to-markdown.com) or [REST API](https://html-to-markdown.com/api) to see it in action!
+
+Here are some _cool features_:
+
+- **Bold & Italic:** Supports bold and italic—even within single words.
+
+  ![](./.github/images/point_bold_italic.png)
+
+- **List:** Handles ordered and unordered lists with full nesting support.
+
+  ![](./.github/images/point_list.png)
+
+- **Blockquote:** Blockquotes can include other elements, with seamless support for nested quotes.
+
+  ![](./.github/images/point_blockquote.png)
+
+- **Inline Code & Code Block:** Correctly handles backticks and multi-line code blocks, preserving code structure.
+
+  ![](./.github/images/point_code.png)
+
+- **Link & Image:** Properly formats multi-line links, adding escapes for blank lines where needed.
+
+  ![](./.github/images/point_link_image.png)
+
+- **Smart Escaping:** Escapes special characters only when necessary, to avoid accidental Markdown rendering.
+  🗒️ [ESCAPING.md](/ESCAPING.md)
+
+  ![](./.github/images/point_escaping.png)
+
+- **Remove/Keep HTML:** Choose to strip or retain specific HTML tags for ultimate control over output.
+
+  ![](./.github/images/point_wrapper.png)
+
+- **Plugins:** Easily extend with plugins. Or create custom ones to enhance functionality.
+
+  ![](./.github/images/point_strikethrough.png)
+
+- **Table Plugin:** Converts tables with support for alignment, rowspan and colspan.
+
+  ![](./.github/images/point_table.png)
+
+---
+
+## Usage
+
+[💻 Golang library](#golang-library) | [📦 CLI](#cli---using-it-on-the-command-line) | [▶️ Hosted Demo](https://html-to-markdown.com) | [🌐 Hosted REST API](https://html-to-markdown.com/api)
+
+---
+
+## Golang Library
+
+### Installation
+
+```bash
+go get -u github.com/JohannesKaufmann/html-to-markdown/v2
+```
+
+_Or if you want a specific commit add the suffix `/v2@commithash`_
+
+> [!NOTE]  
+> This is the documentation for the v2 library. For the old version switch to the ["v1" branch](https://github.com/JohannesKaufmann/html-to-markdown/tree/v1).
+
+### Usage
+
+[![Go V2 Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/html-to-markdown/v2.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/html-to-markdown/v2)
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
+)
+
+func main() {
+	input := `<strong>Bold Text</strong>`
+
+	markdown, err := htmltomarkdown.ConvertString(input)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: **Bold Text**
+}
+```
+
+- 🧑‍💻 [Example code, basics](/examples/basics/main.go)
+
+Use `WithDomain` to convert _relative_ links to _absolute_ links:
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+)
+
+func main() {
+	input := `<img src="/assets/image.png" />`
+
+	markdown, err := htmltomarkdown.ConvertString(
+		input,
+		converter.WithDomain("https://example.com"),
+	)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: ![](https://example.com/assets/image.png)
+}
+```
+
+The function `htmltomarkdown.ConvertString()` is a _small wrapper_ around `converter.NewConverter()` and the _base_ and _commonmark_ plugins. If you want more control, use the following:
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark"
+)
+
+func main() {
+	input := `<strong>Bold Text</strong>`
+
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(
+				commonmark.WithStrongDelimiter("__"),
+				// ...additional configurations for the plugin
+			),
+
+			// ...additional plugins (e.g. table)
+		),
+	)
+
+	markdown, err := conv.ConvertString(input)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: __Bold Text__
+}
+```
+
+- 🧑‍💻 [Example code, options](/examples/options/main.go)
+
+> [!NOTE]  
+> If you use `NewConverter` directly make sure to also **register the commonmark and base plugin**.
+
+---
+
+### Collapse & Tag Type
+
+![](./.github/images/tag_type_renderer.png)
+
+You can specify how different HTML tags should be handled during conversion.
+
+- **Tag Types:** When _collapsing_ whitespace it is useful to know if a node is _block_ or _inline_.
+  - So if you have Web Components/Custom Elements remember to register the type using `TagType` or `RendererFor`.
+  - Additionally, you can _remove_ tags completely from the output.
+- **Pre-built Renderers:** There are several pre-built renderers available. For example:
+  - `RenderAsHTML` will render the node (including children) as HTML.
+  - `RenderAsHTMLWrapper` will render the node as HTML and render the children as markdown.
+
+> [!NOTE]  
+> By default, some tags are automatically removed (e.g. `<style>`). You can override existing configuration by using a different _priority_. For example, you could keep `<style>` tags by registering them with `PriorityEarly`.
+
+Here are the examples for the screenshot above:
+
+```go
+conv.Register.TagType("nav", converter.TagTypeRemove, converter.PriorityStandard)
+
+conv.Register.RendererFor("b", converter.TagTypeInline, base.RenderAsHTML, converter.PriorityEarly)
+
+conv.Register.RendererFor("article", converter.TagTypeBlock, base.RenderAsHTMLWrapper, converter.PriorityStandard)
+```
+
+### Plugins
+
+#### Published Plugins
+
+These are the plugins located in the [plugin folder](/plugin):
+
+| Name                  | Description                                                                                        |
+| --------------------- | -------------------------------------------------------------------------------------------------- |
+| Base                  | Implements basic shared functionality (e.g. removing nodes)                                        |
+| Commonmark            | Implements Markdown according to the [Commonmark Spec](https://spec.commonmark.org/)               |
+|                       |                                                                                                    |
+| GitHubFlavored        | _planned_                                                                                          |
+| TaskListItems         | _planned_                                                                                          |
+| Strikethrough         | Converts `<strike>`, `<s>`, and `<del>` to the `~~` syntax.                                        |
+| Table                 | Implements Tables according to the [GitHub Flavored Markdown Spec](https://github.github.com/gfm/) |
+|                       |                                                                                                    |
+| VimeoEmbed            | _planned_                                                                                          |
+| YoutubeEmbed          | _planned_                                                                                          |
+|                       |                                                                                                    |
+| ConfluenceCodeBlock   | _planned_                                                                                          |
+| ConfluenceAttachments | _planned_                                                                                          |
+
+> [!NOTE]  
+> Not all the plugins from v1 are already ported to v2. These will soon be implemented...
+
+These are the plugins in other repositories:
+
+| Name                         | Description         |
+| ---------------------------- | ------------------- |
+| \[Plugin Name\]\(Your Link\) | A short description |
+
+#### Writing Plugins
+
+You want to write custom logic?
+
+1. Write your logic and **register** it.
+
+   ![](./.github/images/autocomplete_register.png)
+   - 🧑‍💻 [Example code, register](/examples/register/main.go)
+
+2. _Optional:_ Package your logic into a **plugin** and publish it.
+   - 🗒️ [WRITING_PLUGINS.md](/WRITING_PLUGINS.md)
+
+---
+
+---
+
+## CLI - Using it on the command line
+
+Using the Golang library provides the most customization, while the CLI is the simplest way to get started.
+
+### Installation
+
+#### Homebrew Tap
+
+```bash
+brew install JohannesKaufmann/tap/html2markdown
+```
+
+#### Debian
+
+A `deb` package is available. See the [Setup Instructions](https://cloudsmith.io/~html-to-markdown/repos/stable/setup/#formats-deb).
+
+_Note: Support for other Linux distributions is tracked in [#119](https://github.com/JohannesKaufmann/html-to-markdown/issues/119)_
+
+#### Pre-compiled Binaries
+
+Download pre-compiled binaries for Linux, macOS or Windows from the [releases page](https://github.com/JohannesKaufmann/html-to-markdown/releases). Extract the archive and copy the executable to a location in your system PATH (e.g. `/usr/local/bin`).
+
+#### Installation via Go
+
+If you have Go installed, you can install the CLI directly using:
+
+```bash
+go install github.com/JohannesKaufmann/html-to-markdown/v2/cli/html2markdown@latest
+```
+
+This will download the source code and compile it into an executable in your Go binary directory (typically `$GOPATH/bin`).
+
+#### Build from Source
+
+Binaries are automatically built via [GoReleaser](https://goreleaser.com/) and attached to each [release](https://github.com/JohannesKaufmann/html-to-markdown/releases).
+
+To build locally (requires Go):
+
+```bash
+go build ./cli/html2markdown
+```
+
+### Version
+
+```bash
+html2markdown --version
+```
+
+> [!NOTE]  
+> Make sure that `--version` prints `2.X.X` as there is a different CLI for V2 of the converter.
+
+### Usage
+
+```bash
+$ echo "<strong>important</strong>" | html2markdown
+
+**important**
+```
+
+```text
+$ curl --no-progress-meter http://example.com | html2markdown
+
+# Example Domain
+
+This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.
+
+[More information...](https://www.iana.org/domains/example)
+```
+
+```bash
+$ html2markdown --input file.html --output file.md
+
+$ html2markdown --input "src/*.html" --output "dist/"
+```
+
+Use `--help` to learn about the configurations, for example:
+
+- `--domain="https://example.com"` to convert _relative_ links to _absolute_ links.
+- `--exclude-selector=".ad"` to exclude the html elements with `class="ad"` from the conversion.
+- `--include-selector="article"` to only include the `<article>` html elements in the conversion.
+- `--plugin-strikethrough` or `--plugin-table` to enable plugins.
+
+_(The cli does not support every option yet. Over time more customization will be added)_
+
+---
+
+---
+
+## FAQ
+
+### Character Encoding
+
+Regardless of which method you use (`ConvertString()`, `ConvertNode()`, etc.), the input is expected to _already_ be UTF-8 encoded. Under the hood, Go's `html.Parse()` is used to build the DOM and it requires UTF-8. It does not detect or convert other charsets.
+
+If your HTML uses a different charset (e.g. `ISO-8859-1`, `Windows-1252`), the replacement character `�` may appear in the output instead of the correct letters.
+
+This library does not handle charset detection or conversion. It is your responsibility to decode the HTML to UTF-8 before passing it in. When fetching HTML over HTTP, the `Content-Type` header tells you the charset. When reading from a file, you may need to detect it from `<meta>` tags or by inspecting the bytes. The [`golang.org/x/net/html/charset`](https://pkg.go.dev/golang.org/x/net/html/charset) package can help with these cases.
+
+### Extending with Plugins
+
+- Need your own logic? Write your own code and then **register** it.
+  - Don't like the **defaults** that the library uses? You can use `PriorityEarly` to run you logic _earlier_ than others.
+
+  - 🧑‍💻 [Example code, register](/examples/register/main.go)
+
+- If you believe that you logic could also benefit others, you can package it up into a **plugin**.
+  - 🗒️ [WRITING_PLUGINS.md](/WRITING_PLUGINS.md)
+
+### Bugs
+
+You found a bug?
+
+[Open an issue](https://github.com/JohannesKaufmann/html-to-markdown/issues/new/choose) with the HTML snippet that does not produce the expected results. Please, please, plase _submit the HTML snippet_ that caused the problem. Otherwise it is very difficult to reproduce and fix...
+
+### Security
+
+This library produces markdown that is readable and can be changed by humans.
+
+Once you convert this markdown back to HTML (e.g. using [goldmark](https://github.com/yuin/goldmark) or [blackfriday](https://github.com/russross/blackfriday)) you need to be careful of malicious content.
+
+This library does NOT sanitize untrusted content. Use an HTML sanitizer such as [bluemonday](https://github.com/microcosm-cc/bluemonday) before displaying the HTML in the browser.
+
+🗒️ [SECURITY.md](/SECURITY.md) if you find a security vulnerability
+
+### Goroutines
+
+You can use the `Converter` from (multiple) goroutines. Internally a mutex is used & there is a test to verify that behaviour.
+
+### Escaping & Backslash
+
+Some characters have a special meaning in markdown (e.g. "\*" for emphasis). The backslash `\` character is used to "escape" those characters. That is perfectly safe and won't be displayed in the final render.
+
+🗒️ [ESCAPING.md](/ESCAPING.md)
+
+### Contributing
+
+You want to contribute? Thats great to hear! There are many ways to help:
+
+Helping to answer questions, triaging issues, writing documentation, writing code, ...
+
+If you want to make a code change: Please first discuss the change you wish to make, by opening an issue. I'm also happy to guide you to where a change is most likely needed. There are also extensive tests (see below) so you can freely experiment 🧑‍🔬
+
+_Note: The outside API should not change because of backwards compatibility..._
+
+### Testing
+
+You don't have to be afraid of breaking the converter, since there are many "Golden File" tests:
+
+Add your problematic HTML snippet to one of the `.in.html` files in the `testdata` folders. Then run `go test -update` and have a look at which `.out.md` files changed in GIT.
+
+You can now change the internal logic and inspect what impact your change has by running `go test -update` again.
+
+_Note: Before submitting your change as a PR, make sure that you run those tests and check the files into GIT..._
+
+### License
+
+Unless otherwise specified, the project is licensed under the terms of the MIT license.
+
+🗒️ [LICENSE](/LICENSE)

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/SECURITY.md

@@ -0,0 +1,6 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+Please report (suspected) security vulnerabilities to johannes@joina.de with the subject _"Security html-to-markdown"_ and you will receive a response within 48 hours.
+

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/WRITING_PLUGINS.md

@@ -0,0 +1,5 @@
+# Writing Plugins
+
+Still in progress, check back later...
+
+TODO: info about writing plugins

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/collapse.go

@@ -0,0 +1,174 @@
+/*
+
+The function to collapse whitespace was adapted from the "turndown" library by Dom Christie,
+which was adapted from the "collapse-whitespace" library by Luc Thevenard.
+
+It was ported from Javascript to Golang by Johannes Kaufmann for the use in the "html-to-markdown" library.
+To increase performance the use of regex was replaced by custom code.
+
+https://github.com/wooorm/collapse-white-space
+https://github.com/mixmark-io/turndown
+https://github.com/JohannesKaufmann/html-to-markdown
+
+-----------
+
+MIT License
+
+Copyright (c) 2017 Dom Christie
+Copyright (c) 2014 Luc Thevenard <lucthevenard@gmail.com>
+Copyright (c) 2018 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+*/
+
+// collapse can collapse whitespace in html elements.
+//
+// It is a port from the Javascript library "turndown" to Golang.
+package collapse
+
+import (
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func nextNode(prev *html.Node, current *html.Node, domFuncs *DomFuncs) *html.Node {
+	if (prev != nil && prev.Parent == current) || domFuncs.IsPreformattedNode(current) {
+		if current.NextSibling != nil {
+			return current.NextSibling
+		}
+
+		return current.Parent
+	}
+
+	if current.FirstChild != nil {
+		return current.FirstChild
+	}
+	if current.NextSibling != nil {
+		return current.NextSibling
+	}
+
+	return current.Parent
+}
+
+func removeNode(node *html.Node) *html.Node {
+	next := node.NextSibling
+	if next == nil {
+		next = node.Parent
+	}
+
+	node.Parent.RemoveChild(node)
+
+	return next
+
+}
+
+type DomFuncs struct {
+	IsBlockNode        func(node *html.Node) bool
+	IsVoidNode         func(node *html.Node) bool
+	IsPreformattedNode func(node *html.Node) bool
+}
+
+func fillDefaultDomFuncs(domFuncs *DomFuncs) *DomFuncs {
+	if domFuncs == nil {
+		domFuncs = &DomFuncs{}
+	}
+	if domFuncs.IsBlockNode == nil {
+		domFuncs.IsBlockNode = defaultIsBlockNode
+	}
+	if domFuncs.IsVoidNode == nil {
+		domFuncs.IsVoidNode = defaultIsVoidNode
+	}
+	if domFuncs.IsPreformattedNode == nil {
+		domFuncs.IsPreformattedNode = defaultIsPreformattedNode
+	}
+	return domFuncs
+
+}
+func Collapse(element *html.Node, domFuncs *DomFuncs) {
+	domFuncs = fillDefaultDomFuncs(domFuncs)
+	// - - - - - - - - - - - - - - - - - - //
+
+	if element.FirstChild == nil || domFuncs.IsPreformattedNode(element) {
+		return
+	}
+
+	var prevText *html.Node = nil
+	var keepLeadingWs = false
+
+	var prev *html.Node = nil
+	var node = nextNode(prev, element, domFuncs)
+
+	for node != element {
+		if node.Type == html.TextNode /* node.nodeType == 4 */ { // Node.TEXT_NODE or Node.CDATA_SECTION_NODE
+			var text = replaceAnyWhitespaceWithSpace(node.Data)
+
+			if (prevText == nil || strings.HasSuffix(prevText.Data, " ")) &&
+				!keepLeadingWs && text != "" && text[0] == ' ' {
+				text = text[1:]
+			}
+
+			// `text` might be empty at this point.
+			if text == "" {
+				node = removeNode(node)
+				continue
+			}
+
+			node.Data = text
+
+			prevText = node
+		} else if node.Type == html.ElementNode { // Node.ELEMENT_NODE
+			if domFuncs.IsBlockNode(node) || dom.NodeName(node) == "br" {
+				if prevText != nil {
+					prevText.Data = strings.TrimSuffix(prevText.Data, " ")
+				}
+
+				prevText = nil
+				keepLeadingWs = false
+			} else if domFuncs.IsVoidNode(node) || domFuncs.IsPreformattedNode(node) || dom.NodeName(node) == "code" {
+				// Avoid trimming space around non-block, non-BR void elements and inline PRE.
+				prevText = nil
+				keepLeadingWs = true
+			} else if prevText != nil {
+				// Drop protection if set previously.
+				keepLeadingWs = false
+			}
+		} else if node.Type == html.CommentNode {
+			// TODO: Is this enough to keep the comments? Does this cause other problems?
+		} else {
+			// E.g. DoctypeNode
+
+			node = removeNode(node)
+			continue
+		}
+
+		var nextNode = nextNode(prev, node, domFuncs)
+		prev = node
+		node = nextNode
+	}
+
+	if prevText != nil {
+		prevText.Data = strings.TrimSuffix(prevText.Data, " ")
+		if prevText.Data == "" {
+			removeNode(prevText)
+		}
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/is_node.go

@@ -0,0 +1,43 @@
+package collapse
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+var blockElements = []string{
+	"address", "article", "aside", "audio", "blockquote", "body", "canvas", "center", "dd", "dir", "div", "dl", "dt", "fieldset", "figcaption", "figure", "footer", "form", "frameset", "h1", "h2", "h3", "h4", "h5", "h6", "header", "hgroup", "hr", "html", "isindex", "li", "main", "menu", "nav", "noframes", "noscript", "ol", "output", "p", "pre", "section", "table", "tbody", "td", "tfoot", "th", "thead", "tr", "ul",
+}
+
+var voidElements = []string{
+	// Note: Compared to the javascript implementation, I removed "source"
+	"area", "base", "br", "col", "command", "embed", "hr", "img", "input", "keygen", "link", "meta", "param" /* "source, "*/, "track", "wbr",
+}
+
+var defaultIsBlockNode = func(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	for _, elem := range blockElements {
+		if elem == name {
+			return true
+		}
+	}
+	return false
+}
+var defaultIsVoidNode = func(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	for _, elem := range voidElements {
+		if elem == name {
+			return true
+		}
+	}
+	return false
+}
+var defaultIsPreformattedNode = func(node *html.Node) bool {
+	// Note: Originally in the javascript version, this just checked for "pre".
+	// I changed it, to also return true for "code"
+	name := dom.NodeName(node)
+
+	return name == "pre" || name == "code"
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/whitespace.go

@@ -0,0 +1,79 @@
+package collapse
+
+import (
+	"unsafe"
+)
+
+func byteSliceToString(b []byte) string {
+	/* #nosec G103 */
+	return *(*string)(unsafe.Pointer(&b))
+}
+
+func replaceAnyWhitespaceWithSpace(source string) string {
+	if source == "" {
+		return source
+	}
+
+	// Some performance optimizations:
+	// - If no replacement was done, we return the original slice and dont allocate.
+	// - We batch appends
+	var ret []byte
+	makeIfNeeded := func() {
+		if ret == nil {
+			ret = make([]byte, 0, len(source))
+		}
+	}
+
+	startNormal := 0
+	startMatch := -1
+	for i := 0; i < len(source); i++ {
+		isWhitespace := source[i] == ' ' || source[i] == '\r' || source[i] == '\n' || source[i] == '\t'
+
+		if startMatch == -1 && isWhitespace {
+			// Start of newlines
+			startMatch = i
+			continue
+		} else if startMatch != -1 && isWhitespace {
+			// Middle of newlines
+			continue
+		} else if startMatch != -1 {
+			// Character after the last newline character
+
+			count := i - startMatch
+			if count == 1 && source[startMatch] == ' ' {
+				// There was only one `isWhitespace` match & that is a space.
+				// So the replacement would be exactly the same...
+			} else {
+				makeIfNeeded()
+				ret = append(ret, source[startNormal:startMatch]...)
+				ret = append(ret, byte(' '))
+				startNormal = i
+			}
+
+			startMatch = -1
+		}
+	}
+
+	if startMatch == -1 && startNormal == 0 {
+		// a) no changes need to be done
+	} else if startMatch == -1 {
+		// b) Only the normal characters until the end still need to be added
+		makeIfNeeded()
+		ret = append(ret, source[startNormal:]...)
+	} else if ret == nil && len(source)-startMatch == 1 && source[startMatch] == ' ' {
+		// c) There is a match, but it is exactly the same as the replacement
+		//    If there is no new slice, we can skip the replacement.
+	} else {
+		// d) The match still needs to be replaced (and possible the previous normal characters be added)
+		makeIfNeeded()
+		ret = append(ret, source[startNormal:startMatch]...)
+		ret = append(ret, byte(' '))
+	}
+
+	if ret == nil {
+		// Huray, we did not do any allocations with make()
+		// and instead just return the original slice.
+		return source
+	}
+	return byteSliceToString(ret)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/convert.go

@@ -0,0 +1,54 @@
+package htmltomarkdown
+
+import (
+	"io"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark"
+	"golang.org/x/net/html"
+)
+
+// ConvertString converts a html-string to a markdown-string.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func ConvertString(htmlInput string, opts ...converter.ConvertOptionFunc) (string, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertString(htmlInput, opts...)
+}
+
+// ConvertReader converts the html from the reader to markdown.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func ConvertReader(r io.Reader, opts ...converter.ConvertOptionFunc) ([]byte, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertReader(r, opts...)
+}
+
+// ConvertNode converts a `*html.Node` to a markdown byte slice.
+//
+// If you have already parsed an HTML page using the `html.Parse()` function
+// from the "golang.org/x/net/html" package then you can pass this node
+// directly to the converter.
+func ConvertNode(doc *html.Node, opts ...converter.ConvertOptionFunc) ([]byte, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertNode(doc, opts...)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/convert.go

@@ -0,0 +1,142 @@
+package converter
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"io"
+	"slices"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+type convertOption struct {
+	domain  string
+	context context.Context
+}
+type ConvertOptionFunc func(o *convertOption)
+
+func WithContext(ctx context.Context) ConvertOptionFunc {
+	return func(o *convertOption) {
+		o.context = ctx
+	}
+}
+
+// WithDomain provides a base `domain` to the converter and
+// to the `AssembleAbsoluteURL` function.
+//
+// If a *relative* url is encountered (in an image or link) then the `domain` is used
+// to convert it to a *absolute* url.
+func WithDomain(domain string) ConvertOptionFunc {
+	return func(o *convertOption) {
+		o.domain = domain
+	}
+}
+
+func (conv *Converter) setError(err error) {
+	conv.m.Lock()
+	defer conv.m.Unlock()
+
+	conv.err = err
+}
+func (conv *Converter) getError() error {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	return conv.err
+}
+
+var errNoRenderHandlers = errors.New(`no render handlers are registered. did you forget to register the "commonmark" and "base" plugins?`)
+var errBasePluginMissing = errors.New(`you registered the "commonmark" plugin but the "base" plugin is also required`)
+
+// ConvertNode converts a `*html.Node` to a markdown byte slice.
+//
+// If you have already parsed an HTML page using the `html.Parse()` function
+// from the "golang.org/x/net/html" package then you can pass this node
+// directly to the converter.
+func (conv *Converter) ConvertNode(doc *html.Node, opts ...ConvertOptionFunc) ([]byte, error) {
+
+	if err := conv.getError(); err != nil {
+		// There can be errors while calling `Init` on the plugins (e.g. validation errors).
+		// Now is the first opportunity where we can return that error.
+		return nil, err
+	}
+
+	conv.m.Lock()
+	option := &convertOption{}
+	for _, fn := range opts {
+		fn(option)
+	}
+	conv.m.Unlock()
+
+	// If there are no render handlers registered this is
+	// usually a user error - since people want the Commonmark Plugin in 99% of cases.
+	if len(conv.getRenderHandlers()) == 0 {
+		return nil, errNoRenderHandlers
+	}
+
+	containsCommonmark := slices.Contains(conv.registeredPlugins, "commonmark")
+	containsBase := slices.Contains(conv.registeredPlugins, "base")
+	if containsCommonmark && !containsBase {
+		return nil, errBasePluginMissing
+	}
+
+	// - - - - - - - - - - - - - - - - - - - //
+
+	state := newGlobalState()
+
+	if option.context == nil {
+		option.context = context.Background()
+	}
+	ctx := option.context
+	ctx = provideDomain(ctx, option.domain)
+	ctx = provideAssembleAbsoluteURL(ctx, defaultAssembleAbsoluteURL)
+	ctx = state.provideGlobalState(ctx)
+
+	customCtx := newConverterContext(ctx, conv)
+
+	// - - - - - - - - - - - - - - - - - - - //
+
+	// Pre-Render
+	for _, handler := range conv.getPreRenderHandlers() {
+		handler.Value(customCtx, doc)
+	}
+
+	// Render
+	var buf bytes.Buffer
+	conv.handleRenderNode(customCtx, &buf, doc)
+
+	// Post-Render
+	result := buf.Bytes()
+	for _, handler := range conv.getPostRenderHandlers() {
+		result = handler.Value(customCtx, result)
+	}
+
+	return result, nil
+}
+
+// ConvertReader converts the html from the reader to markdown.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func (conv *Converter) ConvertReader(r io.Reader, opts ...ConvertOptionFunc) ([]byte, error) {
+	doc, err := html.Parse(r)
+	if err != nil {
+		return nil, err
+	}
+
+	return conv.ConvertNode(doc, opts...)
+}
+
+// ConvertString converts a html-string to a markdown-string.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func (conv *Converter) ConvertString(htmlInput string, opts ...ConvertOptionFunc) (string, error) {
+	r := strings.NewReader(htmlInput)
+	output, err := conv.ConvertReader(r, opts...)
+	if err != nil {
+		return "", err
+	}
+
+	return string(output), nil
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/converter.go

@@ -0,0 +1,72 @@
+package converter
+
+import "sync"
+
+type Converter struct {
+	m sync.RWMutex
+
+	err error
+
+	registeredPlugins []string
+
+	preRenderHandlers  prioritizedSlice[HandlePreRenderFunc]
+	renderHandlers     prioritizedSlice[HandleRenderFunc]
+	postRenderHandlers prioritizedSlice[HandlePostRenderFunc]
+
+	textTransformHandlers prioritizedSlice[HandleTextTransformFunc]
+
+	markdownChars    map[rune]interface{}
+	unEscapeHandlers prioritizedSlice[HandleUnEscapeFunc]
+
+	tagTypes map[string]prioritizedSlice[tagType]
+
+	escapeMode escapeMode
+
+	Register register
+}
+
+type converterOption = func(c *Converter) error
+
+func NewConverter(opts ...converterOption) *Converter {
+	conv := &Converter{
+		markdownChars: make(map[rune]interface{}),
+		tagTypes:      make(map[string]prioritizedSlice[tagType]),
+	}
+	conv.Register = register{conv}
+
+	for _, opt := range opts {
+		err := opt(conv)
+		if err != nil {
+			conv.setError(err)
+			break
+		}
+	}
+
+	return conv
+}
+
+type escapeMode string
+
+const (
+	EscapeModeDisabled escapeMode = "disabled"
+	EscapeModeSmart    escapeMode = "smart"
+)
+
+// WithEscapeMode changes the strictness of the "escaping".
+//
+// Some characters have a special meaning in markdown.
+// For example, the character "*" can be used for lists, emphasis and dividers.
+// By placing a backlash before that character (e.g. "\*") you can "escape" it.
+// Then the character will render as a raw "*" without the "markdown meaning" applied.
+//
+// Learn more in the documentation
+//
+//	"disabled" or "smart"
+//
+//	default: "smart"
+func WithEscapeMode(mode escapeMode) converterOption {
+	return func(c *Converter) error {
+		c.escapeMode = mode
+		return nil
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/ctx.go

@@ -0,0 +1,190 @@
+package converter
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+// func GetValue[K string, V any](ctx context.Context, key K) V {
+// 	val, _ := ctx.Value(key).(V)
+// 	return val
+// }
+// func SetValue[K string, V any](ctx context.Context, key K, val V) context.Context {
+// 	return context.WithValue(ctx, key, val)
+// }
+
+type ctxKey string
+
+const (
+	ctxKeyAssembleAbsoluteURL ctxKey = "AssembleAbsoluteURL"
+	ctxKeyDomain              ctxKey = "Domain"
+
+	ctxKeySetState    ctxKey = "SetState"
+	ctxKeyUpdateState ctxKey = "UpdateState"
+	ctxKeyGetState    ctxKey = "GetState"
+)
+
+func provideDomain(ctx context.Context, domain string) context.Context {
+	return context.WithValue(ctx, ctxKeyDomain, domain)
+}
+func GetDomain(ctx context.Context) string {
+	domain, ok := ctx.Value(ctxKeyDomain).(string)
+	if !ok {
+		fmt.Println("[warning] value ctxKeyDomain is different")
+		return ""
+	}
+
+	return domain
+}
+
+// - - - - - - - - - - - - - - - - - - - - - //
+
+type AssembleAbsoluteURLFunc func(tagName string, rawURL string, domain string) string
+
+func assembleAbsoluteURL(ctx context.Context, tagName string, rawURL string) string {
+	domain := GetDomain(ctx)
+
+	// TODO: since this gets passed down from the converter, it doesn't have to provided from the ctx anymore
+	fn, ok := ctx.Value(ctxKeyAssembleAbsoluteURL).(AssembleAbsoluteURLFunc)
+	if !ok {
+		fmt.Println("[warning] func ctxKeyAssembleAbsoluteURL is different")
+		return ""
+	}
+
+	return fn(tagName, rawURL, domain)
+}
+
+func provideAssembleAbsoluteURL(ctx context.Context, fn AssembleAbsoluteURLFunc) context.Context {
+	return context.WithValue(ctx, ctxKeyAssembleAbsoluteURL, fn)
+}
+
+// - - - - - - - - - - - - - - - - - - - - - //
+
+type SetStateFunc func(key string, val any)
+type UpdateStateFunc func(key string, fn func(any) any)
+type GetStateFunc func(key string) any
+
+type globalState struct {
+	data map[string]any
+}
+
+func newGlobalState() *globalState {
+
+	return &globalState{
+		data: make(map[string]any),
+	}
+}
+
+func (s *globalState) setState(key string, val any) {
+	s.data[key] = val
+}
+func (s *globalState) updateState(key string, fn func(any) any) {
+	s.data[key] = fn(s.data[key])
+}
+func (s *globalState) getState(key string) any {
+	return s.data[key]
+}
+
+func (s *globalState) provideGlobalState(ctx context.Context) context.Context {
+
+	var setState SetStateFunc = s.setState
+	var updateState UpdateStateFunc = s.updateState
+	var getState GetStateFunc = s.getState
+
+	ctx = context.WithValue(ctx, ctxKeySetState, setState)
+	ctx = context.WithValue(ctx, ctxKeyUpdateState, updateState)
+	ctx = context.WithValue(ctx, ctxKeyGetState, getState)
+
+	return ctx
+}
+
+func GetState[V any](ctx context.Context, key string) V {
+	fn := ctx.Value(ctxKeyGetState).(GetStateFunc)
+
+	val, _ := fn(key).(V)
+
+	return val
+}
+
+func SetState[V any](ctx context.Context, key string, val V) {
+	fn := ctx.Value(ctxKeySetState).(SetStateFunc)
+
+	fn(key, val)
+}
+
+func UpdateState[V any](ctx context.Context, key string, fn func(V) V) {
+	updater := ctx.Value(ctxKeyUpdateState).(UpdateStateFunc)
+
+	updater(key, func(val any) any {
+		value, ok := val.(V)
+		if !ok && val != nil {
+			// TODO: slog?
+			fmt.Println("[warning] val is different than V in UpdateState")
+		}
+
+		return fn(value)
+	})
+}
+
+// - - - - - - //
+
+// Context extends the normal context.Context with some additional
+// methods useful for the process of converting.
+type Context interface {
+	context.Context
+
+	AssembleAbsoluteURL(ctx Context, tagName string, rawURL string) string
+
+	GetTagType(tagName string) (tagType, bool)
+
+	RenderNodes(ctx Context, w Writer, nodes ...*html.Node)
+	RenderChildNodes(ctx Context, w Writer, n *html.Node)
+
+	EscapeContent(content []byte) []byte
+	UnEscapeContent(content []byte) []byte
+
+	WithValue(key any, val any) Context
+}
+
+type converterContext struct {
+	context.Context
+	conv *Converter
+}
+
+func newConverterContext(ctx context.Context, conv *Converter) Context {
+	return &converterContext{
+		Context: ctx,
+		conv:    conv,
+	}
+}
+
+func (c *converterContext) AssembleAbsoluteURL(ctx Context, tagName string, rawURL string) string {
+	return assembleAbsoluteURL(ctx, tagName, rawURL)
+}
+
+func (c *converterContext) RenderNodes(ctx Context, w Writer, nodes ...*html.Node) {
+	c.conv.handleRenderNodes(ctx, w, nodes...)
+}
+func (c *converterContext) RenderChildNodes(ctx Context, w Writer, n *html.Node) {
+	c.conv.handleRenderNodes(ctx, w, dom.AllChildNodes(n)...)
+}
+
+func (c *converterContext) GetTagType(tagName string) (tagType, bool) {
+	return c.conv.getTagType(tagName)
+}
+func (c *converterContext) EscapeContent(content []byte) []byte {
+	return c.conv.escapeContent(content)
+}
+func (c *converterContext) UnEscapeContent(content []byte) []byte {
+	return c.conv.unEscapeContent(content)
+}
+
+func (c *converterContext) WithValue(key any, val any) Context {
+	return &converterContext{
+		Context: context.WithValue(c.Context, key, val),
+		conv:    c.conv,
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/escape.go

@@ -0,0 +1,89 @@
+package converter
+
+import (
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+)
+
+const (
+	actionKeep   = iota
+	actionEscape = iota
+)
+
+// IMPORTANT: Only internally we assume it is only byte
+var placeholderByte byte = marker.BytesMarkerEscaping[0]
+
+func (conv *Converter) escapeContent(chars []byte) []byte {
+	if conv.escapeMode == EscapeModeDisabled {
+		return chars
+	}
+
+	newChars := make([]byte, 0, len(chars))
+	for index := 0; index < len(chars); index++ {
+		if chars[index] == '\u0000' {
+			// For security reasons, the Unicode character U+0000 must be replaced with the REPLACEMENT CHARACTER (U+FFFD).
+			newChars = append(newChars, []byte(string('\ufffd'))...)
+			continue
+		}
+
+		r, _ := utf8.DecodeRune(chars[index:])
+
+		isMarkdownChar := conv.checkIsEscapedChar(r)
+		if isMarkdownChar {
+			newChars = append(newChars, placeholderByte, chars[index])
+		} else {
+			newChars = append(newChars, chars[index])
+		}
+	}
+
+	return newChars
+}
+
+func (conv *Converter) unEscapeContent(chars []byte) []byte {
+	if conv.escapeMode == EscapeModeDisabled {
+		return chars
+	}
+
+	checkElements := func(index int) int {
+		for _, handler := range conv.getUnEscapeHandlers() {
+			if skip := handler.Value(chars, index); skip != -1 {
+				return skip
+			}
+		}
+
+		return -1
+	}
+
+	changes := make([]uint8, len(chars))
+	for index := 0; index < len(chars); index++ {
+
+		if chars[index] != placeholderByte {
+			continue
+		}
+		if index+1 >= len(chars) {
+			break
+		}
+
+		skip := checkElements(index + 1)
+		if skip == -1 {
+			continue
+		}
+		changes[index] = actionEscape
+		index += skip - 1
+	}
+
+	newChars := make([]byte, 0, len(chars))
+	for index, char := range chars {
+		if char != placeholderByte {
+			newChars = append(newChars, char)
+			continue
+		}
+
+		// What to do with this placeholder? Should we escape or not?
+		if changes[index] == actionEscape {
+			newChars = append(newChars, '\\')
+		}
+	}
+	return newChars
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/plugin.go

@@ -0,0 +1,22 @@
+package converter
+
+// Plugin can be used to extends functionality beyond what
+// is offered by commonmark.
+type Plugin interface {
+	// The public name of the plugin, e.g. "strikethrough"
+	Name() string
+
+	// Init is called to initialize the plugin. It can be used to
+	// *validate* the arguments and *register* the rules.
+	Init(conv *Converter) error
+}
+
+// WithPlugins can be used to add additional functionality to the converter.
+func WithPlugins(plugins ...Plugin) converterOption {
+	return func(c *Converter) error {
+		for _, plugin := range plugins {
+			c.Register.Plugin(plugin)
+		}
+		return nil
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/prioritized.go

@@ -0,0 +1,33 @@
+package converter
+
+import "sort"
+
+const (
+	// PriorityEarly means that the handler will be run **early** in the process.
+	// To run it even earlier you need to subtract from this number.
+	PriorityEarly = 100
+
+	// PriorityStandard is for handlers that don't need to be run in a particular order.
+	PriorityStandard = 500
+
+	// PriorityLate means that the handler will be run **late** in the process.
+	// To run it even later you need to add to this number.
+	PriorityLate = 1000
+)
+
+type prioritizedValue[V any] struct {
+	Value    V
+	Priority int
+}
+
+type prioritizedSlice[V any] []prioritizedValue[V]
+
+func (s prioritizedSlice[V]) Sort() {
+	sort.Slice(s, func(i, j int) bool {
+		return s[i].Priority < s[j].Priority
+	})
+}
+
+func prioritized[V any](v V, priority int) prioritizedValue[V] {
+	return prioritizedValue[V]{v, priority}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/register.go

@@ -0,0 +1,222 @@
+package converter
+
+import (
+	"errors"
+	"fmt"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+type register struct {
+	conv *Converter
+}
+
+func (r *register) Plugin(plugin Plugin) {
+	pluginName := plugin.Name()
+	if pluginName == "" {
+		r.conv.setError(errors.New("the plugin has no name"))
+		return
+	}
+
+	r.conv.m.Lock()
+	r.conv.registeredPlugins = append(r.conv.registeredPlugins, pluginName)
+	r.conv.m.Unlock()
+
+	err := plugin.Init(r.conv)
+	if err != nil {
+		r.conv.setError(fmt.Errorf("error while initializing %q plugin: %w", pluginName, err))
+		return
+	}
+}
+
+// - - - - - - - - - - - - - Pre-Render - - - - - - - - - - - - - //
+
+type HandlePreRenderFunc func(ctx Context, doc *html.Node)
+
+func (r *register) PreRenderer(fn HandlePreRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.preRenderHandlers = append(r.conv.preRenderHandlers, handler)
+}
+func (conv *Converter) getPreRenderHandlers() prioritizedSlice[HandlePreRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandlePreRenderFunc], len(conv.preRenderHandlers))
+	copy(handlers, conv.preRenderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Render - - - - - - - - - - - - - //
+
+// Writer is an interface that only conforms to the Write* methods of bytes.Buffer
+type Writer interface {
+	Write(p []byte) (n int, err error)
+	WriteByte(c byte) error
+	WriteRune(r rune) (n int, err error)
+	WriteString(s string) (n int, err error)
+}
+
+type HandleRenderFunc func(ctx Context, w Writer, n *html.Node) RenderStatus
+
+func (r *register) Renderer(fn HandleRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.renderHandlers = append(r.conv.renderHandlers, handler)
+}
+
+// RendererFor registers a renderer for a specific tag (e.g. "div").
+// It is a small wrapper around `TagType()` and `Renderer()`.
+func (r *register) RendererFor(tagName string, tagType tagType, renderFn HandleRenderFunc, priority int) {
+
+	// 1. we add the "tagType" to the map
+	r.TagType(tagName, tagType, priority)
+
+	// 2. we register the render function
+	r.Renderer(func(ctx Context, w Writer, n *html.Node) RenderStatus {
+		if dom.NodeName(n) == tagName {
+			return renderFn(ctx, w, n)
+		}
+		return RenderTryNext
+	}, priority)
+}
+
+func (conv *Converter) getRenderHandlers() prioritizedSlice[HandleRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleRenderFunc], len(conv.renderHandlers))
+	copy(handlers, conv.renderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Post Render - - - - - - - - - - - - - //
+
+type HandlePostRenderFunc func(ctx Context, content []byte) []byte
+
+func (r *register) PostRenderer(fn HandlePostRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.postRenderHandlers = append(r.conv.postRenderHandlers, handler)
+}
+func (conv *Converter) getPostRenderHandlers() prioritizedSlice[HandlePostRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandlePostRenderFunc], len(conv.postRenderHandlers))
+	copy(handlers, conv.postRenderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Text - - - - - - - - - - - - - //
+
+type HandleTextTransformFunc func(ctx Context, content string) string
+
+func (r *register) TextTransformer(fn HandleTextTransformFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.textTransformHandlers = append(r.conv.textTransformHandlers, handler)
+}
+func (conv *Converter) getTextTransformHandlers() prioritizedSlice[HandleTextTransformFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleTextTransformFunc], len(conv.textTransformHandlers))
+	copy(handlers, conv.textTransformHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Escaping - - - - - - - - - - - - - //
+
+func (r *register) EscapedChar(chars ...rune) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	for _, char := range chars {
+		r.conv.markdownChars[char] = struct{}{}
+	}
+}
+func (conv *Converter) checkIsEscapedChar(r rune) bool {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	_, ok := conv.markdownChars[r]
+	return ok
+}
+
+type HandleUnEscapeFunc func(chars []byte, index int) int
+
+func (r *register) UnEscaper(fn HandleUnEscapeFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.unEscapeHandlers = append(r.conv.unEscapeHandlers, handler)
+}
+func (conv *Converter) getUnEscapeHandlers() prioritizedSlice[HandleUnEscapeFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleUnEscapeFunc], len(conv.unEscapeHandlers))
+	copy(handlers, conv.unEscapeHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Tag Type - - - - - - - - - - - - - //
+
+type tagType string
+
+const (
+	TagTypeBlock  tagType = "block"
+	TagTypeInline tagType = "inline"
+
+	// TagTypeRemove will remove that node in the _PreRender_ phase with a high priority.
+	TagTypeRemove tagType = "remove"
+)
+
+func (r *register) TagType(tagName string, tagType tagType, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	val := prioritized(tagType, priority)
+	r.conv.tagTypes[tagName] = append(r.conv.tagTypes[tagName], val)
+}
+func (conv *Converter) getTagType(tagName string) (tagType, bool) {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	types, ok := conv.tagTypes[tagName]
+	if !ok || len(types) == 0 {
+
+		if dom.NameIsBlockNode(tagName) {
+			return TagTypeBlock, true
+		} else if dom.NameIsInlineNode(tagName) {
+			return TagTypeInline, true
+		}
+		return "", false
+	}
+
+	types.Sort()
+	firstType := types[0].Value
+
+	return firstType, true
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/render.go

@@ -0,0 +1,61 @@
+package converter
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func (conv *Converter) handleRenderNodes(ctx Context, w Writer, nodes ...*html.Node) {
+	for _, node := range nodes {
+		conv.handleRenderNode(ctx, w, node)
+	}
+}
+
+func (conv *Converter) handleRenderNode(ctx Context, w Writer, node *html.Node) RenderStatus {
+	name := dom.NodeName(node)
+
+	// - - A: the #text node - - //
+	if name == "#text" {
+		return conv.handleRenderText(ctx, w, node)
+	}
+
+	// - - B: the render handlers - - //
+	for _, handler := range conv.getRenderHandlers() {
+		status := handler.Value(ctx, w, node)
+		if status == RenderSuccess {
+			return status
+		}
+	}
+
+	// - - C: the fallback - - //
+	// If nothing works we fallback to this:
+	return conv.handleRenderFallback(ctx, w, node)
+}
+
+func (conv *Converter) handleRenderFallback(ctx Context, w Writer, node *html.Node) RenderStatus {
+
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == TagTypeBlock {
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+	}
+	ctx.RenderChildNodes(ctx, w, node)
+	if tagType == TagTypeBlock {
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+	}
+
+	return RenderSuccess
+}
+func (conv *Converter) handleRenderText(ctx Context, w Writer, node *html.Node) RenderStatus {
+	content := node.Data
+
+	for _, handler := range conv.getTextTransformHandlers() {
+		content = handler.Value(ctx, content)
+	}
+
+	w.WriteString(content)
+	return RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/status.go

@@ -0,0 +1,8 @@
+package converter
+
+type RenderStatus int
+
+const (
+	RenderTryNext RenderStatus = iota
+	RenderSuccess
+)

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/url.go

@@ -0,0 +1,114 @@
+package converter
+
+import (
+	"net/url"
+	"strings"
+)
+
+var percentEncodingReplacer = strings.NewReplacer(
+	" ", "%20",
+	"[", "%5B",
+	"]", "%5D",
+	"(", "%28",
+	")", "%29",
+	"<", "%3C",
+	">", "%3E",
+)
+
+func parseBaseDomain(rawDomain string) *url.URL {
+	if rawDomain == "" {
+		return nil
+	}
+
+	u1, err := url.Parse(rawDomain)
+	if err == nil && u1.Host != "" {
+		// Yes, we got valid domain (probably with a http/https scheme)
+		return u1
+	}
+
+	u2, err := url.Parse("http://" + rawDomain)
+	if err == nil && u2.Host != "" {
+		// Yes, we got a valid domain (by choosing a fallback scheme)
+		return u2
+	}
+
+	return nil
+}
+func defaultAssembleAbsoluteURL(tagName string, rawURL string, domain string) string {
+	rawURL = strings.TrimSpace(rawURL)
+
+	if rawURL == "#" {
+		// Golangs url.Parse does not seem to distinguish between
+		// no fragment and an empty fragment.
+		return rawURL
+	}
+
+	// Increase the chance that the url will be parsed
+	rawURL = strings.ReplaceAll(rawURL, "\n", "%0A")
+	rawURL = strings.ReplaceAll(rawURL, "\t", "%09")
+
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		// We can't do anything with this url because it is invalid
+		return percentEncodingReplacer.Replace(rawURL)
+	}
+
+	if u.Scheme == "data" {
+		// This is a data uri (for example an inline base64 image)
+		return percentEncodingReplacer.Replace(rawURL)
+	}
+
+	// The default Query().Encode() encodes the query parameters "sorted by key".
+	// Instead we want to keep the original order, but still encode the parameters.
+	u.RawQuery = ParseAndEncodeQuery(u.RawQuery)
+
+	// For better compatibility (especially in regards to mailto links),
+	// instead of encoding a space with a "+" we use ""%20" to prevent
+	// e.g. the email reading "Hi+Johannes" instead of "Hi Johannes"
+	u.RawQuery = strings.ReplaceAll(u.RawQuery, "+", "%20")
+
+	if base := parseBaseDomain(domain); base != nil {
+		// If a "domain" is provided, we use that to convert relative links
+		// to absolute links.
+		u = base.ResolveReference(u)
+	}
+
+	return percentEncodingReplacer.Replace(u.String())
+}
+
+// - - - - //
+
+func decodeAndEncode(original string) string {
+	s, err := url.QueryUnescape(original)
+	if err != nil {
+		return original
+	}
+
+	return url.QueryEscape(s)
+}
+
+func ParseAndEncodeQuery(rawQuery string) string {
+	if rawQuery == "" {
+		return ""
+	}
+
+	rawParts := strings.Split(rawQuery, "&")
+	encodedParts := make([]string, len(rawParts))
+
+	for i, part := range rawParts {
+		splitted := strings.SplitN(part, "=", 2)
+
+		if len(splitted) == 1 {
+			// A: Just the key
+			encodedParts[i] = decodeAndEncode(splitted[0])
+		} else if splitted[1] == "" {
+			// B: The key and the equal sign
+			encodedParts[i] = decodeAndEncode(splitted[0]) + "="
+		} else {
+			// C: The key and the equal sign and the value
+			encodedParts[i] = decodeAndEncode(splitted[0]) + "=" + decodeAndEncode(splitted[1])
+		}
+	}
+
+	return strings.Join(encodedParts, "&")
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/add_space.go

@@ -0,0 +1,66 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func getFirstChildNode(startNode *html.Node, matchFn func(n *html.Node) bool) *html.Node {
+	node := startNode.FirstChild
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+		} else if matchFn(node) {
+			return node
+		} else {
+			return nil
+		}
+	}
+
+	return nil
+}
+
+func getLastChildNode(startNode *html.Node, matchFn func(n *html.Node) bool) *html.Node {
+	node := startNode.LastChild
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetPrevNeighborNode(node)
+		} else if matchFn(node) {
+			return node
+		} else {
+			return nil
+		}
+	}
+
+	return nil
+}
+func AddSpace(ctx context.Context, doc *html.Node, isOuterNode, isInnerNode func(*html.Node) bool) {
+	node := doc
+	for node != nil {
+		if isOuterNode(node) {
+			firstChild := getFirstChildNode(node, isInnerNode)
+			if firstChild != nil {
+				prev := getPrevTextNode(node)
+				if prev != nil {
+					prev.Data = prev.Data + " "
+				}
+			}
+
+			lastChild := getLastChildNode(node, isInnerNode)
+			if lastChild != nil {
+				next := getNextTextNode(node)
+				if next != nil {
+					next.Data = " " + next.Data
+				}
+			}
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/adjacent.go

@@ -0,0 +1,75 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func collectAdjacentNodes(node *html.Node, matchFn func(n *html.Node) bool) []*html.Node {
+	var collectedNodes []*html.Node
+
+	node = node.NextSibling
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+		} else if matchFn(node) {
+			collectedNodes = append(collectedNodes, node)
+			node = dom.GetNextNeighborNodeExcludingOwnChild(node)
+		} else {
+			// Return the collected nodes
+			return collectedNodes
+		}
+	}
+
+	return collectedNodes
+}
+
+func mergeChildren(destinationNode *html.Node, nodes ...*html.Node) {
+	for _, node := range nodes {
+		// We move all the children to the `destinationNode`.
+		children := dom.AllChildNodes(node)
+		for _, child := range children {
+			dom.RemoveNode(child)
+			destinationNode.AppendChild(child)
+		}
+		dom.RemoveNode(node)
+	}
+}
+
+func MergeAdjacent(doc *html.Node, matchFn func(*html.Node) bool) {
+	node := doc
+
+	for node != nil {
+		if matchFn(node) {
+			nextNodes := collectAdjacentNodes(node, matchFn)
+
+			mergeChildren(node, nextNodes...)
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}
+
+// - - - - - - - - //
+
+func MergeAdjacentTextNodes(n *html.Node) {
+	if n == nil {
+		return
+	}
+
+	var prev *html.Node
+	for c := n.FirstChild; c != nil; {
+		next := c.NextSibling
+		if c.Type == html.TextNode && prev != nil && prev.Type == html.TextNode {
+			// Combine adjacent text nodes
+			prev.Data += c.Data
+			n.RemoveChild(c)
+		} else {
+			MergeAdjacentTextNodes(c)
+			prev = c
+		}
+		c = next
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/alternatives.go

@@ -0,0 +1,114 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+// TODO: make this configurable via the options???
+func getMarkdownStructure(name string) string {
+	switch name {
+	case "#document", "html", "head", "body",
+		"blockquote", "ul", "ol", "li":
+		// A container block can also contain other blocks.
+		return "container_block"
+
+	// Note: "p" would also be part of "leaf_block"
+	case "hr", "pre",
+		"h1", "h2", "h3", "h4", "h5", "h6":
+		// Leaf blocks can contain inline content
+		// but NOT other blocks.
+		return "leaf_block"
+
+	case "#text", "span", "code",
+		"b", "strong", "i", "em",
+		"a", "img", "br":
+		return "inline"
+
+	case "div", "p":
+		// Since these are just placing newlines,
+		// we dont categorize them.
+		return ""
+
+	default:
+		return ""
+	}
+}
+
+func headingAlternative(ctx context.Context, node *html.Node) {
+	node.Data = "strong"
+
+	newChild := &html.Node{
+		Type: html.ElementNode,
+		Data: "br",
+	}
+	node.Parent.InsertBefore(newChild, node.NextSibling)
+}
+func blockquoteAlternative(ctx context.Context, node *html.Node) {
+	newBefore := &html.Node{Type: html.TextNode, Data: ` "`}
+	node.Parent.InsertBefore(newBefore, node)
+
+	node.Data = "span"
+
+	newAfter := &html.Node{Type: html.TextNode, Data: `" `}
+	node.Parent.InsertBefore(newAfter, node.NextSibling)
+}
+func preAlternative(ctx context.Context, node *html.Node) {
+	node.Data = "code"
+}
+func hrAlternative(ctx context.Context, node *html.Node) {
+	dom.RemoveNode(node)
+}
+
+// TODO: make this configurable via the options?
+var alternatives = map[string]func(ctx context.Context, node *html.Node){
+	"h1":         headingAlternative,
+	"h2":         headingAlternative,
+	"h3":         headingAlternative,
+	"h4":         headingAlternative,
+	"h5":         headingAlternative,
+	"h6":         headingAlternative,
+	"blockquote": blockquoteAlternative,
+	"pre":        preAlternative,
+	"hr":         hrAlternative,
+}
+
+func LeafBlockAlternatives(ctx context.Context, doc *html.Node) {
+	var finder func(node *html.Node, isInsideLeafBlock bool, isInsideInline bool)
+	finder = func(node *html.Node, isInsideLeafBlock bool, isInsideInline bool) {
+		name := dom.NodeName(node)
+
+		structure := getMarkdownStructure(name)
+		if (structure == "container_block" || structure == "leaf_block") && (isInsideLeafBlock || isInsideInline) {
+			// A block inside an inline OR a block inside a leaf-block
+			// is not valid markdown so cannot be rendered.
+			//
+			// For example, you cannot place a blockquote inside a heading.
+			//
+			// Instead of this weird output (## Heading > My Quote)
+			// we try to find alternatives (## Heading "My Quote")
+			fn, ok := alternatives[name]
+			if ok {
+				fn(ctx, node)
+			} else {
+				node.Data = "span"
+			}
+		}
+
+		// - - - - - - - - - - - - - - - - - - - - - - //
+
+		if structure == "leaf_block" {
+			isInsideLeafBlock = true
+		}
+		if structure == "inline" {
+			isInsideInline = true
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			defer finder(child, isInsideLeafBlock, isInsideInline)
+		}
+	}
+	finder(doc, false, false)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/domutils.go

@@ -0,0 +1,43 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func getNextTextNode(startNode *html.Node) *html.Node {
+	node := dom.GetNextNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		if node.Type == html.TextNode {
+			return node
+		}
+
+		if dom.NodeName(node) == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+			continue
+		}
+
+		return nil
+	}
+	return nil
+}
+func getPrevTextNode(startNode *html.Node) *html.Node {
+	node := dom.GetPrevNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		if node.Type == html.TextNode {
+			return node
+		}
+
+		if dom.NodeName(node) == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetPrevNeighborNode(node)
+			continue
+		}
+
+		return nil
+	}
+	return nil
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/empty_code.go

@@ -0,0 +1,42 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func hasTextChildNodes(startNode *html.Node) bool {
+	var found bool
+
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		if node.Type == html.TextNode && node.Data != "" {
+			found = true
+			return
+		}
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(startNode)
+
+	return found
+}
+
+func RemoveEmptyCode(ctx context.Context, doc *html.Node) {
+	node := doc
+	for node != nil {
+		if dom.NodeName(node) == "code" && !hasTextChildNodes(node) {
+			next := dom.GetNextNeighborNodeExcludingOwnChild(node)
+
+			dom.RemoveNode(node)
+
+			node = next
+			continue
+		}
+
+		node = dom.GetNextNeighborNode(node)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/list_end_comment.go

@@ -0,0 +1,74 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+var ListEndCommentData = "THE END"
+
+func AddListEndComments(ctx context.Context, doc *html.Node) {
+	node := doc
+	for node != nil {
+		if nameIsList(node) && nextNameIsList(node) {
+			insertComment(node)
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}
+
+func nameIsList(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "ul" || name == "ol"
+}
+
+func insertComment(listNode *html.Node) {
+	comment := &html.Node{
+		Type: html.CommentNode,
+		Data: ListEndCommentData,
+	}
+	listNode.Parent.InsertBefore(comment, listNode.NextSibling)
+}
+
+func nextNameIsList(startNode *html.Node) bool {
+	node := dom.GetNextNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "ul" || name == "ol" {
+			return true
+		}
+		if name == "li" {
+			return false
+		}
+		if name == "#comment" && node.Data == ListEndCommentData {
+			return false
+		}
+
+		// If there is any text between two lists
+		// they are automatically not connected anymore.
+		if node.Type == html.TextNode {
+			return false
+		}
+
+		// - - - - //
+
+		if name == "hr" {
+			// A divider already seperates two lists...
+			return false
+		}
+
+		// TODO: RunContext.Render()
+		// -> get acess to keepRemoveMap
+
+		// TODO: look in the KeepRemoveMap?
+		// e.g. ul then script then ul
+
+		node = dom.GetNextNeighborNode(node)
+		continue
+	}
+	return false
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/list_items.go

@@ -0,0 +1,53 @@
+package domutils
+
+import (
+	"context"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+	"golang.org/x/net/html/atom"
+)
+
+// MoveListItems moves non-"li" nodes into the previous "li" nodes.
+func MoveListItems(ctx context.Context, n *html.Node) {
+	if n.Type == html.ElementNode && (n.Data == "ol" || n.Data == "ul") {
+		var previousLi *html.Node
+
+		// Collect children to avoid modifying the slice while iterating.
+		children := dom.AllChildNodes(n)
+
+		for _, child := range children {
+			if child.Type == html.ElementNode && child.Data == "li" {
+				previousLi = child
+			} else if child.Type == html.TextNode && strings.TrimSpace(child.Data) == "" {
+				// Skip the node, probably just formatting of code
+			} else {
+				// We expect that inside an "ol"/"ul" there are *only* "li" nodes.
+				// But sometimes that is not the case...
+
+				if previousLi != nil {
+					// There is a previous "li" node,
+					// so we move this content into the other "li" node.
+					n.RemoveChild(child)
+
+					previousLi.AppendChild(child)
+				} else {
+					// There is no previous "li" node,
+					// so we wrap this node with it's own "li" node.
+
+					newNode := &html.Node{
+						Type:     html.ElementNode,
+						DataAtom: atom.Li,
+						Data:     "li",
+					}
+					previousLi = dom.WrapNode(child, newNode)
+				}
+			}
+		}
+	}
+
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		MoveListItems(ctx, c)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/redundant.go

@@ -0,0 +1,29 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func RemoveRedundant(doc *html.Node, matchFn func(*html.Node, *html.Node) bool) {
+	for _, node := range dom.AllNodes(doc) {
+		if hasSameTypeAncestor(node, matchFn) {
+			dom.UnwrapNode(node)
+		}
+	}
+
+}
+
+func hasSameTypeAncestor(n *html.Node, matchFn func(*html.Node, *html.Node) bool) bool {
+	if !matchFn(n, n) {
+		return false
+	}
+
+	for p := n.Parent; p != nil; p = p.Parent {
+		if matchFn(n, p) {
+			return true
+		}
+	}
+
+	return false
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/span.go

@@ -0,0 +1,50 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func isFakeSpan(node *html.Node) bool {
+	name := dom.NodeName(node)
+	if name != "span" {
+		return false
+	}
+
+	var containsBlockNode = false
+
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		name := dom.NodeName(node)
+		if dom.NameIsBlockNode(name) {
+			containsBlockNode = true
+			return
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(node)
+
+	return containsBlockNode
+}
+
+// RenameFakeSpans renames all "span" nodes to "div" if
+// any block element is found as a child.
+func RenameFakeSpans(ctx context.Context, doc *html.Node) {
+	var finder func(node *html.Node)
+	finder = func(node *html.Node) {
+		if isFakeSpan(node) {
+			node.Data = "div"
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/swap.go

@@ -0,0 +1,51 @@
+package domutils
+
+import (
+	"context"
+	"slices"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func swapTagsOfNodes(node1, node2 *html.Node) {
+	if node1.Type != html.ElementNode || node2.Type != html.ElementNode {
+		panic("swap only works with element nodes")
+	}
+
+	tempDataAtom := node1.DataAtom
+	tempData := node1.Data
+	tempAttr := node1.Attr
+
+	node1.DataAtom = node2.DataAtom
+	node1.Data = node2.Data
+	node1.Attr = node2.Attr
+
+	node2.DataAtom = tempDataAtom
+	node2.Data = tempData
+	node2.Attr = tempAttr
+}
+
+func isEmptyText(node *html.Node) bool {
+	return node.Type == html.TextNode && strings.TrimSpace(node.Data) == ""
+}
+func SwapTags(ctx context.Context, doc *html.Node, isOuterNode, isInnerNode func(*html.Node) bool) {
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		if isOuterNode(node) {
+			childs := dom.AllChildNodes(node)
+			childs = slices.DeleteFunc(childs, isEmptyText)
+
+			if len(childs) == 1 && isInnerNode(childs[0]) {
+				swapTagsOfNodes(node, childs[0])
+				return
+			}
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_backslash.go

@@ -0,0 +1,9 @@
+package escape
+
+func IsBackslash(chars []byte, index int) int {
+	if chars[index] != '\\' {
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_code.go

@@ -0,0 +1,45 @@
+package escape
+
+func IsFencedCode(chars []byte, index int) int {
+	if chars[index] != '`' && chars[index] != '~' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == ' ' || chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == '\n' {
+			break
+		}
+
+		return -1
+	}
+
+	count := 1
+	i := index + 1
+	for ; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == '`' || chars[i] == '~' {
+			count++
+			continue
+		}
+
+		break
+	}
+	if count < 3 {
+		return -1
+	}
+
+	return i - index
+}
+
+func IsInlineCode(chars []byte, index int) int {
+	if chars[index] != '`' {
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_divider.go

@@ -0,0 +1,47 @@
+package escape
+
+func IsDivider(chars []byte, index int) int {
+	if chars[index] != '-' && chars[index] != '_' && chars[index] != '*' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+
+		return -1
+	}
+
+	count := 1
+	lastChar := len(chars)
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == chars[index] {
+			count++
+			continue
+		}
+		if chars[i] == '\n' {
+			lastChar = i
+			break
+		}
+
+		return -1
+	}
+
+	if count >= 3 {
+		return lastChar - index
+	}
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_header.go

@@ -0,0 +1,77 @@
+package escape
+
+func IsAtxHeader(chars []byte, index int) int {
+	if chars[index] != '#' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		return -1
+	}
+
+	nPoundSigns := 1
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == '#' {
+			nPoundSigns++
+
+			if nPoundSigns > 6 {
+				return -1
+			}
+			continue
+		}
+
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' || chars[i] == '\t' || chars[i] == '\n' || chars[i] == '\r' {
+			// TODO: fix calculation with placeholder (maybe own for loop construct?)
+			// Returns the count of # that we encountered
+			return i - index
+		}
+
+		return -1
+	}
+	return 1
+}
+
+func IsSetextHeader(chars []byte, index int) int {
+	if chars[index] != '=' && chars[index] != '-' {
+		return -1
+	}
+
+	var newlineCount int
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte || chars[i] == ' ' {
+			continue
+		}
+
+		if chars[i] == '\n' {
+			newlineCount++
+			continue
+		}
+
+		if newlineCount == 0 {
+			// Without any newlines, this character is on the same line
+			// as the delimiter. So the delimiter is inside a normal text...
+			return -1
+		} else if newlineCount == 1 {
+			// The heading content is on the line above the delimiter
+			// which qualifies for a setext heading...
+			return 1
+		} else {
+			return -1
+		}
+
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_image.go

@@ -0,0 +1,36 @@
+package escape
+
+func IsImageOrLink(chars []byte, index int) int {
+	if chars[index] == '!' {
+		return isImageOrLinkStartExclamation(chars, index)
+	}
+	if chars[index] == '[' {
+		return isImageOrLinkStartBracket(chars, index)
+	}
+
+	return -1
+}
+
+func isImageOrLinkStartExclamation(chars []byte, index int) int {
+	nextIndex := index + 1
+	if nextIndex < len(chars) && chars[nextIndex] == '[' {
+		// It could be the start of an image
+		return 1
+	}
+
+	return -1
+}
+
+func isImageOrLinkStartBracket(chars []byte, index int) int {
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == '\n' {
+			return -1
+		}
+
+		if chars[i] == ']' {
+			return 1
+		}
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_italic_bold.go

@@ -0,0 +1,21 @@
+package escape
+
+import (
+	"unicode"
+)
+
+func IsItalicOrBold(chars []byte, index int) int {
+	if chars[index] != '*' && chars[index] != '_' {
+		return -1
+	}
+
+	next := getNextAsRune(chars, index)
+
+	nextIsWhitespace := unicode.IsSpace(next) || next == 0
+	if nextIsWhitespace {
+		// "not followed by Unicode whitespace"
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_list.go

@@ -0,0 +1,67 @@
+package escape
+
+import (
+	"unicode"
+)
+
+func IsUnorderedList(chars []byte, index int) int {
+	if chars[index] != '-' && chars[index] != '*' && chars[index] != '+' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return -1
+	}
+
+	next := getNext(chars, index)
+	if IsSpace(next) || next == 0 {
+		return 1
+	}
+
+	return -1
+}
+
+func IsOrderedList(chars []byte, index int) int {
+	if chars[index] != '.' && chars[index] != ')' {
+		return -1
+	}
+
+	// Directly before the dot needs to be a digit
+	prev := getPrevAsRune(chars, index)
+	if !unicode.IsDigit(prev) {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if IsDigit(chars[i]) {
+			continue
+		}
+
+		return -1
+	}
+
+	next := getNext(chars, index)
+	if IsSpace(next) || next == 0 {
+		return 1
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_quote.go

@@ -0,0 +1,23 @@
+package escape
+
+func IsBlockQuote(chars []byte, index int) int {
+	if chars[index] != '>' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/replacer.go

@@ -0,0 +1,8 @@
+package escape
+
+import "github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+
+var placeholderRune rune = marker.MarkerEscaping
+
+// IMPORTANT: Only internally we assume it is only byte
+var placeholderByte byte = marker.BytesMarkerEscaping[0]

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/util.go

@@ -0,0 +1,70 @@
+package escape
+
+import "unicode/utf8"
+
+// TODO: move to markers package?
+
+func IsSpace(b byte) bool {
+	switch b {
+	case '\t', '\n', '\v', '\f', '\r', ' ', 0x85, 0xA0:
+		return true
+	}
+	return false
+}
+
+func IsDigit(b byte) bool {
+	switch b {
+	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
+		return true
+	}
+	return false
+}
+
+func getPrev(chars []byte, index int) byte {
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return chars[i]
+	}
+	return 0
+}
+
+func getNext(chars []byte, index int) byte {
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return chars[i]
+	}
+	return 0
+}
+
+func getPrevAsRune(chars []byte, index int) rune {
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte {
+			continue
+		}
+
+		r, _ := utf8.DecodeLastRune(chars[:i+1])
+
+		return r
+	}
+	return 0
+}
+func getNextAsRune(source []byte, index int) rune {
+	for i := index + 1; i < len(source); i++ {
+		if source[i] == placeholderByte {
+			continue
+		}
+
+		r, _ := utf8.DecodeRune(source[i:])
+		return r
+	}
+	return 0
+}
+
+// TODO: make public?
+func GetNextAsRune(source []byte, index int) rune {
+	return getNextAsRune(source, index)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/codefence.go

@@ -0,0 +1,61 @@
+package textutils
+
+import "strings"
+
+func CalculateCodeFenceOccurrences(fenceChar rune, content string) int {
+	var occurrences []int
+
+	var charsTogether int
+	for _, char := range content {
+		// We encountered a fence character, now count how many
+		// are directly afterwards
+		if char == fenceChar {
+			charsTogether++
+		} else if charsTogether != 0 {
+			occurrences = append(occurrences, charsTogether)
+			charsTogether = 0
+		}
+	}
+
+	// If the last element in the content was a fenceChar
+	if charsTogether != 0 {
+		occurrences = append(occurrences, charsTogether)
+	}
+
+	return findMax(occurrences)
+}
+
+// CalculateCodeFence can be passed the content of a code block and it returns
+// how many fence characters (` or ~) should be used.
+//
+// This is useful if the html content includes the same fence characters
+// for example ```
+// -> https://stackoverflow.com/a/49268657
+func CalculateCodeFence(fenceChar rune, content string) string {
+	repeat := CalculateCodeFenceOccurrences(fenceChar, content)
+
+	// The outer fence block always has to have
+	// at least one character more than any content inside
+	repeat++
+
+	// You have to have at least three fence characters
+	// to be recognized as a code block
+	if repeat < 3 {
+		repeat = 3
+	}
+
+	return strings.Repeat(string(fenceChar), repeat)
+}
+
+func findMax(a []int) (max int) {
+	for i, value := range a {
+		if i == 0 {
+			max = a[i]
+		}
+
+		if value > max {
+			max = value
+		}
+	}
+	return max
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/collapse_code.go

@@ -0,0 +1,31 @@
+package textutils
+
+import (
+	"bytes"
+)
+
+func CollapseInlineCodeContent(content []byte) []byte {
+	// TODO: what about other characters like the reset char? Maybe unicode.IsSpace?
+	content = bytes.ReplaceAll(content, []byte{'\n'}, []byte{' '})
+	content = bytes.ReplaceAll(content, []byte{'\t'}, []byte{' '})
+
+	content = bytes.TrimSpace(content)
+
+	newChars := make([]byte, 0, len(content))
+
+	var count int
+	for _, char := range content {
+		if char == ' ' {
+			count++
+		} else {
+			count = 0
+		}
+
+		if count > 1 {
+			continue
+		}
+		newChars = append(newChars, char)
+	}
+
+	return newChars
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/consecutive_newlines.go

@@ -0,0 +1,139 @@
+package textutils
+
+import (
+	"bytes"
+	"unicode/utf8"
+)
+
+func TrimUnnecessaryHardLineBreaks(content []byte) []byte {
+	content = bytes.ReplaceAll(content, []byte("  \n\n"), []byte("\n\n"))
+	content = bytes.ReplaceAll(content, []byte("  \n  \n"), []byte("\n\n"))
+	content = bytes.ReplaceAll(content, []byte("  \n \n"), []byte("\n\n"))
+	// out = bytes.ReplaceAll(out, []byte("\n  \n"), []byte("\n\n"))
+
+	return content
+}
+
+func TrimConsecutiveNewlines(input []byte) []byte {
+	var result []byte
+	newlineCount := 0
+	spaceBuffer := []byte{}
+
+	for i := 0; i < len(input); {
+		r, size := utf8.DecodeRune(input[i:])
+
+		if r == '\n' {
+			newlineCount++
+			if newlineCount <= 2 {
+				// Preserve up to 2 newlines, including preceding spaces
+				result = append(result, spaceBuffer...)
+				result = append(result, '\n')
+				spaceBuffer = spaceBuffer[:0] // Clear space buffer
+			} else {
+				// Skip additional newlines
+				spaceBuffer = spaceBuffer[:0] // Clear space buffer
+			}
+		} else if r == ' ' {
+			// Collect spaces into the space buffer
+			spaceBuffer = append(spaceBuffer, input[i:i+size]...)
+		} else {
+			// Reset newline count and append non-newline characters
+			newlineCount = 0
+			result = append(result, spaceBuffer...)
+			result = append(result, input[i:i+size]...)
+			spaceBuffer = spaceBuffer[:0] // Clear space buffer
+		}
+
+		i += size
+	}
+
+	// Append any trailing spaces
+	result = append(result, spaceBuffer...)
+
+	return result
+}
+
+/*
+func TrimConsecutiveNewlines(source []byte) []byte {
+	// Some performance optimizations:
+	// - If no replacement was done, we return the original slice and don't allocate.
+	// - We batch appends
+
+	var ret []byte
+
+	startNormal := 0
+	startMatch := -1
+
+	count := 0
+	// for i, b := range source {
+	for i := 0; i < len(source); i++ {
+		r, size := utf8.DecodeRune(source[i:])
+		_ = size
+
+		isNewline := r == '\n' // || r == marker.MarkerLineBreak
+		if isNewline {
+			count += 1
+		}
+
+		if startMatch == -1 && isNewline {
+			// Start of newlines
+			startMatch = i
+			i = i + size - 1
+			continue
+		} else if startMatch != -1 && isNewline {
+			// Middle of newlines
+			i = i + size - 1
+			continue
+		} else if startMatch != -1 {
+			// Character after the last newline character
+
+			if count > 2 {
+				if ret == nil {
+					ret = make([]byte, 0, len(source))
+				}
+
+				ret = append(ret, source[startNormal:startMatch]...)
+				ret = append(ret, '\n', '\n')
+				startNormal = i
+			}
+
+			startMatch = -1
+			count = 0
+		}
+	}
+
+	getStartEnd := func() (int, int, bool, bool) {
+		if startMatch == -1 && startNormal == 0 {
+			// a) no changes need to be done
+			return -1, -1, false, false
+		}
+
+		if count <= 2 {
+			// b) Only the normal characters still need to be added
+			return startNormal, len(source), true, false
+		}
+
+		// c) The match still needs to be replaced (and possible the previous normal characters be added)
+		return startNormal, startMatch, true, true
+	}
+
+	start, end, isKeepNeeded, isReplaceNeeded := getStartEnd()
+	if isKeepNeeded {
+		if ret == nil {
+			ret = make([]byte, 0, len(source))
+		}
+
+		ret = append(ret, source[start:end]...)
+		if isReplaceNeeded {
+			ret = append(ret, '\n', '\n')
+		}
+	}
+
+	if ret == nil {
+		// Hurray, we did not do any allocations with make()
+		// and instead just return the original slice.
+		return source
+	}
+	return ret
+}
+*/

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/delimiter.go

@@ -0,0 +1,37 @@
+package textutils
+
+import (
+	"bytes"
+)
+
+// DelimiterForEveryLine puts the delimiter not just at the start and end of the string
+// but if the text is divided on multiple lines, puts the delimiters on every line with content.
+//
+// Otherwise the bold/italic delimiters won't be recognized if it contains new line characters.
+func DelimiterForEveryLine(text []byte, delimiter []byte) []byte {
+	var buf bytes.Buffer
+
+	lines := bytes.Split(text, []byte("\n"))
+	for i, line := range lines {
+		leftExtra, trimmed, rightExtra := SurroundingSpaces(line)
+
+		if trimmed == nil {
+			// For empty lines, we don't need a delimiter
+			buf.Write(leftExtra)
+			buf.Write(rightExtra)
+		} else {
+			buf.Write(leftExtra)
+			buf.Write(delimiter)
+			buf.Write(trimmed)
+			buf.Write(delimiter)
+			buf.Write(rightExtra)
+		}
+
+		// To join the lines again, add a newlines character
+		if i < len(lines)-1 {
+			buf.WriteRune('\n')
+		}
+	}
+
+	return buf.Bytes()
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/escape_multiline.go

@@ -0,0 +1,56 @@
+package textutils
+
+import (
+	"bytes"
+	"unicode"
+)
+
+var (
+	doubleSpace = []byte{' ', ' '}
+
+	newlineBreak              = []byte{'\n'}
+	hardLineBreak             = []byte{' ', ' ', '\n'}
+	escapedNoContentLineBreak = []byte{'\\', '\n'}
+)
+
+// EscapeMultiLine deals with multiline content inside a link or a heading.
+func EscapeMultiLine(content []byte) []byte {
+	parts := bytes.Split(content, newlineBreak)
+	if len(parts) == 1 {
+		return content
+	}
+
+	output := make([]byte, 0, len(content))
+	for i := range parts {
+		trimmedLeft := bytes.TrimLeftFunc(parts[i], unicode.IsSpace)
+
+		if len(trimmedLeft) == 0 {
+			// A blank line would interrupt the link.
+			// So we need to escape the line
+			output = append(output, escapedNoContentLineBreak...)
+			continue
+		}
+
+		isLast := i == len(parts)-1
+		if isLast {
+			// For the last line we don't need to add any "\n" anymore
+			output = append(output, trimmedLeft...)
+			continue
+		}
+
+		// Now decide what ending we want:
+		if bytes.HasSuffix(trimmedLeft, doubleSpace) {
+			// We already have "  " so adding a "\n" is enough
+			output = append(output, trimmedLeft...)
+			output = append(output, newlineBreak...)
+			continue
+		} else {
+			// We *prefer* having a hard-line-break "  \n"
+			output = append(output, trimmedLeft...)
+			output = append(output, hardLineBreak...)
+			continue
+		}
+	}
+
+	return output
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/prefix_lines.go

@@ -0,0 +1,16 @@
+package textutils
+
+func PrefixLines(source []byte, repl []byte) []byte {
+	newSlice := make([]byte, 0, len(source))
+
+	newSlice = append(newSlice, repl...)
+	for _, b := range source {
+		newSlice = append(newSlice, b)
+		if b == '\n' {
+			newSlice = append(newSlice, repl...)
+		}
+
+	}
+
+	return newSlice
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/quote.go

@@ -0,0 +1,39 @@
+package textutils
+
+import "bytes"
+
+const (
+	DOUBLE_QUOTE = '"'
+	SINGLE_QUOTE = '\''
+)
+
+func SurroundBy(content []byte, chars []byte) []byte {
+	content = append(chars, content...)
+	content = append(content, chars...)
+	return content
+}
+func SurroundByQuotes(content []byte) []byte {
+	if len(content) == 0 {
+		return nil
+	}
+
+	containsDoubleQuote := bytes.ContainsRune(content, DOUBLE_QUOTE)
+	containsSingleQuote := bytes.ContainsRune(content, SINGLE_QUOTE)
+
+	if containsDoubleQuote && containsSingleQuote {
+		// Escape all quotes
+		content = bytes.ReplaceAll(content, []byte(`"`), []byte(`\"`))
+
+		// Surround the content by double quotes
+		return SurroundBy(content, []byte(`"`))
+	}
+	if containsDoubleQuote {
+		// Since it contains double quotes (but no single quotes)
+		// we can surround it by single quotes
+		return SurroundBy(content, []byte(`'`))
+	}
+
+	// It may contain single quotes, but definitely no double quotes,
+	// so we can safely surround it by double quotes.
+	return SurroundBy(content, []byte(`"`))
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/surrounding_spaces.go

@@ -0,0 +1,58 @@
+/*
+
+The logic to handle whitespace around delimiters was initially developed
+in the fork from "anyproto" by Roman Khafizianov and Mikhail.
+
+The changes were then merged upstream by Johannes Kaufmann.
+
+https://github.com/anyproto/html-to-markdown
+https://github.com/JohannesKaufmann/html-to-markdown
+
+-----------
+
+MIT License
+
+Copyright (c) 2018 Johannes Kaufmann
+Copyright (c) 2020 Roman Khafizianov
+Copyright (c) 2023 Mikhail
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+*/
+
+package textutils
+
+import (
+	"bytes"
+	"unicode"
+)
+
+func SurroundingSpaces(content []byte) ([]byte, []byte, []byte) {
+	rightTrimmed := bytes.TrimRightFunc(content, func(r rune) bool {
+		return unicode.IsSpace(r)
+	})
+	rightExtra := content[len(rightTrimmed):]
+
+	trimmed := bytes.TrimLeftFunc(rightTrimmed, func(r rune) bool {
+		return unicode.IsSpace(r)
+	})
+	leftExtra := content[0 : len(rightTrimmed)-len(trimmed)]
+
+	return leftExtra, trimmed, rightExtra
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/marker/marker.go

@@ -0,0 +1,34 @@
+package marker
+
+import (
+	"bytes"
+)
+
+const (
+	// For simplicity we are using a rune that is one byte wide. A character
+	// that is not used widely (apart from cli's) is the bell character (7).
+	MarkerEscaping rune = '\a'
+
+	// - - - - //
+
+	// Marker0                rune = '\uF000' // 61440
+	// Marker1                rune = '\uF001' // 61441
+	MarkerCodeBlockNewline rune = '\uF002' // 61442
+)
+
+var (
+	BytesMarkerEscaping = []byte{7}
+
+	BytesMarkerCodeBlockNewline = []byte{239, 128, 130}
+)
+
+func init() {
+	checkRuneAndByteSlice(MarkerEscaping, BytesMarkerEscaping)
+	checkRuneAndByteSlice(MarkerCodeBlockNewline, BytesMarkerCodeBlockNewline)
+}
+
+func checkRuneAndByteSlice(r rune, b []byte) {
+	if !bytes.Equal([]byte(string(r)), b) {
+		panic("the rune and byte slice do not represent the same character")
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base/base.go

@@ -0,0 +1,138 @@
+package base
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/collapse"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+
+	"golang.org/x/net/html"
+)
+
+type base struct{}
+
+// NewBasePlugin registers a bunch of stuff that is not necessarily related to commonmark,
+// like removing nodes, trimming whitespace, collapsing whitespace, ...
+func NewBasePlugin() converter.Plugin {
+	base := base{}
+	return &base
+}
+
+func (s *base) Name() string {
+	return "base"
+}
+func (b *base) Init(conv *converter.Converter) error {
+	conv.Register.TagType("#comment", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("head", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("script", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("style", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("link", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("meta", converter.TagTypeRemove, converter.PriorityStandard)
+
+	conv.Register.TagType("iframe", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("noscript", converter.TagTypeRemove, converter.PriorityStandard)
+
+	conv.Register.TagType("input", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("textarea", converter.TagTypeRemove, converter.PriorityStandard)
+
+	// "tr" is not in the `IsBlockNode` list,
+	// but we want to treat is as a block anyway.
+	// conv.Register.TagStrategy("tr", converter.StrategyMarkdownBlock, converter.PriorityStandard)
+	// conv.Register.TagType("tr", converter.BlockTagType, converter.PriorityStandard)
+
+	conv.Register.PreRenderer(b.preRenderRemove, converter.PriorityEarly)
+	// Note: The priority is low, so that collapse runs _after_ all the other functions
+	conv.Register.PreRenderer(b.preRenderCollapse, converter.PriorityLate)
+
+	conv.Register.TextTransformer(b.handleTextTransform, converter.PriorityStandard)
+
+	conv.Register.PostRenderer(b.postRenderTrimContent, converter.PriorityStandard)
+	conv.Register.PostRenderer(b.postRenderUnescapeContent, converter.PriorityStandard+20)
+
+	return nil
+}
+
+func (b *base) preRenderRemove(ctx converter.Context, doc *html.Node) {
+	var finder func(node *html.Node)
+	finder = func(node *html.Node) {
+		name := dom.NodeName(node)
+
+		if tagType, _ := ctx.GetTagType(name); tagType == converter.TagTypeRemove {
+			dom.RemoveNode(node)
+			return
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			// Because we are sometimes removing a node, this causes problems
+			// with the for loop. Using `defer` is a cool trick!
+			// https://gist.github.com/loopthrough/17da0f416054401fec355d338727c46e
+			defer finder(child)
+		}
+	}
+	finder(doc)
+
+	// - - - - - - - //
+
+	// After removing elements (see above) it can happen that we have
+	// two #text nodes right next to each other. This would cause problems
+	// with the collapse so we merge them together.
+	domutils.MergeAdjacentTextNodes(doc)
+}
+
+func (b *base) preRenderCollapse(ctx converter.Context, doc *html.Node) {
+	collapse.Collapse(doc, &collapse.DomFuncs{
+		IsBlockNode: func(node *html.Node) bool {
+			tagName := dom.NodeName(node)
+			tagType, ok := ctx.GetTagType(tagName)
+			if ok {
+				return tagType == converter.TagTypeBlock
+			}
+
+			return dom.NameIsBlockNode(tagName)
+		},
+	})
+}
+
+var characterEntityReplacer = strings.NewReplacer(
+	// We are not using `html.EscapeString` because we
+	// care about fewer characters
+	"<", "&lt;",
+	">", "&gt;",
+
+	// Note: We are not escaping "&" as "&amp;" anymore.
+	// In most cases the "&" is completely fine.
+	// https://github.com/JohannesKaufmann/html-to-markdown/issues/178
+)
+
+func (b *base) handleTextTransform(ctx converter.Context, content string) string {
+
+	// TODO: similar to UnEscapers also only escape if nessesary.
+	//       "<" only if not followed by space
+	//       "&" only if character entity
+	content = characterEntityReplacer.Replace(content)
+
+	// TODO: reduce conversion between types
+	content = string(ctx.EscapeContent([]byte(content)))
+
+	return content
+}
+
+func (b *base) postRenderTrimContent(ctx converter.Context, result []byte) []byte {
+	// Remove whitespace from the beginning & end
+	result = bytes.TrimSpace(result)
+
+	// Remove too many newlines
+	result = textutils.TrimConsecutiveNewlines(result)
+	result = textutils.TrimUnnecessaryHardLineBreaks(result)
+
+	return result
+}
+func (b *base) postRenderUnescapeContent(ctx converter.Context, result []byte) []byte {
+	result = ctx.UnEscapeContent(result)
+	return result
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base/renderers.go

@@ -0,0 +1,62 @@
+package base
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+// RenderAsHTML will render the node as HTML using `html.Render()`
+// Newlines will be inserted depending on the configured `TagType`.
+//
+// As an example, you could do such a combination:
+//
+//	"A text with <strong>bold</strong> and *italic* text"`
+func RenderAsHTML(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+	_ = html.Render(w, node) // TODO: what to do with error?
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}
+
+// RenderAsHTMLWrapper will render the node as HTML
+// and render the children as markdown.
+func RenderAsHTMLWrapper(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	name := dom.NodeName(node)
+
+	w.WriteString("<")
+	w.WriteString(name)
+	// TODO: also render the attributes?
+	w.WriteString(">\n\n")
+
+	ctx.RenderChildNodes(ctx, w, node)
+
+	w.WriteString("\n\n</")
+	w.WriteString(name)
+	w.WriteString(">")
+	return converter.RenderSuccess
+}
+
+// RenderAsPlaintextWrapper will keep the children of this node as markdown.
+func RenderAsPlaintextWrapper(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+	ctx.RenderChildNodes(ctx, w, node)
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/commonmark.go

@@ -0,0 +1,206 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+type commonmark struct {
+	config
+}
+
+type OptionFunc = func(config *config)
+
+// _ or *
+//
+// default: *
+func WithEmDelimiter(delimiter string) OptionFunc {
+	return func(config *config) {
+		config.EmDelimiter = delimiter
+	}
+}
+
+// ** or __
+//
+// default: **
+func WithStrongDelimiter(delimiter string) OptionFunc {
+	return func(config *config) {
+		config.StrongDelimiter = delimiter
+	}
+}
+
+// Any Thematic break
+//
+// default: "* * *"
+func WithHorizontalRule(rule string) OptionFunc {
+	return func(config *config) {
+		config.HorizontalRule = rule
+	}
+}
+
+// "-", "+", or "*"
+//
+// default: "-"
+func WithBulletListMarker(marker string) OptionFunc {
+	return func(config *config) {
+		config.BulletListMarker = marker
+	}
+}
+func WithListEndComment(enabled bool) OptionFunc {
+	return func(config *config) {
+		config.DisableListEndComment = !enabled
+	}
+}
+
+// ``` or ~~~
+//
+// default: ```
+func WithCodeBlockFence(fence string) OptionFunc {
+	return func(config *config) {
+		config.CodeBlockFence = fence
+	}
+}
+
+// "setext" or "atx"
+//
+// default: "atx"
+func WithHeadingStyle(style headingStyle) OptionFunc {
+	return func(config *config) {
+		config.HeadingStyle = style
+	}
+}
+
+// WithLinkEmptyHrefBehavior configures how links with *empty hrefs* are rendered.
+// Take for example:
+//
+//	<a href="">the link content</a>
+//
+// LinkBehaviorRenderAsLink would result in "[the link content]()"
+//
+// LinkBehaviorSkipLink would result in "the link content"
+func WithLinkEmptyHrefBehavior(behavior linkRenderingBehavior) OptionFunc {
+	return func(config *config) {
+		config.LinkEmptyHrefBehavior = behavior
+	}
+}
+
+// WithLinkEmptyContentBehavior configures how links *without content* are rendered.
+// Take for example:
+//
+//	<a href="/page"></a>
+//
+// LinkBehaviorRenderAsLink would result in "[](/page)"
+//
+// LinkBehaviorSkipLink would result in an empty string.
+func WithLinkEmptyContentBehavior(behavior linkRenderingBehavior) OptionFunc {
+	return func(config *config) {
+		config.LinkEmptyContentBehavior = behavior
+	}
+}
+
+// TODO: allow changing the link style once the render logic is implemented
+//
+// "inlined" or "referenced_index" or "referenced_short"
+//
+// default: inlined
+// func WithLinkStyle(style linkStyle) OptionFunc {
+// 	return func(config *config) {
+// 		config.LinkStyle = style
+// 	}
+// }
+
+// NewCommonmarkPlugin registers the markdown syntax of commonmark.
+func NewCommonmarkPlugin(opts ...OptionFunc) converter.Plugin {
+
+	cfg := &config{}
+	for _, opt := range opts {
+		opt(cfg)
+	}
+
+	cm := commonmark{
+		config: fillInDefaultConfig(cfg),
+	}
+
+	return &cm
+}
+
+func (s *commonmark) Name() string {
+	return "commonmark"
+}
+func (cm *commonmark) Init(conv *converter.Converter) error {
+	if err := validateConfig(&cm.config); err != nil {
+		return err
+	}
+
+	// - - - - - - - - //
+
+	conv.Register.PreRenderer(cm.handlePreRender, converter.PriorityStandard)
+
+	// Note: Should run after "collapse" & also after "remove"
+	conv.Register.PreRenderer(func(ctx converter.Context, doc *html.Node) {
+		if cm.DisableListEndComment {
+			// Early return if the feature is unwanted
+			return
+		}
+
+		domutils.AddListEndComments(ctx, doc)
+	}, converter.PriorityLate+100)
+
+	conv.Register.EscapedChar(
+		'\\',
+		'*', '_', '-', '+',
+		'.', '>', '|',
+		'$',
+		'#', '=',
+		'[', ']', '(', ')',
+		'!',
+		'~', '`', '"', '\'',
+	)
+	conv.Register.UnEscaper(escape.IsItalicOrBold, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsBlockQuote, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsAtxHeader, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsSetextHeader, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsDivider, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsOrderedList, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsUnorderedList, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsImageOrLink, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsFencedCode, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsInlineCode, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsBackslash, converter.PriorityStandard)
+
+	conv.Register.Renderer(cm.handleRender, converter.PriorityStandard)
+
+	conv.Register.TextTransformer(cm.handleTextTransform, converter.PriorityLate)
+
+	conv.Register.PostRenderer(cm.handlePostRenderCodeBlockNewline, converter.PriorityLate)
+
+	return nil
+}
+
+func (cm commonmark) handlePostRenderCodeBlockNewline(ctx converter.Context, content []byte) []byte {
+	return bytes.ReplaceAll(
+		content,
+		[]byte(string(marker.BytesMarkerCodeBlockNewline)),
+		[]byte("\n"),
+	)
+}
+
+func (cm commonmark) handleTextTransform(ctx converter.Context, content string) string {
+
+	if isEnabled, ok := ctx.Value("is_inside_link").(bool); ok && isEnabled {
+		content = strings.Replace(content, string(marker.MarkerEscaping)+`]`, `\]`, -1)
+	}
+	// if isEnabled, ok := ctx.Value("is_inside_heading").(bool); ok && isEnabled {
+	// 	// The "#" character would be completely removed, if at the _end_
+	// 	// of the heading content. So always escape it inside headings.
+	// 	content = strings.Replace(content, string(marker.MarkerEscaping)+`#`, `\#`, -1)
+	// }
+
+	return content
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/handle_pre_render.go

@@ -0,0 +1,94 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"golang.org/x/net/html"
+)
+
+func nameIsBold(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "strong" || name == "b"
+}
+func nameIsItalic(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "em" || name == "i"
+}
+
+func nameIsBoldOrItalic(node *html.Node) bool {
+	return nameIsBold(node) || nameIsItalic(node)
+}
+func nameIsBothBoldOrItalic(a, b *html.Node) bool {
+	if nameIsBold(a) && nameIsBold(b) {
+		return true
+	}
+	if nameIsItalic(a) && nameIsItalic(b) {
+		return true
+	}
+
+	return false
+}
+
+func nameIsPre(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "pre"
+}
+func nameIsInlineCode(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "code" || name == "var" || name == "samp" || name == "kbd" || name == "tt"
+}
+
+func nameIsLink(node *html.Node) bool {
+	return dom.NodeName(node) == "a"
+}
+
+func nameIsBothLink(a, b *html.Node) bool {
+	return dom.NodeName(a) == "a" && dom.NodeName(b) == "a"
+}
+
+func nameIsHeading(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	if name == "h1" || name == "h2" || name == "h3" || name == "h4" || name == "h5" || name == "h6" {
+		return true
+	}
+	return false
+}
+
+// func nameIsBlockquote(node *html.Node) bool {
+// 	return dom.NodeName(node) == "blockquote"
+// }
+
+func (c *commonmark) handlePreRender(ctx converter.Context, doc *html.Node) {
+	domutils.RenameFakeSpans(ctx, doc)
+
+	// domutils.SplitUp(ctx, doc, nameIsBoldOrItalic, nameIsLink, atom.Span)
+
+	// domutils.SplitUp(ctx, doc, nameIsLink, nameIsHeading, atom.Div)
+	// domutils.SplitUp(ctx, doc, nameIsLink, nameIsBlockquote, atom.Div)
+
+	// - - - Bold / Italic - - - //
+	domutils.RemoveRedundant(doc, nameIsBothBoldOrItalic)
+	domutils.MergeAdjacent(doc, nameIsBoldOrItalic)
+
+	// domutils.MovePunctuation(ctx, doc, nameIsBoldOrItalic)
+
+	// - - - Code - - - //
+	domutils.RemoveEmptyCode(ctx, doc)
+	domutils.SwapTags(ctx, doc, nameIsInlineCode, nameIsPre)
+	domutils.MergeAdjacent(doc, nameIsInlineCode)
+
+	domutils.AddSpace(ctx, doc, nameIsBoldOrItalic, nameIsInlineCode)
+
+	// - - - Link - - - //
+	domutils.RemoveRedundant(doc, nameIsBothLink)
+	domutils.SwapTags(ctx, doc, nameIsBoldOrItalic, nameIsLink)
+
+	// - - - Heading - - - //
+	domutils.SwapTags(ctx, doc, nameIsLink, nameIsHeading)
+	domutils.LeafBlockAlternatives(ctx, doc)
+
+	// - - - List - - - //
+	domutils.MoveListItems(ctx, doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/handle_render.go

@@ -0,0 +1,47 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) handleRender(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	name := dom.NodeName(n)
+
+	switch name {
+	case "strong", "b",
+		"em", "i":
+		return c.renderBoldItalic(ctx, w, n)
+	case "hr":
+		return c.renderDivider(ctx, w, n)
+	case "br":
+		return c.renderBreak(ctx, w, n)
+	case "ul", "ol":
+		return c.renderListContainer(ctx, w, n)
+
+	case "pre":
+		return c.renderBlockCode(ctx, w, n)
+	case "code",
+		"var", "samp", "kbd", "tt":
+		return c.renderInlineCode(ctx, w, n)
+
+	case "blockquote":
+		return c.renderBlockquote(ctx, w, n)
+
+	case "h1", "h2", "h3", "h4", "h5", "h6":
+		return c.renderHeading(ctx, w, n)
+
+	case "img":
+		return c.renderImage(ctx, w, n)
+
+	case "a":
+		return c.renderLink(ctx, w, n)
+
+	case "#comment":
+		return c.renderComment(ctx, w, n)
+	}
+
+	return converter.RenderTryNext
+
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/options.go

@@ -0,0 +1,130 @@
+package commonmark
+
+type linkStyle string
+
+const (
+	// For example:
+	//
+	//  [view more](/about.html)
+	LinkStyleInlined linkStyle = "inlined"
+
+	LinkStyleReferencedIndex linkStyle = "referenced_index"
+	LinkStyleReferencedShort linkStyle = "referenced_short"
+)
+
+type headingStyle string
+
+const (
+	// HeadingStyleATX is the heading style of prefixing the heading with "#" signs indicating the level. For example:
+	//
+	//  ## Heading
+	HeadingStyleATX headingStyle = "atx"
+
+	// HeadingStyleSetext is the heading style of putting "=" or "-" on the followed line. For example:
+	//
+	//  Heading
+	//  -------
+	HeadingStyleSetext headingStyle = "setext"
+)
+
+type linkRenderingBehavior string
+
+const (
+	// LinkBehaviorRender renders the element as a link
+	LinkBehaviorRender linkRenderingBehavior = "render"
+	// LinkBehaviorSkip skips link rendering and falls back to the other rules (e.g. paragraph)
+	LinkBehaviorSkip linkRenderingBehavior = "skip"
+)
+
+// config to customize the output. You can change stuff like
+// the character that is used for strong text.
+type config struct {
+	// _ or *
+	//
+	// default: *
+	EmDelimiter string
+
+	// ** or __
+	//
+	// default: **
+	StrongDelimiter string
+
+	// Any Thematic break
+	//
+	// default: "* * *"
+	HorizontalRule string
+
+	// "-", "+", or "*"
+	//
+	// default: "-"
+	BulletListMarker string
+
+	DisableListEndComment bool
+
+	// "indented" or "fenced"
+	//
+	// default: "indented"
+	// TODO: CodeBlockStyle string
+
+	// ``` or ~~~
+	//
+	// default: ```
+	CodeBlockFence string
+
+	// "setext" or "atx"
+	//
+	// default: "atx"
+	HeadingStyle headingStyle
+
+	// TODO: LineBreakStyle string "hard" or "soft"
+
+	// "inlined" or "referenced_index" or "referenced_short"
+	//
+	// default: inlined
+	LinkStyle linkStyle
+
+	LinkEmptyHrefBehavior    linkRenderingBehavior
+	LinkEmptyContentBehavior linkRenderingBehavior
+}
+
+func fillInDefaultConfig(cfg *config) config {
+	if cfg.EmDelimiter == "" {
+		// The new default is now "*" (instead of "_") as that works better inside words.
+		cfg.EmDelimiter = "*"
+	}
+	if cfg.StrongDelimiter == "" {
+		cfg.StrongDelimiter = "**"
+	}
+
+	if cfg.HorizontalRule == "" {
+		cfg.HorizontalRule = "* * *"
+	}
+
+	if cfg.BulletListMarker == "" {
+		cfg.BulletListMarker = "-"
+	}
+
+	// TODO: also check for spelling mistakes in "indented"
+	// if opt.CodeBlockStyle == "" {
+	// 	opt.CodeBlockStyle = "indented"
+	// }
+	if cfg.CodeBlockFence == "" {
+		cfg.CodeBlockFence = "```"
+	}
+
+	if cfg.HeadingStyle == "" {
+		cfg.HeadingStyle = "atx"
+	}
+
+	if cfg.LinkEmptyHrefBehavior == "" {
+		cfg.LinkEmptyHrefBehavior = LinkBehaviorRender
+	}
+	if cfg.LinkEmptyContentBehavior == "" {
+		cfg.LinkEmptyContentBehavior = LinkBehaviorRender
+	}
+	if cfg.LinkStyle == "" {
+		cfg.LinkStyle = LinkStyleInlined
+	}
+
+	return *cfg
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_blockquote.go

@@ -0,0 +1,32 @@
+package commonmark
+
+import (
+	"bytes"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderBlockquote(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+
+	content := buf.Bytes()
+	content = bytes.TrimSpace(content)
+	if content == nil {
+		return converter.RenderSuccess
+	}
+
+	content = textutils.TrimConsecutiveNewlines(content)
+	content = textutils.TrimUnnecessaryHardLineBreaks(content)
+	content = textutils.PrefixLines(content, []byte{'>', ' '})
+
+	w.WriteRune('\n')
+	w.WriteRune('\n')
+	w.Write(content)
+	w.WriteRune('\n')
+	w.WriteRune('\n')
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_bold_italic.go

@@ -0,0 +1,39 @@
+package commonmark
+
+import (
+	"bytes"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func (c commonmark) getDelimiter(n *html.Node) []byte {
+	name := dom.NodeName(n)
+	if name == "strong" || name == "b" {
+		return []byte(c.StrongDelimiter)
+	} else if name == "em" || name == "i" {
+		return []byte(c.EmDelimiter)
+	} else {
+		return nil
+	}
+}
+func (c commonmark) renderBoldItalic(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+
+	// Depending on the options & whether it is bold or italic there
+	// is going to be a different delimiter.
+	delimiter := c.getDelimiter(n)
+	content := buf.Bytes()
+
+	// If there is a newline character between the start and end delimiter
+	// the delimiters won't be recognized. Either we remove all newline characters
+	// OR on _every_ line we put start & end delimiters.
+	content = textutils.DelimiterForEveryLine(content, delimiter)
+
+	w.Write(content)
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_break.go

@@ -0,0 +1,12 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderBreak(_ converter.Context, w converter.Writer, _ *html.Node) converter.RenderStatus {
+	// Render a "hard line break"
+	w.WriteString("  \n")
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_code.go

@@ -0,0 +1,149 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderInlineCode(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	// TODO: configure delimeter in options?
+	fenceChar := '`'
+
+	codeContent, _ := getCodeWithoutTags(n)
+
+	// TODO: debug flag?
+	if len(codeContent) == 0 {
+		// fmt.Println("expected an empty inline code to be already removed")
+		// panic("expected an empty inline code to be already removed")
+	}
+	// TODO: configurable function to decide if inline or block?
+	if bytes.Contains(codeContent, []byte("\n")) {
+		// fmt.Println("inline code contains newlines")
+		// return c.renderBlockCode(ctx, w, n, render)
+	}
+
+	if bytes.TrimSpace(codeContent) == nil {
+		// No stripping occurs if the code span contains _only_ spaces:
+		w.WriteRune(fenceChar)
+		w.Write(codeContent)
+		w.WriteRune(fenceChar)
+		return converter.RenderSuccess
+	}
+
+	// Newlines in the text aren't great, since this is inline code and not a code block.
+	// Newlines will be stripped anyway in the browser, but it won't be recognized as code
+	// from the markdown parser when there is more than one newline.
+	codeContent = textutils.CollapseInlineCodeContent(codeContent)
+
+	code := string(codeContent)
+
+	maxCount := textutils.CalculateCodeFenceOccurrences(fenceChar, code)
+	maxCount++
+
+	fence := strings.Repeat(string(fenceChar), maxCount)
+
+	// Code contains a backtick as first character
+	if strings.HasPrefix(code, "`") {
+		code = " " + code
+	}
+	// Code contains a backtick as last character
+	if strings.HasSuffix(code, "`") {
+		code = code + " "
+	}
+
+	w.WriteString(fence)
+	w.WriteString(code)
+	w.WriteString(fence)
+
+	return converter.RenderSuccess
+}
+func (c *commonmark) renderBlockCode(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	code, infoString := getCodeWithoutTags(n)
+
+	if bytes.HasSuffix(code, []byte("\n")) {
+		code = code[:len(code)-1]
+	}
+
+	fenceChar, _ := utf8.DecodeRuneInString(c.CodeBlockFence)
+	fence := textutils.CalculateCodeFence(fenceChar, string(code))
+
+	// We want to keep the original content inside the code block untouched.
+	// Because multiple newlines would be trimmed, we temporarily replace it with another character.
+	code = bytes.ReplaceAll(code, []byte("\n"), marker.BytesMarkerCodeBlockNewline)
+
+	w.WriteString("\n\n")
+	w.WriteString(fence)
+	w.WriteString(infoString)
+	w.WriteRune('\n')
+	w.Write(code)
+	w.WriteRune('\n')
+	w.WriteString(fence)
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}
+
+func getCodeLanguage(n *html.Node) string {
+	class := dom.GetAttributeOr(n, "class", "")
+
+	parts := strings.Split(class, " ")
+	for _, part := range parts {
+		if !strings.Contains(part, "language-") && !strings.Contains(part, "lang-") {
+			continue
+		}
+
+		part = strings.Replace(part, "language-", "", 1)
+		part = strings.Replace(part, "lang-", "", 1)
+
+		return part
+	}
+
+	return ""
+}
+func getCodeWithoutTags(startNode *html.Node) ([]byte, string) {
+	var buf bytes.Buffer
+	var infoString string
+
+	var f func(*html.Node)
+	f = func(n *html.Node) {
+		if n.Type == html.ElementNode && (n.Data == "code" || n.Data == "pre") {
+
+			// TODO: what if multiple elements have an info string?
+			if infoString == "" {
+				infoString = getCodeLanguage(n)
+			}
+		}
+
+		// - - - //
+
+		if n.Type == html.ElementNode && (n.Data == "style" || n.Data == "script" || n.Data == "textarea") {
+			return
+		}
+		if n.Type == html.ElementNode && (n.Data == "br" || n.Data == "div") {
+			buf.WriteString("\n")
+		}
+
+		if n.Type == html.TextNode {
+			// if strings.TrimSpace(n.Data) == "" && strings.Contains(n.Data, "\n") {
+			// 	buf.WriteString("\n")
+			// }
+			buf.WriteString(n.Data)
+			return
+		}
+
+		for c := n.FirstChild; c != nil; c = c.NextSibling {
+			f(c)
+		}
+	}
+
+	f(startNode)
+
+	return buf.Bytes(), infoString
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_comment.go

@@ -0,0 +1,25 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderComment(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+
+	if n.Data == domutils.ListEndCommentData {
+		// We definitely want to render the list end comments
+		// that were just added
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+		_ = html.Render(w, n)
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+		return converter.RenderSuccess
+
+	}
+
+	// Fallback to the normal settings for comments
+	return converter.RenderTryNext
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_divider.go

@@ -0,0 +1,15 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderDivider(_ converter.Context, w converter.Writer, _ *html.Node) converter.RenderStatus {
+
+	w.WriteString("\n\n")
+	w.WriteString(c.HorizontalRule)
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_heading.go

@@ -0,0 +1,150 @@
+package commonmark
+
+import (
+	"bytes"
+	"regexp"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+// TODO: remove regex
+var multipleSpacesR = regexp.MustCompile(`  +`)
+
+func (r *commonmark) setextUnderline(level int, width int) []byte {
+	line := "-"
+	if level == 1 {
+		line = "="
+	}
+
+	return bytes.Repeat([]byte(line), width)
+}
+func (r *commonmark) atxPrefix(level int) []byte {
+	return bytes.Repeat([]byte("#"), level)
+}
+
+func getHeadingLevel(name string) int {
+	switch name {
+	case "h1":
+		return 1
+	case "h2":
+		return 2
+	case "h3":
+		return 3
+	case "h4":
+		return 4
+	case "h5":
+		return 5
+	case "h6":
+		return 6
+	default:
+		return 6
+	}
+}
+func runeCount(chars []rune) (count int) {
+	for _, char := range chars {
+		if char == marker.MarkerEscaping {
+			continue
+		}
+		count++
+	}
+	return
+}
+func getUnderlineWidth(content []byte, minVal int) int {
+	var width int
+
+	parts := bytes.Split(content, []byte("\n"))
+	for _, part := range parts {
+		// Count how wide the line should be,
+		// while using RuneCount to correctly count ä, ö, ...
+		//
+		// TODO: optimize function w := utf8.RuneCount(part)
+		w := runeCount([]rune(string(part)))
+		if w > width {
+			width = w
+		}
+	}
+
+	// Technically the minimum value is only one character,
+	// but one dash could easily trigger a heading.
+	if width < minVal {
+		return minVal
+	}
+
+	return width
+}
+
+func escapePoundSignAtEnd(s []byte) []byte {
+	// -1 #
+	// -2 placeholder
+	// -3 maybe \
+
+	if s[len(s)-1] != '#' {
+		// We don't have a # at the end,
+		// so there is no work to do...
+		return s
+	}
+	if len(s) >= 3 && s[len(s)-3] == '\\' {
+		// It is already escaped,
+		// so there is no work to do...
+		return s
+	}
+
+	// Because we have a # at the end,
+	// we should manually force the escaping
+	// by overriding the placeholder.
+	s[len(s)-2] = '\\'
+
+	return s
+}
+
+func (c *commonmark) renderHeading(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	// ctx = context.WithValue(ctx, "is_inside_heading", true)
+
+	level := getHeadingLevel(dom.NodeName(n))
+
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+	content := buf.Bytes()
+
+	if len(bytes.TrimSpace(content)) == 0 {
+		return converter.RenderSuccess
+	}
+
+	if c.HeadingStyle == HeadingStyleSetext && level < 3 {
+		// Note: We don't want to use `TrimUnnecessaryHardLineBreaks` here,
+		// since `EscapeMultiLine` also takes care of newlines.
+		content = textutils.TrimConsecutiveNewlines(content)
+		content = textutils.EscapeMultiLine(content)
+
+		width := getUnderlineWidth(content, 3)
+		underline := c.setextUnderline(level, width)
+
+		w.WriteString("\n\n")
+		w.Write(content)
+		w.WriteRune('\n')
+		w.Write(underline)
+		w.WriteString("\n\n")
+	} else {
+		content = bytes.ReplaceAll(content, []byte("\n"), []byte(" "))
+		content = bytes.ReplaceAll(content, []byte("\r"), []byte(" "))
+		// Replace multiple spaces by one space.
+		content = multipleSpacesR.ReplaceAll(content, []byte(" "))
+
+		content = bytes.TrimSpace(content)
+
+		// A # sign at the end would be removed otherwise
+		content = escapePoundSignAtEnd(content)
+
+		w.WriteString("\n\n")
+		w.Write(c.atxPrefix(level))
+		w.WriteRune(' ')
+		w.Write(content)
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_image.go

@@ -0,0 +1,63 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func escapeAlt(altString string) string {
+	alt := []byte(altString)
+
+	var buf bytes.Buffer
+	for i := range alt {
+		if alt[i] == '[' || alt[i] == ']' {
+			prevIndex := i - 1
+			if prevIndex < 0 || alt[prevIndex] != '\\' {
+				buf.WriteRune('\\')
+			}
+		}
+		buf.WriteByte(alt[i])
+	}
+
+	return buf.String()
+}
+
+func (c *commonmark) renderImage(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	src := dom.GetAttributeOr(n, "src", "")
+	src = strings.TrimSpace(src)
+	if src == "" {
+		return converter.RenderTryNext
+	}
+
+	src = ctx.AssembleAbsoluteURL(ctx, "img", src)
+
+	title := dom.GetAttributeOr(n, "title", "")
+	title = strings.ReplaceAll(title, "\n", " ")
+
+	alt := dom.GetAttributeOr(n, "alt", "")
+	alt = strings.ReplaceAll(alt, "\n", " ")
+
+	// The alt description will be placed between two square brackets `[alt]`
+	// so make sure that those characters are escaped.
+	alt = escapeAlt(alt)
+
+	w.WriteRune('!')
+	w.WriteRune('[')
+	w.WriteString(alt)
+	w.WriteRune(']')
+	w.WriteRune('(')
+	w.WriteString(src)
+	if title != "" {
+		// The destination and title must be seperated by a space
+		w.WriteRune(' ')
+		w.Write(textutils.SurroundByQuotes([]byte(title)))
+	}
+	w.WriteRune(')')
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_link.go

@@ -0,0 +1,103 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+// link in commonmark contains
+// - the link text (the visible text)
+// - a link destination (the URI that is the link destination)
+// - an optional link title
+type link struct {
+	*html.Node
+
+	before  []byte
+	content []byte
+	after   []byte
+
+	href  string
+	title string
+}
+
+func (c *commonmark) renderLinkInlined(w converter.Writer, l *link) converter.RenderStatus {
+
+	w.Write(l.before)
+	w.WriteRune('[')
+	w.Write(l.content)
+	w.WriteRune(']')
+	w.WriteRune('(')
+	w.WriteString(l.href)
+	if l.title != "" {
+		// The destination and title must be separated by a space
+		w.WriteRune(' ')
+		w.Write(textutils.SurroundByQuotes([]byte(l.title)))
+	}
+	w.WriteRune(')')
+	w.Write(l.after)
+
+	return converter.RenderSuccess
+}
+
+func (c *commonmark) renderLink(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	ctx = ctx.WithValue("is_inside_link", true)
+
+	href := dom.GetAttributeOr(n, "href", "")
+
+	href = strings.TrimSpace(href)
+	href = ctx.AssembleAbsoluteURL(ctx, "a", href)
+
+	if href == "" && c.config.LinkEmptyHrefBehavior == LinkBehaviorSkip {
+		// There is *no href* for the link. Now we have two options:
+		// Continue rendering as a link OR skip to let other renderers take over.
+		return converter.RenderTryNext
+	}
+
+	title := dom.GetAttributeOr(n, "title", "")
+	title = strings.ReplaceAll(title, "\n", " ")
+
+	l := &link{
+		Node:  n,
+		href:  href,
+		title: title,
+	}
+
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+	content := buf.Bytes()
+
+	if len(bytes.TrimSpace(content)) == 0 && c.config.LinkEmptyContentBehavior == LinkBehaviorSkip {
+		// There is *no content* inside the link. Now we have two options:
+		// Continue rendering as a link OR skip to let other renderers take over.
+		return converter.RenderTryNext
+	}
+
+	if l.href == "" {
+		// A link without href is valid, like e.g. [text]()
+		// But a title would make it invalid.
+		l.title = ""
+	}
+
+	leftExtra, trimmed, rightExtra := textutils.SurroundingSpaces(content)
+
+	// Note: We don't want to use `TrimUnnecessaryHardLineBreaks` here,
+	// since `EscapeMultiLine` also takes care of newlines.
+	trimmed = textutils.TrimConsecutiveNewlines(trimmed)
+	trimmed = textutils.EscapeMultiLine(trimmed)
+
+	l.before = leftExtra
+	l.content = trimmed
+	l.after = rightExtra
+
+	switch c.LinkStyle {
+	case LinkStyleInlined:
+		return c.renderLinkInlined(w, l)
+	default:
+		return converter.RenderTryNext
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_list.go

@@ -0,0 +1,111 @@
+package commonmark
+
+import (
+	"bytes"
+	"fmt"
+	"strconv"
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+func getStartAt(node *html.Node) int {
+	startVal := dom.GetAttributeOr(node, "start", "1")
+	startAt, err := strconv.Atoi(startVal)
+	if err != nil {
+		startAt = 1
+	}
+
+	return startAt
+}
+
+func (c commonmark) getPrefixFunc(n *html.Node, sliceLength int) func(int) string {
+	startAt := getStartAt(n)
+
+	return func(sliceIndex int) string {
+		if n.Data == "ul" {
+			return c.BulletListMarker + " "
+		}
+
+		currentIndex := startAt + sliceIndex
+		lastIndex := startAt + sliceLength - 1
+		maxLength := utf8.RuneCountInString(strconv.Itoa(lastIndex))
+
+		// Pad the numbers so that all prefix numbers in the list take up the same space
+		// `%02d.` -> "01. "
+		format := `%0` + strconv.Itoa(maxLength) + `d. `
+		return fmt.Sprintf(format, currentIndex)
+	}
+}
+
+func renderMultiLineListItem(w converter.Writer, content []byte, indentCount int) {
+	lines := bytes.Split(content, []byte("\n"))
+	indent := bytes.Repeat([]byte(" "), indentCount)
+
+	indentedCodeBlockNewline := append(marker.BytesMarkerCodeBlockNewline, indent...)
+
+	for i := range lines {
+		// Add indent to code block newlines
+		line := bytes.ReplaceAll(lines[i], marker.BytesMarkerCodeBlockNewline, indentedCodeBlockNewline)
+
+		if i != 0 {
+			// The first line is already indented through the prefix,
+			// all other lines need the correct amount of spaces.
+			w.Write(indent)
+		}
+		w.Write(line)
+
+		if i < len(lines)-1 {
+			w.WriteRune('\n')
+		}
+	}
+}
+
+func (c commonmark) renderListContainer(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	children := dom.AllChildNodes(n)
+	items := make([][]byte, 0, len(children))
+
+	for _, child := range children {
+		var buf bytes.Buffer
+		ctx.RenderNodes(ctx, &buf, child)
+
+		content := buf.Bytes()
+		content = bytes.TrimSpace(content)
+		if content == nil {
+			continue
+		}
+
+		items = append(items, content)
+	}
+
+	if len(items) == 0 {
+		return converter.RenderSuccess
+	}
+
+	getPrefix := c.getPrefixFunc(n, len(items))
+	indentCount := utf8.RuneCountInString(getPrefix(0))
+
+	w.WriteString("\n\n")
+	for i, item := range items {
+		w.WriteString(getPrefix(i))
+
+		item = textutils.TrimConsecutiveNewlines(item)
+		item = textutils.TrimUnnecessaryHardLineBreaks(item)
+		item = ctx.UnEscapeContent(item)
+
+		// An item might have different lines that each
+		// must be indented with the correct count of spaces.
+		renderMultiLineListItem(w, item, indentCount)
+
+		if i < len(items)-1 {
+			w.WriteRune('\n')
+		}
+	}
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/validation.go

@@ -0,0 +1,101 @@
+package commonmark
+
+import (
+	"fmt"
+	"strings"
+)
+
+func contains(values []string, searchVal string) bool {
+	for _, val := range values {
+		if val == searchVal {
+			return true
+		}
+	}
+	return false
+}
+
+// TODO: should this be with the commonmark package? Or more general?
+// TODO: Maybe make it an interface? And also have a GetPluginName function?
+type ValidateConfigError struct {
+	Key   string
+	Value string
+
+	// By default is "Key:Value" but can be
+	// overriden to e.g. "--key=value"
+	KeyWithValue string
+
+	patternDescription string
+}
+
+func (e *ValidateConfigError) setDefaultKeyWithValue() {
+	e.KeyWithValue = fmt.Sprintf("%s:%q", e.Key, e.Value)
+}
+func (e *ValidateConfigError) Error() string {
+	if e.KeyWithValue == "" {
+		e.setDefaultKeyWithValue()
+	}
+
+	return fmt.Sprintf("invalid value for %s must be %s", e.KeyWithValue, e.patternDescription)
+}
+
+func validateConfig(cfg *config) error {
+	if strings.Count(cfg.EmDelimiter, "_") != 1 && strings.Count(cfg.EmDelimiter, "*") != 1 {
+		return &ValidateConfigError{
+			Key:                "EmDelimiter",
+			Value:              cfg.EmDelimiter,
+			patternDescription: `exactly 1 character of "*" or "_"`,
+		}
+	}
+	if strings.Count(cfg.StrongDelimiter, "_") != 2 && strings.Count(cfg.StrongDelimiter, "*") != 2 {
+		return &ValidateConfigError{
+			Key:                "StrongDelimiter",
+			Value:              cfg.StrongDelimiter,
+			patternDescription: `exactly 2 characters of "**" or "__"`,
+		}
+	}
+
+	if strings.Count(cfg.HorizontalRule, "*") < 3 &&
+		strings.Count(cfg.HorizontalRule, "_") < 3 &&
+		strings.Count(cfg.HorizontalRule, "-") < 3 {
+		return &ValidateConfigError{
+			Key:                "HorizontalRule",
+			Value:              cfg.HorizontalRule,
+			patternDescription: `at least 3 characters of "*", "_" or "-"`,
+		}
+	}
+
+	if !contains([]string{"-", "+", "*"}, cfg.BulletListMarker) {
+		return &ValidateConfigError{
+			Key:                "BulletListMarker",
+			Value:              cfg.BulletListMarker,
+			patternDescription: `one of "-", "+" or "*"`,
+		}
+	}
+
+	if !contains([]string{"```", "~~~"}, cfg.CodeBlockFence) {
+		return &ValidateConfigError{
+			Key:                "CodeBlockFence",
+			Value:              cfg.CodeBlockFence,
+			patternDescription: "one of \"```\" or \"~~~\"",
+		}
+	}
+
+	if !contains([]string{"atx", "setext"}, string(cfg.HeadingStyle)) {
+		return &ValidateConfigError{
+			Key:                "HeadingStyle",
+			Value:              string(cfg.HeadingStyle),
+			patternDescription: `one of "atx" or "setext"`,
+		}
+	}
+
+	possibleLinkStyles := []string{string(LinkStyleInlined), string(LinkStyleReferencedIndex), string(LinkStyleReferencedShort)}
+	if !contains(possibleLinkStyles, string(cfg.LinkStyle)) {
+		return &ValidateConfigError{
+			Key:                "LinkStyle",
+			Value:              string(cfg.LinkStyle),
+			patternDescription: `one of "inlined", "referenced_index" or "referenced_short"`,
+		}
+	}
+
+	return nil
+}

vendor/github.com/andybalholm/cascadia/.travis.yml

@@ -0,0 +1,14 @@
+language: go
+
+go:
+  - 1.3
+  - 1.4
+
+install:
+  - go get github.com/andybalholm/cascadia
+
+script:
+ - go test -v
+
+notifications:
+  email: false

vendor/github.com/andybalholm/cascadia/LICENSE

@@ -0,0 +1,24 @@
+Copyright (c) 2011 Andy Balholm. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+   * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+   * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

vendor/github.com/andybalholm/cascadia/README.md

@@ -0,0 +1,144 @@
+# cascadia
+
+[![](https://travis-ci.org/andybalholm/cascadia.svg)](https://travis-ci.org/andybalholm/cascadia)
+
+The Cascadia package implements CSS selectors for use with the parse trees produced by the html package.
+
+To test CSS selectors without writing Go code, check out [cascadia](https://github.com/suntong/cascadia) the command line tool, a thin wrapper around this package.
+
+[Refer to godoc here](https://godoc.org/github.com/andybalholm/cascadia).
+
+## Example
+
+The following is an example of how you can use Cascadia.
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/andybalholm/cascadia"
+	"golang.org/x/net/html"
+)
+
+var pricingHtml string = `
+<div class="card mb-4 box-shadow">
+	<div class="card-header">
+		<h4 class="my-0 font-weight-normal">Free</h4>
+	</div>
+	<div class="card-body">
+		<h1 class="card-title pricing-card-title">$0/mo</h1>
+		<ul class="list-unstyled mt-3 mb-4">
+			<li>10 users included</li>
+			<li>2 GB of storage</li>
+			<li><a href="https://example.com">See more</a></li>
+		</ul>
+	</div>
+</div>
+
+<div class="card mb-4 box-shadow">
+	<div class="card-header">
+		<h4 class="my-0 font-weight-normal">Pro</h4>
+	</div>
+	<div class="card-body">
+		<h1 class="card-title pricing-card-title">$15/mo</h1>
+		<ul class="list-unstyled mt-3 mb-4">
+			<li>20 users included</li>
+			<li>10 GB of storage</li>
+			<li><a href="https://example.com">See more</a></li>
+		</ul>
+	</div>
+</div>
+
+<div class="card mb-4 box-shadow">
+	<div class="card-header">
+		<h4 class="my-0 font-weight-normal">Enterprise</h4>
+	</div>
+	<div class="card-body">
+		<h1 class="card-title pricing-card-title">$29/mo</h1>
+		<ul class="list-unstyled mt-3 mb-4">
+			<li>30 users included</li>
+			<li>15 GB of storage</li>
+			<li><a>See more</a></li>
+		</ul>
+	</div>
+</div>
+`
+
+func Query(n *html.Node, query string) *html.Node {
+	sel, err := cascadia.Parse(query)
+	if err != nil {
+		return &html.Node{}
+	}
+	return cascadia.Query(n, sel)
+}
+
+func QueryAll(n *html.Node, query string) []*html.Node {
+	sel, err := cascadia.Parse(query)
+	if err != nil {
+		return []*html.Node{}
+	}
+	return cascadia.QueryAll(n, sel)
+}
+
+func AttrOr(n *html.Node, attrName, or string) string {
+	for _, a := range n.Attr {
+		if a.Key == attrName {
+			return a.Val
+		}
+	}
+	return or
+}
+
+func main() {
+	doc, err := html.Parse(strings.NewReader(pricingHtml))
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Printf("List of pricing plans:\n\n")
+	for i, p := range QueryAll(doc, "div.card.mb-4.box-shadow") {
+		planName := Query(p, "h4").FirstChild.Data
+		price := Query(p, ".pricing-card-title").FirstChild.Data
+		usersIncluded := Query(p, "li:first-child").FirstChild.Data
+		storage := Query(p, "li:nth-child(2)").FirstChild.Data
+		detailsUrl := AttrOr(Query(p, "li:last-child a"), "href", "(No link available)")
+		fmt.Printf(
+			"Plan #%d\nName: %s\nPrice: %s\nUsers: %s\nStorage: %s\nDetails: %s\n\n",
+			i+1,
+			planName,
+			price,
+			usersIncluded,
+			storage,
+			detailsUrl,
+		)
+	}
+}
+```
+The output is:
+```
+List of pricing plans:
+
+Plan #1
+Name: Free
+Price: $0/mo
+Users: 10 users included
+Storage: 2 GB of storage
+Details: https://example.com
+
+Plan #2
+Name: Pro
+Price: $15/mo
+Users: 20 users included
+Storage: 10 GB of storage
+Details: https://example.com
+
+Plan #3
+Name: Enterprise
+Price: $29/mo
+Users: 30 users included
+Storage: 15 GB of storage
+Details: (No link available)
+```

vendor/github.com/andybalholm/cascadia/parser.go

@@ -0,0 +1,889 @@
+// Package cascadia is an implementation of CSS selectors.
+package cascadia
+
+import (
+	"errors"
+	"fmt"
+	"regexp"
+	"strconv"
+	"strings"
+)
+
+// a parser for CSS selectors
+type parser struct {
+	s string // the source text
+	i int    // the current position
+
+	// if `false`, parsing a pseudo-element
+	// returns an error.
+	acceptPseudoElements bool
+}
+
+// parseEscape parses a backslash escape.
+func (p *parser) parseEscape() (result string, err error) {
+	if len(p.s) < p.i+2 || p.s[p.i] != '\\' {
+		return "", errors.New("invalid escape sequence")
+	}
+
+	start := p.i + 1
+	c := p.s[start]
+	switch {
+	case c == '\r' || c == '\n' || c == '\f':
+		return "", errors.New("escaped line ending outside string")
+	case hexDigit(c):
+		// unicode escape (hex)
+		var i int
+		for i = start; i < start+6 && i < len(p.s) && hexDigit(p.s[i]); i++ {
+			// empty
+		}
+		v, _ := strconv.ParseUint(p.s[start:i], 16, 64)
+		if len(p.s) > i {
+			switch p.s[i] {
+			case '\r':
+				i++
+				if len(p.s) > i && p.s[i] == '\n' {
+					i++
+				}
+			case ' ', '\t', '\n', '\f':
+				i++
+			}
+		}
+		p.i = i
+		return string(rune(v)), nil
+	}
+
+	// Return the literal character after the backslash.
+	result = p.s[start : start+1]
+	p.i += 2
+	return result, nil
+}
+
+// toLowerASCII returns s with all ASCII capital letters lowercased.
+func toLowerASCII(s string) string {
+	var b []byte
+	for i := 0; i < len(s); i++ {
+		if c := s[i]; 'A' <= c && c <= 'Z' {
+			if b == nil {
+				b = make([]byte, len(s))
+				copy(b, s)
+			}
+			b[i] = s[i] + ('a' - 'A')
+		}
+	}
+
+	if b == nil {
+		return s
+	}
+
+	return string(b)
+}
+
+func hexDigit(c byte) bool {
+	return '0' <= c && c <= '9' || 'a' <= c && c <= 'f' || 'A' <= c && c <= 'F'
+}
+
+// nameStart returns whether c can be the first character of an identifier
+// (not counting an initial hyphen, or an escape sequence).
+func nameStart(c byte) bool {
+	return 'a' <= c && c <= 'z' || 'A' <= c && c <= 'Z' || c == '_' || c > 127
+}
+
+// nameChar returns whether c can be a character within an identifier
+// (not counting an escape sequence).
+func nameChar(c byte) bool {
+	return 'a' <= c && c <= 'z' || 'A' <= c && c <= 'Z' || c == '_' || c > 127 ||
+		c == '-' || '0' <= c && c <= '9'
+}
+
+// parseIdentifier parses an identifier.
+func (p *parser) parseIdentifier() (result string, err error) {
+	const prefix = '-'
+	var numPrefix int
+
+	for len(p.s) > p.i && p.s[p.i] == prefix {
+		p.i++
+		numPrefix++
+	}
+
+	if len(p.s) <= p.i {
+		return "", errors.New("expected identifier, found EOF instead")
+	}
+
+	if c := p.s[p.i]; !(nameStart(c) || c == '\\') {
+		return "", fmt.Errorf("expected identifier, found %c instead", c)
+	}
+
+	result, err = p.parseName()
+	if numPrefix > 0 && err == nil {
+		result = strings.Repeat(string(prefix), numPrefix) + result
+	}
+	return
+}
+
+// parseName parses a name (which is like an identifier, but doesn't have
+// extra restrictions on the first character).
+func (p *parser) parseName() (result string, err error) {
+	i := p.i
+loop:
+	for i < len(p.s) {
+		c := p.s[i]
+		switch {
+		case nameChar(c):
+			start := i
+			for i < len(p.s) && nameChar(p.s[i]) {
+				i++
+			}
+			result += p.s[start:i]
+		case c == '\\':
+			p.i = i
+			val, err := p.parseEscape()
+			if err != nil {
+				return "", err
+			}
+			i = p.i
+			result += val
+		default:
+			break loop
+		}
+	}
+
+	if result == "" {
+		return "", errors.New("expected name, found EOF instead")
+	}
+
+	p.i = i
+	return result, nil
+}
+
+// parseString parses a single- or double-quoted string.
+func (p *parser) parseString() (result string, err error) {
+	i := p.i
+	if len(p.s) < i+2 {
+		return "", errors.New("expected string, found EOF instead")
+	}
+
+	quote := p.s[i]
+	i++
+
+loop:
+	for i < len(p.s) {
+		switch p.s[i] {
+		case '\\':
+			if len(p.s) > i+1 {
+				switch c := p.s[i+1]; c {
+				case '\r':
+					if len(p.s) > i+2 && p.s[i+2] == '\n' {
+						i += 3
+						continue loop
+					}
+					fallthrough
+				case '\n', '\f':
+					i += 2
+					continue loop
+				}
+			}
+			p.i = i
+			val, err := p.parseEscape()
+			if err != nil {
+				return "", err
+			}
+			i = p.i
+			result += val
+		case quote:
+			break loop
+		case '\r', '\n', '\f':
+			return "", errors.New("unexpected end of line in string")
+		default:
+			start := i
+			for i < len(p.s) {
+				if c := p.s[i]; c == quote || c == '\\' || c == '\r' || c == '\n' || c == '\f' {
+					break
+				}
+				i++
+			}
+			result += p.s[start:i]
+		}
+	}
+
+	if i >= len(p.s) {
+		return "", errors.New("EOF in string")
+	}
+
+	// Consume the final quote.
+	i++
+
+	p.i = i
+	return result, nil
+}
+
+// parseRegex parses a regular expression; the end is defined by encountering an
+// unmatched closing ')' or ']' which is not consumed
+func (p *parser) parseRegex() (rx *regexp.Regexp, err error) {
+	i := p.i
+	if len(p.s) < i+2 {
+		return nil, errors.New("expected regular expression, found EOF instead")
+	}
+
+	// number of open parens or brackets;
+	// when it becomes negative, finished parsing regex
+	open := 0
+
+loop:
+	for i < len(p.s) {
+		switch p.s[i] {
+		case '(', '[':
+			open++
+		case ')', ']':
+			open--
+			if open < 0 {
+				break loop
+			}
+		}
+		i++
+	}
+
+	if i >= len(p.s) {
+		return nil, errors.New("EOF in regular expression")
+	}
+	rx, err = regexp.Compile(p.s[p.i:i])
+	p.i = i
+	return rx, err
+}
+
+// skipWhitespace consumes whitespace characters and comments.
+// It returns true if there was actually anything to skip.
+func (p *parser) skipWhitespace() bool {
+	i := p.i
+	for i < len(p.s) {
+		switch p.s[i] {
+		case ' ', '\t', '\r', '\n', '\f':
+			i++
+			continue
+		case '/':
+			if strings.HasPrefix(p.s[i:], "/*") {
+				end := strings.Index(p.s[i+len("/*"):], "*/")
+				if end != -1 {
+					i += end + len("/**/")
+					continue
+				}
+			}
+		}
+		break
+	}
+
+	if i > p.i {
+		p.i = i
+		return true
+	}
+
+	return false
+}
+
+// consumeParenthesis consumes an opening parenthesis and any following
+// whitespace. It returns true if there was actually a parenthesis to skip.
+func (p *parser) consumeParenthesis() bool {
+	if p.i < len(p.s) && p.s[p.i] == '(' {
+		p.i++
+		p.skipWhitespace()
+		return true
+	}
+	return false
+}
+
+// consumeClosingParenthesis consumes a closing parenthesis and any preceding
+// whitespace. It returns true if there was actually a parenthesis to skip.
+func (p *parser) consumeClosingParenthesis() bool {
+	i := p.i
+	p.skipWhitespace()
+	if p.i < len(p.s) && p.s[p.i] == ')' {
+		p.i++
+		return true
+	}
+	p.i = i
+	return false
+}
+
+// parseTypeSelector parses a type selector (one that matches by tag name).
+func (p *parser) parseTypeSelector() (result tagSelector, err error) {
+	tag, err := p.parseIdentifier()
+	if err != nil {
+		return
+	}
+	return tagSelector{tag: toLowerASCII(tag)}, nil
+}
+
+// parseIDSelector parses a selector that matches by id attribute.
+func (p *parser) parseIDSelector() (idSelector, error) {
+	if p.i >= len(p.s) {
+		return idSelector{}, fmt.Errorf("expected id selector (#id), found EOF instead")
+	}
+	if p.s[p.i] != '#' {
+		return idSelector{}, fmt.Errorf("expected id selector (#id), found '%c' instead", p.s[p.i])
+	}
+
+	p.i++
+	id, err := p.parseName()
+	if err != nil {
+		return idSelector{}, err
+	}
+
+	return idSelector{id: id}, nil
+}
+
+// parseClassSelector parses a selector that matches by class attribute.
+func (p *parser) parseClassSelector() (classSelector, error) {
+	if p.i >= len(p.s) {
+		return classSelector{}, fmt.Errorf("expected class selector (.class), found EOF instead")
+	}
+	if p.s[p.i] != '.' {
+		return classSelector{}, fmt.Errorf("expected class selector (.class), found '%c' instead", p.s[p.i])
+	}
+
+	p.i++
+	class, err := p.parseIdentifier()
+	if err != nil {
+		return classSelector{}, err
+	}
+
+	return classSelector{class: class}, nil
+}
+
+// parseAttributeSelector parses a selector that matches by attribute value.
+func (p *parser) parseAttributeSelector() (attrSelector, error) {
+	if p.i >= len(p.s) {
+		return attrSelector{}, fmt.Errorf("expected attribute selector ([attribute]), found EOF instead")
+	}
+	if p.s[p.i] != '[' {
+		return attrSelector{}, fmt.Errorf("expected attribute selector ([attribute]), found '%c' instead", p.s[p.i])
+	}
+
+	p.i++
+	p.skipWhitespace()
+	key, err := p.parseIdentifier()
+	if err != nil {
+		return attrSelector{}, err
+	}
+	key = toLowerASCII(key)
+
+	p.skipWhitespace()
+	if p.i >= len(p.s) {
+		return attrSelector{}, errors.New("unexpected EOF in attribute selector")
+	}
+
+	if p.s[p.i] == ']' {
+		p.i++
+		return attrSelector{key: key, operation: ""}, nil
+	}
+
+	if p.i+2 >= len(p.s) {
+		return attrSelector{}, errors.New("unexpected EOF in attribute selector")
+	}
+
+	op := p.s[p.i : p.i+2]
+	if op[0] == '=' {
+		op = "="
+	} else if op[1] != '=' {
+		return attrSelector{}, fmt.Errorf(`expected equality operator, found "%s" instead`, op)
+	}
+	p.i += len(op)
+
+	p.skipWhitespace()
+	if p.i >= len(p.s) {
+		return attrSelector{}, errors.New("unexpected EOF in attribute selector")
+	}
+	var val string
+	var rx *regexp.Regexp
+	if op == "#=" {
+		rx, err = p.parseRegex()
+	} else {
+		switch p.s[p.i] {
+		case '\'', '"':
+			val, err = p.parseString()
+		default:
+			val, err = p.parseIdentifier()
+		}
+	}
+	if err != nil {
+		return attrSelector{}, err
+	}
+
+	p.skipWhitespace()
+	if p.i >= len(p.s) {
+		return attrSelector{}, errors.New("unexpected EOF in attribute selector")
+	}
+
+	// check if the attribute contains an ignore case flag
+	ignoreCase := false
+	if p.s[p.i] == 'i' || p.s[p.i] == 'I' {
+		ignoreCase = true
+		p.i++
+	}
+
+	p.skipWhitespace()
+	if p.i >= len(p.s) {
+		return attrSelector{}, errors.New("unexpected EOF in attribute selector")
+	}
+
+	if p.s[p.i] != ']' {
+		return attrSelector{}, fmt.Errorf("expected ']', found '%c' instead", p.s[p.i])
+	}
+	p.i++
+
+	switch op {
+	case "=", "!=", "~=", "|=", "^=", "$=", "*=", "#=":
+		return attrSelector{key: key, val: val, operation: op, regexp: rx, insensitive: ignoreCase}, nil
+	default:
+		return attrSelector{}, fmt.Errorf("attribute operator %q is not supported", op)
+	}
+}
+
+var (
+	errExpectedParenthesis        = errors.New("expected '(' but didn't find it")
+	errExpectedClosingParenthesis = errors.New("expected ')' but didn't find it")
+	errUnmatchedParenthesis       = errors.New("unmatched '('")
+)
+
+// parsePseudoclassSelector parses a pseudoclass selector like :not(p) or a pseudo-element
+// For backwards compatibility, both ':' and '::' prefix are allowed for pseudo-elements.
+// https://drafts.csswg.org/selectors-3/#pseudo-elements
+// Returning a nil `Sel` (and a nil `error`) means we found a pseudo-element.
+func (p *parser) parsePseudoclassSelector() (out Sel, pseudoElement string, err error) {
+	if p.i >= len(p.s) {
+		return nil, "", fmt.Errorf("expected pseudoclass selector (:pseudoclass), found EOF instead")
+	}
+	if p.s[p.i] != ':' {
+		return nil, "", fmt.Errorf("expected attribute selector (:pseudoclass), found '%c' instead", p.s[p.i])
+	}
+
+	p.i++
+	var mustBePseudoElement bool
+	if p.i >= len(p.s) {
+		return nil, "", fmt.Errorf("got empty pseudoclass (or pseudoelement)")
+	}
+	if p.s[p.i] == ':' { // we found a pseudo-element
+		mustBePseudoElement = true
+		p.i++
+	}
+
+	name, err := p.parseIdentifier()
+	if err != nil {
+		return
+	}
+	name = toLowerASCII(name)
+	if mustBePseudoElement && (name != "after" && name != "backdrop" && name != "before" &&
+		name != "cue" && name != "first-letter" && name != "first-line" && name != "grammar-error" &&
+		name != "marker" && name != "placeholder" && name != "selection" && name != "spelling-error") {
+		return out, "", fmt.Errorf("unknown pseudoelement :%s", name)
+	}
+
+	switch name {
+	case "not", "has", "haschild":
+		if !p.consumeParenthesis() {
+			return out, "", errExpectedParenthesis
+		}
+		sel, parseErr := p.parseSelectorGroup()
+		if parseErr != nil {
+			return out, "", parseErr
+		}
+		if !p.consumeClosingParenthesis() {
+			return out, "", errExpectedClosingParenthesis
+		}
+
+		out = relativePseudoClassSelector{name: name, match: sel}
+
+	case "contains", "containsown":
+		if !p.consumeParenthesis() {
+			return out, "", errExpectedParenthesis
+		}
+		if p.i == len(p.s) {
+			return out, "", errUnmatchedParenthesis
+		}
+		var val string
+		switch p.s[p.i] {
+		case '\'', '"':
+			val, err = p.parseString()
+		default:
+			val, err = p.parseIdentifier()
+		}
+		if err != nil {
+			return out, "", err
+		}
+		val = strings.ToLower(val)
+		p.skipWhitespace()
+		if p.i >= len(p.s) {
+			return out, "", errors.New("unexpected EOF in pseudo selector")
+		}
+		if !p.consumeClosingParenthesis() {
+			return out, "", errExpectedClosingParenthesis
+		}
+
+		out = containsPseudoClassSelector{own: name == "containsown", value: val}
+
+	case "matches", "matchesown":
+		if !p.consumeParenthesis() {
+			return out, "", errExpectedParenthesis
+		}
+		rx, err := p.parseRegex()
+		if err != nil {
+			return out, "", err
+		}
+		if p.i >= len(p.s) {
+			return out, "", errors.New("unexpected EOF in pseudo selector")
+		}
+		if !p.consumeClosingParenthesis() {
+			return out, "", errExpectedClosingParenthesis
+		}
+
+		out = regexpPseudoClassSelector{own: name == "matchesown", regexp: rx}
+
+	case "nth-child", "nth-last-child", "nth-of-type", "nth-last-of-type":
+		if !p.consumeParenthesis() {
+			return out, "", errExpectedParenthesis
+		}
+		a, b, err := p.parseNth()
+		if err != nil {
+			return out, "", err
+		}
+		if !p.consumeClosingParenthesis() {
+			return out, "", errExpectedClosingParenthesis
+		}
+		last := name == "nth-last-child" || name == "nth-last-of-type"
+		ofType := name == "nth-of-type" || name == "nth-last-of-type"
+		out = nthPseudoClassSelector{a: a, b: b, last: last, ofType: ofType}
+
+	case "first-child":
+		out = nthPseudoClassSelector{a: 0, b: 1, ofType: false, last: false}
+	case "last-child":
+		out = nthPseudoClassSelector{a: 0, b: 1, ofType: false, last: true}
+	case "first-of-type":
+		out = nthPseudoClassSelector{a: 0, b: 1, ofType: true, last: false}
+	case "last-of-type":
+		out = nthPseudoClassSelector{a: 0, b: 1, ofType: true, last: true}
+	case "only-child":
+		out = onlyChildPseudoClassSelector{ofType: false}
+	case "only-of-type":
+		out = onlyChildPseudoClassSelector{ofType: true}
+	case "input":
+		out = inputPseudoClassSelector{}
+	case "empty":
+		out = emptyElementPseudoClassSelector{}
+	case "root":
+		out = rootPseudoClassSelector{}
+	case "link":
+		out = linkPseudoClassSelector{}
+	case "lang":
+		if !p.consumeParenthesis() {
+			return out, "", errExpectedParenthesis
+		}
+		if p.i == len(p.s) {
+			return out, "", errUnmatchedParenthesis
+		}
+		val, err := p.parseIdentifier()
+		if err != nil {
+			return out, "", err
+		}
+		val = strings.ToLower(val)
+		p.skipWhitespace()
+		if p.i >= len(p.s) {
+			return out, "", errors.New("unexpected EOF in pseudo selector")
+		}
+		if !p.consumeClosingParenthesis() {
+			return out, "", errExpectedClosingParenthesis
+		}
+		out = langPseudoClassSelector{lang: val}
+	case "enabled":
+		out = enabledPseudoClassSelector{}
+	case "disabled":
+		out = disabledPseudoClassSelector{}
+	case "checked":
+		out = checkedPseudoClassSelector{}
+	case "visited", "hover", "active", "focus", "target":
+		// Not applicable in a static context: never match.
+		out = neverMatchSelector{value: ":" + name}
+	case "after", "backdrop", "before", "cue", "first-letter", "first-line", "grammar-error", "marker", "placeholder", "selection", "spelling-error":
+		return nil, name, nil
+	default:
+		return out, "", fmt.Errorf("unknown pseudoclass or pseudoelement :%s", name)
+	}
+	return
+}
+
+// parseInteger parses a  decimal integer.
+func (p *parser) parseInteger() (int, error) {
+	i := p.i
+	start := i
+	for i < len(p.s) && '0' <= p.s[i] && p.s[i] <= '9' {
+		i++
+	}
+	if i == start {
+		return 0, errors.New("expected integer, but didn't find it")
+	}
+	p.i = i
+
+	val, err := strconv.Atoi(p.s[start:i])
+	if err != nil {
+		return 0, err
+	}
+
+	return val, nil
+}
+
+// parseNth parses the argument for :nth-child (normally of the form an+b).
+func (p *parser) parseNth() (a, b int, err error) {
+	// initial state
+	if p.i >= len(p.s) {
+		goto eof
+	}
+	switch p.s[p.i] {
+	case '-':
+		p.i++
+		goto negativeA
+	case '+':
+		p.i++
+		goto positiveA
+	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
+		goto positiveA
+	case 'n', 'N':
+		a = 1
+		p.i++
+		goto readN
+	case 'o', 'O', 'e', 'E':
+		id, nameErr := p.parseName()
+		if nameErr != nil {
+			return 0, 0, nameErr
+		}
+		id = toLowerASCII(id)
+		if id == "odd" {
+			return 2, 1, nil
+		}
+		if id == "even" {
+			return 2, 0, nil
+		}
+		return 0, 0, fmt.Errorf("expected 'odd' or 'even', but found '%s' instead", id)
+	default:
+		goto invalid
+	}
+
+positiveA:
+	if p.i >= len(p.s) {
+		goto eof
+	}
+	switch p.s[p.i] {
+	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
+		a, err = p.parseInteger()
+		if err != nil {
+			return 0, 0, err
+		}
+		goto readA
+	case 'n', 'N':
+		a = 1
+		p.i++
+		goto readN
+	default:
+		goto invalid
+	}
+
+negativeA:
+	if p.i >= len(p.s) {
+		goto eof
+	}
+	switch p.s[p.i] {
+	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
+		a, err = p.parseInteger()
+		if err != nil {
+			return 0, 0, err
+		}
+		a = -a
+		goto readA
+	case 'n', 'N':
+		a = -1
+		p.i++
+		goto readN
+	default:
+		goto invalid
+	}
+
+readA:
+	if p.i >= len(p.s) {
+		goto eof
+	}
+	switch p.s[p.i] {
+	case 'n', 'N':
+		p.i++
+		goto readN
+	default:
+		// The number we read as a is actually b.
+		return 0, a, nil
+	}
+
+readN:
+	p.skipWhitespace()
+	if p.i >= len(p.s) {
+		goto eof
+	}
+	switch p.s[p.i] {
+	case '+':
+		p.i++
+		p.skipWhitespace()
+		b, err = p.parseInteger()
+		if err != nil {
+			return 0, 0, err
+		}
+		return a, b, nil
+	case '-':
+		p.i++
+		p.skipWhitespace()
+		b, err = p.parseInteger()
+		if err != nil {
+			return 0, 0, err
+		}
+		return a, -b, nil
+	default:
+		return a, 0, nil
+	}
+
+eof:
+	return 0, 0, errors.New("unexpected EOF while attempting to parse expression of form an+b")
+
+invalid:
+	return 0, 0, errors.New("unexpected character while attempting to parse expression of form an+b")
+}
+
+// parseSimpleSelectorSequence parses a selector sequence that applies to
+// a single element.
+func (p *parser) parseSimpleSelectorSequence() (Sel, error) {
+	var selectors []Sel
+
+	if p.i >= len(p.s) {
+		return nil, errors.New("expected selector, found EOF instead")
+	}
+
+	switch p.s[p.i] {
+	case '*':
+		// It's the universal selector. Just skip over it, since it doesn't affect the meaning.
+		p.i++
+		if p.i+2 < len(p.s) && p.s[p.i:p.i+2] == "|*" { // other version of universal selector
+			p.i += 2
+		}
+	case '#', '.', '[', ':':
+		// There's no type selector. Wait to process the other till the main loop.
+	default:
+		r, err := p.parseTypeSelector()
+		if err != nil {
+			return nil, err
+		}
+		selectors = append(selectors, r)
+	}
+
+	var pseudoElement string
+loop:
+	for p.i < len(p.s) {
+		var (
+			ns               Sel
+			newPseudoElement string
+			err              error
+		)
+		switch p.s[p.i] {
+		case '#':
+			ns, err = p.parseIDSelector()
+		case '.':
+			ns, err = p.parseClassSelector()
+		case '[':
+			ns, err = p.parseAttributeSelector()
+		case ':':
+			ns, newPseudoElement, err = p.parsePseudoclassSelector()
+		default:
+			break loop
+		}
+		if err != nil {
+			return nil, err
+		}
+		// From https://drafts.csswg.org/selectors-3/#pseudo-elements :
+		// "Only one pseudo-element may appear per selector, and if present
+		// it must appear after the sequence of simple selectors that
+		// represents the subjects of the selector.""
+		if ns == nil { // we found a pseudo-element
+			if pseudoElement != "" {
+				return nil, fmt.Errorf("only one pseudo-element is accepted per selector, got %s and %s", pseudoElement, newPseudoElement)
+			}
+			if !p.acceptPseudoElements {
+				return nil, fmt.Errorf("pseudo-element %s found, but pseudo-elements support is disabled", newPseudoElement)
+			}
+			pseudoElement = newPseudoElement
+		} else {
+			if pseudoElement != "" {
+				return nil, fmt.Errorf("pseudo-element %s must be at the end of selector", pseudoElement)
+			}
+			selectors = append(selectors, ns)
+		}
+
+	}
+	if len(selectors) == 1 && pseudoElement == "" { // no need wrap the selectors in compoundSelector
+		return selectors[0], nil
+	}
+	return compoundSelector{selectors: selectors, pseudoElement: pseudoElement}, nil
+}
+
+// parseSelector parses a selector that may include combinators.
+func (p *parser) parseSelector() (Sel, error) {
+	p.skipWhitespace()
+	result, err := p.parseSimpleSelectorSequence()
+	if err != nil {
+		return nil, err
+	}
+
+	for {
+		var (
+			combinator byte
+			c          Sel
+		)
+		if p.skipWhitespace() {
+			combinator = ' '
+		}
+		if p.i >= len(p.s) {
+			return result, nil
+		}
+
+		switch p.s[p.i] {
+		case '+', '>', '~':
+			combinator = p.s[p.i]
+			p.i++
+			p.skipWhitespace()
+		case ',', ')':
+			// These characters can't begin a selector, but they can legally occur after one.
+			return result, nil
+		}
+
+		if combinator == 0 {
+			return result, nil
+		}
+
+		c, err = p.parseSimpleSelectorSequence()
+		if err != nil {
+			return nil, err
+		}
+		result = combinedSelector{first: result, combinator: combinator, second: c}
+	}
+}
+
+// parseSelectorGroup parses a group of selectors, separated by commas.
+func (p *parser) parseSelectorGroup() (SelectorGroup, error) {
+	current, err := p.parseSelector()
+	if err != nil {
+		return nil, err
+	}
+	result := SelectorGroup{current}
+
+	for p.i < len(p.s) {
+		if p.s[p.i] != ',' {
+			break
+		}
+		p.i++
+		c, err := p.parseSelector()
+		if err != nil {
+			return nil, err
+		}
+		result = append(result, c)
+	}
+	return result, nil
+}

vendor/github.com/andybalholm/cascadia/pseudo_classes.go

@@ -0,0 +1,458 @@
+package cascadia
+
+import (
+	"bytes"
+	"fmt"
+	"regexp"
+	"strings"
+
+	"golang.org/x/net/html"
+	"golang.org/x/net/html/atom"
+)
+
+// This file implements the pseudo classes selectors,
+// which share the implementation of PseudoElement() and Specificity()
+
+type abstractPseudoClass struct{}
+
+func (s abstractPseudoClass) Specificity() Specificity {
+	return Specificity{0, 1, 0}
+}
+
+func (c abstractPseudoClass) PseudoElement() string {
+	return ""
+}
+
+type relativePseudoClassSelector struct {
+	name  string // one of "not", "has", "haschild"
+	match SelectorGroup
+}
+
+func (s relativePseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	switch s.name {
+	case "not":
+		// matches elements that do not match a.
+		return !s.match.Match(n)
+	case "has":
+		//  matches elements with any descendant that matches a.
+		return hasDescendantMatch(n, s.match)
+	case "haschild":
+		// matches elements with a child that matches a.
+		return hasChildMatch(n, s.match)
+	default:
+		panic(fmt.Sprintf("unsupported relative pseudo class selector : %s", s.name))
+	}
+}
+
+// hasChildMatch returns whether n has any child that matches a.
+func hasChildMatch(n *html.Node, a SelectorGroup) bool {
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		if a.Match(c) {
+			return true
+		}
+	}
+	return false
+}
+
+// hasDescendantMatch performs a depth-first search of n's descendants,
+// testing whether any of them match a. It returns true as soon as a match is
+// found, or false if no match is found.
+func hasDescendantMatch(n *html.Node, a SelectorGroup) bool {
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		if a.Match(c) || (c.Type == html.ElementNode && hasDescendantMatch(c, a)) {
+			return true
+		}
+	}
+	return false
+}
+
+// Specificity returns the specificity of the most specific selectors
+// in the pseudo-class arguments.
+// See https://www.w3.org/TR/selectors/#specificity-rules
+func (s relativePseudoClassSelector) Specificity() Specificity {
+	var max Specificity
+	for _, sel := range s.match {
+		newSpe := sel.Specificity()
+		if max.Less(newSpe) {
+			max = newSpe
+		}
+	}
+	return max
+}
+
+func (c relativePseudoClassSelector) PseudoElement() string {
+	return ""
+}
+
+type containsPseudoClassSelector struct {
+	abstractPseudoClass
+	value string
+	own   bool
+}
+
+func (s containsPseudoClassSelector) Match(n *html.Node) bool {
+	var text string
+	if s.own {
+		// matches nodes that directly contain the given text
+		text = strings.ToLower(nodeOwnText(n))
+	} else {
+		// matches nodes that contain the given text.
+		text = strings.ToLower(nodeText(n))
+	}
+	return strings.Contains(text, s.value)
+}
+
+type regexpPseudoClassSelector struct {
+	abstractPseudoClass
+	regexp *regexp.Regexp
+	own    bool
+}
+
+func (s regexpPseudoClassSelector) Match(n *html.Node) bool {
+	var text string
+	if s.own {
+		// matches nodes whose text directly matches the specified regular expression
+		text = nodeOwnText(n)
+	} else {
+		// matches nodes whose text matches the specified regular expression
+		text = nodeText(n)
+	}
+	return s.regexp.MatchString(text)
+}
+
+// writeNodeText writes the text contained in n and its descendants to b.
+func writeNodeText(n *html.Node, b *bytes.Buffer) {
+	switch n.Type {
+	case html.TextNode:
+		b.WriteString(n.Data)
+	case html.ElementNode:
+		for c := n.FirstChild; c != nil; c = c.NextSibling {
+			writeNodeText(c, b)
+		}
+	}
+}
+
+// nodeText returns the text contained in n and its descendants.
+func nodeText(n *html.Node) string {
+	var b bytes.Buffer
+	writeNodeText(n, &b)
+	return b.String()
+}
+
+// nodeOwnText returns the contents of the text nodes that are direct
+// children of n.
+func nodeOwnText(n *html.Node) string {
+	var b bytes.Buffer
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		if c.Type == html.TextNode {
+			b.WriteString(c.Data)
+		}
+	}
+	return b.String()
+}
+
+type nthPseudoClassSelector struct {
+	abstractPseudoClass
+	a, b         int
+	last, ofType bool
+}
+
+func (s nthPseudoClassSelector) Match(n *html.Node) bool {
+	if s.a == 0 {
+		if s.last {
+			return simpleNthLastChildMatch(s.b, s.ofType, n)
+		} else {
+			return simpleNthChildMatch(s.b, s.ofType, n)
+		}
+	}
+	return nthChildMatch(s.a, s.b, s.last, s.ofType, n)
+}
+
+// nthChildMatch implements :nth-child(an+b).
+// If last is true, implements :nth-last-child instead.
+// If ofType is true, implements :nth-of-type instead.
+func nthChildMatch(a, b int, last, ofType bool, n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+
+	parent := n.Parent
+	if parent == nil {
+		return false
+	}
+
+	i := -1
+	count := 0
+	for c := parent.FirstChild; c != nil; c = c.NextSibling {
+		if (c.Type != html.ElementNode) || (ofType && c.Data != n.Data) {
+			continue
+		}
+		count++
+		if c == n {
+			i = count
+			if !last {
+				break
+			}
+		}
+	}
+
+	if i == -1 {
+		// This shouldn't happen, since n should always be one of its parent's children.
+		return false
+	}
+
+	if last {
+		i = count - i + 1
+	}
+
+	i -= b
+	if a == 0 {
+		return i == 0
+	}
+
+	return i%a == 0 && i/a >= 0
+}
+
+// simpleNthChildMatch implements :nth-child(b).
+// If ofType is true, implements :nth-of-type instead.
+func simpleNthChildMatch(b int, ofType bool, n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+
+	parent := n.Parent
+	if parent == nil {
+		return false
+	}
+
+	count := 0
+	for c := parent.FirstChild; c != nil; c = c.NextSibling {
+		if c.Type != html.ElementNode || (ofType && c.Data != n.Data) {
+			continue
+		}
+		count++
+		if c == n {
+			return count == b
+		}
+		if count >= b {
+			return false
+		}
+	}
+	return false
+}
+
+// simpleNthLastChildMatch implements :nth-last-child(b).
+// If ofType is true, implements :nth-last-of-type instead.
+func simpleNthLastChildMatch(b int, ofType bool, n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+
+	parent := n.Parent
+	if parent == nil {
+		return false
+	}
+
+	count := 0
+	for c := parent.LastChild; c != nil; c = c.PrevSibling {
+		if c.Type != html.ElementNode || (ofType && c.Data != n.Data) {
+			continue
+		}
+		count++
+		if c == n {
+			return count == b
+		}
+		if count >= b {
+			return false
+		}
+	}
+	return false
+}
+
+type onlyChildPseudoClassSelector struct {
+	abstractPseudoClass
+	ofType bool
+}
+
+// Match implements :only-child.
+// If `ofType` is true, it implements :only-of-type instead.
+func (s onlyChildPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+
+	parent := n.Parent
+	if parent == nil {
+		return false
+	}
+
+	count := 0
+	for c := parent.FirstChild; c != nil; c = c.NextSibling {
+		if (c.Type != html.ElementNode) || (s.ofType && c.Data != n.Data) {
+			continue
+		}
+		count++
+		if count > 1 {
+			return false
+		}
+	}
+
+	return count == 1
+}
+
+type inputPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+// Matches input, select, textarea and button elements.
+func (s inputPseudoClassSelector) Match(n *html.Node) bool {
+	return n.Type == html.ElementNode && (n.Data == "input" || n.Data == "select" || n.Data == "textarea" || n.Data == "button")
+}
+
+type emptyElementPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+// Matches empty elements.
+func (s emptyElementPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		switch c.Type {
+		case html.ElementNode:
+			return false
+		case html.TextNode:
+			if strings.TrimSpace(c.Data) == "" {
+				continue
+			} else {
+				return false
+			}
+		}
+	}
+
+	return true
+}
+
+type rootPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+// Match implements :root
+func (s rootPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	if n.Parent == nil {
+		return false
+	}
+	return n.Parent.Type == html.DocumentNode
+}
+
+func hasAttr(n *html.Node, attr string) bool {
+	return matchAttribute(n, attr, func(string) bool { return true })
+}
+
+type linkPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+// Match implements :link
+func (s linkPseudoClassSelector) Match(n *html.Node) bool {
+	return (n.DataAtom == atom.A || n.DataAtom == atom.Area || n.DataAtom == atom.Link) && hasAttr(n, "href")
+}
+
+type langPseudoClassSelector struct {
+	abstractPseudoClass
+	lang string
+}
+
+func (s langPseudoClassSelector) Match(n *html.Node) bool {
+	own := matchAttribute(n, "lang", func(val string) bool {
+		return val == s.lang || strings.HasPrefix(val, s.lang+"-")
+	})
+	if n.Parent == nil {
+		return own
+	}
+	return own || s.Match(n.Parent)
+}
+
+type enabledPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+func (s enabledPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	switch n.DataAtom {
+	case atom.A, atom.Area, atom.Link:
+		return hasAttr(n, "href")
+	case atom.Optgroup, atom.Menuitem, atom.Fieldset:
+		return !hasAttr(n, "disabled")
+	case atom.Button, atom.Input, atom.Select, atom.Textarea, atom.Option:
+		return !hasAttr(n, "disabled") && !inDisabledFieldset(n)
+	}
+	return false
+}
+
+type disabledPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+func (s disabledPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	switch n.DataAtom {
+	case atom.Optgroup, atom.Menuitem, atom.Fieldset:
+		return hasAttr(n, "disabled")
+	case atom.Button, atom.Input, atom.Select, atom.Textarea, atom.Option:
+		return hasAttr(n, "disabled") || inDisabledFieldset(n)
+	}
+	return false
+}
+
+func hasLegendInPreviousSiblings(n *html.Node) bool {
+	for s := n.PrevSibling; s != nil; s = s.PrevSibling {
+		if s.DataAtom == atom.Legend {
+			return true
+		}
+	}
+	return false
+}
+
+func inDisabledFieldset(n *html.Node) bool {
+	if n.Parent == nil {
+		return false
+	}
+	if n.Parent.DataAtom == atom.Fieldset && hasAttr(n.Parent, "disabled") &&
+		(n.DataAtom != atom.Legend || hasLegendInPreviousSiblings(n)) {
+		return true
+	}
+	return inDisabledFieldset(n.Parent)
+}
+
+type checkedPseudoClassSelector struct {
+	abstractPseudoClass
+}
+
+func (s checkedPseudoClassSelector) Match(n *html.Node) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	switch n.DataAtom {
+	case atom.Input, atom.Menuitem:
+		return hasAttr(n, "checked") && matchAttribute(n, "type", func(val string) bool {
+			t := toLowerASCII(val)
+			return t == "checkbox" || t == "radio"
+		})
+	case atom.Option:
+		return hasAttr(n, "selected")
+	}
+	return false
+}

vendor/github.com/andybalholm/cascadia/selector.go

@@ -0,0 +1,586 @@
+package cascadia
+
+import (
+	"fmt"
+	"regexp"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+// Matcher is the interface for basic selector functionality.
+// Match returns whether a selector matches n.
+type Matcher interface {
+	Match(n *html.Node) bool
+}
+
+// Sel is the interface for all the functionality provided by selectors.
+type Sel interface {
+	Matcher
+	Specificity() Specificity
+
+	// Returns a CSS input compiling to this selector.
+	String() string
+
+	// Returns a pseudo-element, or an empty string.
+	PseudoElement() string
+}
+
+// Parse parses a selector. Use `ParseWithPseudoElement`
+// if you need support for pseudo-elements.
+func Parse(sel string) (Sel, error) {
+	p := &parser{s: sel}
+	compiled, err := p.parseSelector()
+	if err != nil {
+		return nil, err
+	}
+
+	if p.i < len(sel) {
+		return nil, fmt.Errorf("parsing %q: %d bytes left over", sel, len(sel)-p.i)
+	}
+
+	return compiled, nil
+}
+
+// ParseWithPseudoElement parses a single selector,
+// with support for pseudo-element.
+func ParseWithPseudoElement(sel string) (Sel, error) {
+	p := &parser{s: sel, acceptPseudoElements: true}
+	compiled, err := p.parseSelector()
+	if err != nil {
+		return nil, err
+	}
+
+	if p.i < len(sel) {
+		return nil, fmt.Errorf("parsing %q: %d bytes left over", sel, len(sel)-p.i)
+	}
+
+	return compiled, nil
+}
+
+// ParseGroup parses a selector, or a group of selectors separated by commas.
+// Use `ParseGroupWithPseudoElements`
+// if you need support for pseudo-elements.
+func ParseGroup(sel string) (SelectorGroup, error) {
+	p := &parser{s: sel}
+	compiled, err := p.parseSelectorGroup()
+	if err != nil {
+		return nil, err
+	}
+
+	if p.i < len(sel) {
+		return nil, fmt.Errorf("parsing %q: %d bytes left over", sel, len(sel)-p.i)
+	}
+
+	return compiled, nil
+}
+
+// ParseGroupWithPseudoElements parses a selector, or a group of selectors separated by commas.
+// It supports pseudo-elements.
+func ParseGroupWithPseudoElements(sel string) (SelectorGroup, error) {
+	p := &parser{s: sel, acceptPseudoElements: true}
+	compiled, err := p.parseSelectorGroup()
+	if err != nil {
+		return nil, err
+	}
+
+	if p.i < len(sel) {
+		return nil, fmt.Errorf("parsing %q: %d bytes left over", sel, len(sel)-p.i)
+	}
+
+	return compiled, nil
+}
+
+// A Selector is a function which tells whether a node matches or not.
+//
+// This type is maintained for compatibility; I recommend using the newer and
+// more idiomatic interfaces Sel and Matcher.
+type Selector func(*html.Node) bool
+
+// Compile parses a selector and returns, if successful, a Selector object
+// that can be used to match against html.Node objects.
+func Compile(sel string) (Selector, error) {
+	compiled, err := ParseGroup(sel)
+	if err != nil {
+		return nil, err
+	}
+
+	return Selector(compiled.Match), nil
+}
+
+// MustCompile is like Compile, but panics instead of returning an error.
+func MustCompile(sel string) Selector {
+	compiled, err := Compile(sel)
+	if err != nil {
+		panic(err)
+	}
+	return compiled
+}
+
+// MatchAll returns a slice of the nodes that match the selector,
+// from n and its children.
+func (s Selector) MatchAll(n *html.Node) []*html.Node {
+	return s.matchAllInto(n, nil)
+}
+
+func (s Selector) matchAllInto(n *html.Node, storage []*html.Node) []*html.Node {
+	if s(n) {
+		storage = append(storage, n)
+	}
+
+	for child := n.FirstChild; child != nil; child = child.NextSibling {
+		storage = s.matchAllInto(child, storage)
+	}
+
+	return storage
+}
+
+func queryInto(n *html.Node, m Matcher, storage []*html.Node) []*html.Node {
+	for child := n.FirstChild; child != nil; child = child.NextSibling {
+		if m.Match(child) {
+			storage = append(storage, child)
+		}
+		storage = queryInto(child, m, storage)
+	}
+
+	return storage
+}
+
+// QueryAll returns a slice of all the nodes that match m, from the descendants
+// of n.
+func QueryAll(n *html.Node, m Matcher) []*html.Node {
+	return queryInto(n, m, nil)
+}
+
+// Match returns true if the node matches the selector.
+func (s Selector) Match(n *html.Node) bool {
+	return s(n)
+}
+
+// MatchFirst returns the first node that matches s, from n and its children.
+func (s Selector) MatchFirst(n *html.Node) *html.Node {
+	if s.Match(n) {
+		return n
+	}
+
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		m := s.MatchFirst(c)
+		if m != nil {
+			return m
+		}
+	}
+	return nil
+}
+
+// Query returns the first node that matches m, from the descendants of n.
+// If none matches, it returns nil.
+func Query(n *html.Node, m Matcher) *html.Node {
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		if m.Match(c) {
+			return c
+		}
+		if matched := Query(c, m); matched != nil {
+			return matched
+		}
+	}
+
+	return nil
+}
+
+// Filter returns the nodes in nodes that match the selector.
+func (s Selector) Filter(nodes []*html.Node) (result []*html.Node) {
+	for _, n := range nodes {
+		if s(n) {
+			result = append(result, n)
+		}
+	}
+	return result
+}
+
+// Filter returns the nodes that match m.
+func Filter(nodes []*html.Node, m Matcher) (result []*html.Node) {
+	for _, n := range nodes {
+		if m.Match(n) {
+			result = append(result, n)
+		}
+	}
+	return result
+}
+
+type tagSelector struct {
+	tag string
+}
+
+// Matches elements with a given tag name.
+func (t tagSelector) Match(n *html.Node) bool {
+	return n.Type == html.ElementNode && n.Data == t.tag
+}
+
+func (c tagSelector) Specificity() Specificity {
+	return Specificity{0, 0, 1}
+}
+
+func (c tagSelector) PseudoElement() string {
+	return ""
+}
+
+type classSelector struct {
+	class string
+}
+
+// Matches elements by class attribute.
+func (t classSelector) Match(n *html.Node) bool {
+	return matchAttribute(n, "class", func(s string) bool {
+		return matchInclude(t.class, s, false)
+	})
+}
+
+func (c classSelector) Specificity() Specificity {
+	return Specificity{0, 1, 0}
+}
+
+func (c classSelector) PseudoElement() string {
+	return ""
+}
+
+type idSelector struct {
+	id string
+}
+
+// Matches elements by id attribute.
+func (t idSelector) Match(n *html.Node) bool {
+	return matchAttribute(n, "id", func(s string) bool {
+		return s == t.id
+	})
+}
+
+func (c idSelector) Specificity() Specificity {
+	return Specificity{1, 0, 0}
+}
+
+func (c idSelector) PseudoElement() string {
+	return ""
+}
+
+type attrSelector struct {
+	key, val, operation string
+	regexp              *regexp.Regexp
+	insensitive         bool
+}
+
+// Matches elements by attribute value.
+func (t attrSelector) Match(n *html.Node) bool {
+	switch t.operation {
+	case "":
+		return matchAttribute(n, t.key, func(string) bool { return true })
+	case "=":
+		return matchAttribute(n, t.key, func(s string) bool { return matchInsensitiveValue(s, t.val, t.insensitive) })
+	case "!=":
+		return attributeNotEqualMatch(t.key, t.val, n, t.insensitive)
+	case "~=":
+		// matches elements where the attribute named key is a whitespace-separated list that includes val.
+		return matchAttribute(n, t.key, func(s string) bool { return matchInclude(t.val, s, t.insensitive) })
+	case "|=":
+		return attributeDashMatch(t.key, t.val, n, t.insensitive)
+	case "^=":
+		return attributePrefixMatch(t.key, t.val, n, t.insensitive)
+	case "$=":
+		return attributeSuffixMatch(t.key, t.val, n, t.insensitive)
+	case "*=":
+		return attributeSubstringMatch(t.key, t.val, n, t.insensitive)
+	case "#=":
+		return attributeRegexMatch(t.key, t.regexp, n)
+	default:
+		panic(fmt.Sprintf("unsuported operation : %s", t.operation))
+	}
+}
+
+// matches elements where we ignore (or not) the case of the attribute value
+// the user attribute is the value set by the user to match elements
+// the real attribute is the attribute value found in the code parsed
+func matchInsensitiveValue(userAttr string, realAttr string, ignoreCase bool) bool {
+	if ignoreCase {
+		return strings.EqualFold(userAttr, realAttr)
+	}
+	return userAttr == realAttr
+
+}
+
+// matches elements where the attribute named key satisifes the function f.
+func matchAttribute(n *html.Node, key string, f func(string) bool) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	for _, a := range n.Attr {
+		if a.Key == key && f(a.Val) {
+			return true
+		}
+	}
+	return false
+}
+
+// attributeNotEqualMatch matches elements where
+// the attribute named key does not have the value val.
+func attributeNotEqualMatch(key, val string, n *html.Node, ignoreCase bool) bool {
+	if n.Type != html.ElementNode {
+		return false
+	}
+	for _, a := range n.Attr {
+		if a.Key == key && matchInsensitiveValue(a.Val, val, ignoreCase) {
+			return false
+		}
+	}
+	return true
+}
+
+// returns true if s is a whitespace-separated list that includes val.
+func matchInclude(val string, s string, ignoreCase bool) bool {
+	for s != "" {
+		i := strings.IndexAny(s, " \t\r\n\f")
+		if i == -1 {
+			return matchInsensitiveValue(s, val, ignoreCase)
+		}
+		if matchInsensitiveValue(s[:i], val, ignoreCase) {
+			return true
+		}
+		s = s[i+1:]
+	}
+	return false
+}
+
+//  matches elements where the attribute named key equals val or starts with val plus a hyphen.
+func attributeDashMatch(key, val string, n *html.Node, ignoreCase bool) bool {
+	return matchAttribute(n, key,
+		func(s string) bool {
+			if matchInsensitiveValue(s, val, ignoreCase) {
+				return true
+			}
+			if len(s) <= len(val) {
+				return false
+			}
+			if matchInsensitiveValue(s[:len(val)], val, ignoreCase) && s[len(val)] == '-' {
+				return true
+			}
+			return false
+		})
+}
+
+// attributePrefixMatch returns a Selector that matches elements where
+// the attribute named key starts with val.
+func attributePrefixMatch(key, val string, n *html.Node, ignoreCase bool) bool {
+	return matchAttribute(n, key,
+		func(s string) bool {
+			if strings.TrimSpace(s) == "" {
+				return false
+			}
+			if ignoreCase {
+				return strings.HasPrefix(strings.ToLower(s), strings.ToLower(val))
+			}
+			return strings.HasPrefix(s, val)
+		})
+}
+
+// attributeSuffixMatch matches elements where
+// the attribute named key ends with val.
+func attributeSuffixMatch(key, val string, n *html.Node, ignoreCase bool) bool {
+	return matchAttribute(n, key,
+		func(s string) bool {
+			if strings.TrimSpace(s) == "" {
+				return false
+			}
+			if ignoreCase {
+				return strings.HasSuffix(strings.ToLower(s), strings.ToLower(val))
+			}
+			return strings.HasSuffix(s, val)
+		})
+}
+
+// attributeSubstringMatch matches nodes where
+// the attribute named key contains val.
+func attributeSubstringMatch(key, val string, n *html.Node, ignoreCase bool) bool {
+	return matchAttribute(n, key,
+		func(s string) bool {
+			if strings.TrimSpace(s) == "" {
+				return false
+			}
+			if ignoreCase {
+				return strings.Contains(strings.ToLower(s), strings.ToLower(val))
+			}
+			return strings.Contains(s, val)
+		})
+}
+
+// attributeRegexMatch  matches nodes where
+// the attribute named key matches the regular expression rx
+func attributeRegexMatch(key string, rx *regexp.Regexp, n *html.Node) bool {
+	return matchAttribute(n, key,
+		func(s string) bool {
+			return rx.MatchString(s)
+		})
+}
+
+func (c attrSelector) Specificity() Specificity {
+	return Specificity{0, 1, 0}
+}
+
+func (c attrSelector) PseudoElement() string {
+	return ""
+}
+
+// see pseudo_classes.go for pseudo classes selectors
+
+// on a static context, some selectors can't match anything
+type neverMatchSelector struct {
+	value string
+}
+
+func (s neverMatchSelector) Match(n *html.Node) bool {
+	return false
+}
+
+func (s neverMatchSelector) Specificity() Specificity {
+	return Specificity{0, 0, 0}
+}
+
+func (c neverMatchSelector) PseudoElement() string {
+	return ""
+}
+
+type compoundSelector struct {
+	selectors     []Sel
+	pseudoElement string
+}
+
+// Matches elements if each sub-selectors matches.
+func (t compoundSelector) Match(n *html.Node) bool {
+	if len(t.selectors) == 0 {
+		return n.Type == html.ElementNode
+	}
+
+	for _, sel := range t.selectors {
+		if !sel.Match(n) {
+			return false
+		}
+	}
+	return true
+}
+
+func (s compoundSelector) Specificity() Specificity {
+	var out Specificity
+	for _, sel := range s.selectors {
+		out = out.Add(sel.Specificity())
+	}
+	if s.pseudoElement != "" {
+		// https://drafts.csswg.org/selectors-3/#specificity
+		out = out.Add(Specificity{0, 0, 1})
+	}
+	return out
+}
+
+func (c compoundSelector) PseudoElement() string {
+	return c.pseudoElement
+}
+
+type combinedSelector struct {
+	first      Sel
+	combinator byte
+	second     Sel
+}
+
+func (t combinedSelector) Match(n *html.Node) bool {
+	if t.first == nil {
+		return false // maybe we should panic
+	}
+	switch t.combinator {
+	case 0:
+		return t.first.Match(n)
+	case ' ':
+		return descendantMatch(t.first, t.second, n)
+	case '>':
+		return childMatch(t.first, t.second, n)
+	case '+':
+		return siblingMatch(t.first, t.second, true, n)
+	case '~':
+		return siblingMatch(t.first, t.second, false, n)
+	default:
+		panic("unknown combinator")
+	}
+}
+
+// matches an element if it matches d and has an ancestor that matches a.
+func descendantMatch(a, d Matcher, n *html.Node) bool {
+	if !d.Match(n) {
+		return false
+	}
+
+	for p := n.Parent; p != nil; p = p.Parent {
+		if a.Match(p) {
+			return true
+		}
+	}
+
+	return false
+}
+
+// matches an element if it matches d and its parent matches a.
+func childMatch(a, d Matcher, n *html.Node) bool {
+	return d.Match(n) && n.Parent != nil && a.Match(n.Parent)
+}
+
+// matches an element if it matches s2 and is preceded by an element that matches s1.
+// If adjacent is true, the sibling must be immediately before the element.
+func siblingMatch(s1, s2 Matcher, adjacent bool, n *html.Node) bool {
+	if !s2.Match(n) {
+		return false
+	}
+
+	if adjacent {
+		for n = n.PrevSibling; n != nil; n = n.PrevSibling {
+			if n.Type == html.TextNode || n.Type == html.CommentNode {
+				continue
+			}
+			return s1.Match(n)
+		}
+		return false
+	}
+
+	// Walk backwards looking for element that matches s1
+	for c := n.PrevSibling; c != nil; c = c.PrevSibling {
+		if s1.Match(c) {
+			return true
+		}
+	}
+
+	return false
+}
+
+func (s combinedSelector) Specificity() Specificity {
+	spec := s.first.Specificity()
+	if s.second != nil {
+		spec = spec.Add(s.second.Specificity())
+	}
+	return spec
+}
+
+// on combinedSelector, a pseudo-element only makes sens on the last
+// selector, although others increase specificity.
+func (c combinedSelector) PseudoElement() string {
+	if c.second == nil {
+		return ""
+	}
+	return c.second.PseudoElement()
+}
+
+// A SelectorGroup is a list of selectors, which matches if any of the
+// individual selectors matches.
+type SelectorGroup []Sel
+
+// Match returns true if the node matches one of the single selectors.
+func (s SelectorGroup) Match(n *html.Node) bool {
+	for _, sel := range s {
+		if sel.Match(n) {
+			return true
+		}
+	}
+	return false
+}

vendor/github.com/andybalholm/cascadia/serialize.go

@@ -0,0 +1,176 @@
+package cascadia
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+// implements the reverse operation Sel -> string
+
+var specialCharReplacer *strings.Replacer
+
+func init() {
+	var pairs []string
+	for _, s := range ",!\"#$%&'()*+ -./:;<=>?@[\\]^`{|}~" {
+		pairs = append(pairs, string(s), "\\"+string(s))
+	}
+	specialCharReplacer = strings.NewReplacer(pairs...)
+}
+
+// espace special CSS char
+func escape(s string) string { return specialCharReplacer.Replace(s) }
+
+func (c tagSelector) String() string {
+	return c.tag
+}
+
+func (c idSelector) String() string {
+	return "#" + escape(c.id)
+}
+
+func (c classSelector) String() string {
+	return "." + escape(c.class)
+}
+
+func (c attrSelector) String() string {
+	val := c.val
+	if c.operation == "#=" {
+		val = c.regexp.String()
+	} else if c.operation != "" {
+		val = fmt.Sprintf(`"%s"`, val)
+	}
+
+	ignoreCase := ""
+
+	if c.insensitive {
+		ignoreCase = " i"
+	}
+
+	return fmt.Sprintf(`[%s%s%s%s]`, c.key, c.operation, val, ignoreCase)
+}
+
+func (c relativePseudoClassSelector) String() string {
+	return fmt.Sprintf(":%s(%s)", c.name, c.match.String())
+}
+
+func (c containsPseudoClassSelector) String() string {
+	s := "contains"
+	if c.own {
+		s += "Own"
+	}
+	return fmt.Sprintf(`:%s("%s")`, s, c.value)
+}
+
+func (c regexpPseudoClassSelector) String() string {
+	s := "matches"
+	if c.own {
+		s += "Own"
+	}
+	return fmt.Sprintf(":%s(%s)", s, c.regexp.String())
+}
+
+func (c nthPseudoClassSelector) String() string {
+	if c.a == 0 && c.b == 1 { // special cases
+		s := ":first-"
+		if c.last {
+			s = ":last-"
+		}
+		if c.ofType {
+			s += "of-type"
+		} else {
+			s += "child"
+		}
+		return s
+	}
+	var name string
+	switch [2]bool{c.last, c.ofType} {
+	case [2]bool{true, true}:
+		name = "nth-last-of-type"
+	case [2]bool{true, false}:
+		name = "nth-last-child"
+	case [2]bool{false, true}:
+		name = "nth-of-type"
+	case [2]bool{false, false}:
+		name = "nth-child"
+	}
+	s := fmt.Sprintf("+%d", c.b)
+	if c.b < 0 { // avoid +-8 invalid syntax
+		s = strconv.Itoa(c.b)
+	}
+	return fmt.Sprintf(":%s(%dn%s)", name, c.a, s)
+}
+
+func (c onlyChildPseudoClassSelector) String() string {
+	if c.ofType {
+		return ":only-of-type"
+	}
+	return ":only-child"
+}
+
+func (c inputPseudoClassSelector) String() string {
+	return ":input"
+}
+
+func (c emptyElementPseudoClassSelector) String() string {
+	return ":empty"
+}
+
+func (c rootPseudoClassSelector) String() string {
+	return ":root"
+}
+
+func (c linkPseudoClassSelector) String() string {
+	return ":link"
+}
+
+func (c langPseudoClassSelector) String() string {
+	return fmt.Sprintf(":lang(%s)", c.lang)
+}
+
+func (c neverMatchSelector) String() string {
+	return c.value
+}
+
+func (c enabledPseudoClassSelector) String() string {
+	return ":enabled"
+}
+
+func (c disabledPseudoClassSelector) String() string {
+	return ":disabled"
+}
+
+func (c checkedPseudoClassSelector) String() string {
+	return ":checked"
+}
+
+func (c compoundSelector) String() string {
+	if len(c.selectors) == 0 && c.pseudoElement == "" {
+		return "*"
+	}
+	chunks := make([]string, len(c.selectors))
+	for i, sel := range c.selectors {
+		chunks[i] = sel.String()
+	}
+	s := strings.Join(chunks, "")
+	if c.pseudoElement != "" {
+		s += "::" + c.pseudoElement
+	}
+	return s
+}
+
+func (c combinedSelector) String() string {
+	start := c.first.String()
+	if c.second != nil {
+		start += fmt.Sprintf(" %s %s", string(c.combinator), c.second.String())
+	}
+	return start
+}
+
+func (c SelectorGroup) String() string {
+	ck := make([]string, len(c))
+	for i, s := range c {
+		ck[i] = s.String()
+	}
+	return strings.Join(ck, ", ")
+}

vendor/github.com/andybalholm/cascadia/specificity.go

@@ -0,0 +1,26 @@
+package cascadia
+
+// Specificity is the CSS specificity as defined in
+// https://www.w3.org/TR/selectors/#specificity-rules
+// with the convention Specificity = [A,B,C].
+type Specificity [3]int
+
+// returns `true` if s < other (strictly), false otherwise
+func (s Specificity) Less(other Specificity) bool {
+	for i := range s {
+		if s[i] < other[i] {
+			return true
+		}
+		if s[i] > other[i] {
+			return false
+		}
+	}
+	return false
+}
+
+func (s Specificity) Add(other Specificity) Specificity {
+	for i, sp := range other {
+		s[i] += sp
+	}
+	return s
+}

vendor/github.com/araddon/dateparse/.travis.yml

@@ -0,0 +1,13 @@
+language: go
+
+go:
+  - 1.13.x
+
+before_install:
+  - go get -t -v ./...
+
+script:
+  - go test -race -coverprofile=coverage.txt -covermode=atomic
+
+after_success:
+  - bash <(curl -s https://codecov.io/bash)

vendor/github.com/araddon/dateparse/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2015-2017 Aaron Raddon
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

vendor/github.com/araddon/dateparse/README.md

@@ -0,0 +1,323 @@
+Go Date Parser 
+---------------------------
+
+Parse many date strings without knowing format in advance.  Uses a scanner to read bytes and use a state machine to find format.  Much faster than shotgun based parse methods.  See [bench_test.go](https://github.com/araddon/dateparse/blob/master/bench_test.go) for performance comparison.
+
+
+[![Code Coverage](https://codecov.io/gh/araddon/dateparse/branch/master/graph/badge.svg)](https://codecov.io/gh/araddon/dateparse)
+[![GoDoc](https://godoc.org/github.com/araddon/dateparse?status.svg)](http://godoc.org/github.com/araddon/dateparse)
+[![Build Status](https://travis-ci.org/araddon/dateparse.svg?branch=master)](https://travis-ci.org/araddon/dateparse)
+[![Go ReportCard](https://goreportcard.com/badge/araddon/dateparse)](https://goreportcard.com/report/araddon/dateparse)
+
+**MM/DD/YYYY VS DD/MM/YYYY** Right now this uses mm/dd/yyyy WHEN ambiguous if this is not desired behavior, use `ParseStrict` which will fail on ambiguous date strings.
+
+**Timezones** The location your server is configured affects the results!  See example or https://play.golang.org/p/IDHRalIyXh and last paragraph here https://golang.org/pkg/time/#Parse.
+
+
+```go
+
+// Normal parse.  Equivalent Timezone rules as time.Parse()
+t, err := dateparse.ParseAny("3/1/2014")
+
+// Parse Strict, error on ambigous mm/dd vs dd/mm dates
+t, err := dateparse.ParseStrict("3/1/2014")
+> returns error 
+
+// Return a string that represents the layout to parse the given date-time.
+layout, err := dateparse.ParseFormat("May 8, 2009 5:57:51 PM")
+> "Jan 2, 2006 3:04:05 PM"
+
+```
+
+cli tool for testing dateformats
+----------------------------------
+
+[Date Parse CLI](https://github.com/araddon/dateparse/blob/master/dateparse)
+
+
+Extended example
+-------------------
+
+https://github.com/araddon/dateparse/blob/master/example/main.go
+
+```go
+package main
+
+import (
+	"flag"
+	"fmt"
+	"time"
+
+	"github.com/scylladb/termtables"
+	"github.com/araddon/dateparse"
+)
+
+var examples = []string{
+	"May 8, 2009 5:57:51 PM",
+	"oct 7, 1970",
+	"oct 7, '70",
+	"oct. 7, 1970",
+	"oct. 7, 70",
+	"Mon Jan  2 15:04:05 2006",
+	"Mon Jan  2 15:04:05 MST 2006",
+	"Mon Jan 02 15:04:05 -0700 2006",
+	"Monday, 02-Jan-06 15:04:05 MST",
+	"Mon, 02 Jan 2006 15:04:05 MST",
+	"Tue, 11 Jul 2017 16:28:13 +0200 (CEST)",
+	"Mon, 02 Jan 2006 15:04:05 -0700",
+	"Mon 30 Sep 2018 09:09:09 PM UTC",
+	"Mon Aug 10 15:44:11 UTC+0100 2015",
+	"Thu, 4 Jan 2018 17:53:36 +0000",
+	"Fri Jul 03 2015 18:04:07 GMT+0100 (GMT Daylight Time)",
+	"Sun, 3 Jan 2021 00:12:23 +0800 (GMT+08:00)",
+	"September 17, 2012 10:09am",
+	"September 17, 2012 at 10:09am PST-08",
+	"September 17, 2012, 10:10:09",
+	"October 7, 1970",
+	"October 7th, 1970",
+	"12 Feb 2006, 19:17",
+	"12 Feb 2006 19:17",
+	"14 May 2019 19:11:40.164",
+	"7 oct 70",
+	"7 oct 1970",
+	"03 February 2013",
+	"1 July 2013",
+	"2013-Feb-03",
+	// dd/Mon/yyy  alpha Months
+	"06/Jan/2008:15:04:05 -0700",
+	"06/Jan/2008 15:04:05 -0700",
+	//   mm/dd/yy
+	"3/31/2014",
+	"03/31/2014",
+	"08/21/71",
+	"8/1/71",
+	"4/8/2014 22:05",
+	"04/08/2014 22:05",
+	"4/8/14 22:05",
+	"04/2/2014 03:00:51",
+	"8/8/1965 12:00:00 AM",
+	"8/8/1965 01:00:01 PM",
+	"8/8/1965 01:00 PM",
+	"8/8/1965 1:00 PM",
+	"8/8/1965 12:00 AM",
+	"4/02/2014 03:00:51",
+	"03/19/2012 10:11:59",
+	"03/19/2012 10:11:59.3186369",
+	// yyyy/mm/dd
+	"2014/3/31",
+	"2014/03/31",
+	"2014/4/8 22:05",
+	"2014/04/08 22:05",
+	"2014/04/2 03:00:51",
+	"2014/4/02 03:00:51",
+	"2012/03/19 10:11:59",
+	"2012/03/19 10:11:59.3186369",
+	// yyyy:mm:dd
+	"2014:3:31",
+	"2014:03:31",
+	"2014:4:8 22:05",
+	"2014:04:08 22:05",
+	"2014:04:2 03:00:51",
+	"2014:4:02 03:00:51",
+	"2012:03:19 10:11:59",
+	"2012:03:19 10:11:59.3186369",
+	// Chinese
+	"2014年04月08日",
+	//   yyyy-mm-ddThh
+	"2006-01-02T15:04:05+0000",
+	"2009-08-12T22:15:09-07:00",
+	"2009-08-12T22:15:09",
+	"2009-08-12T22:15:09.988",
+	"2009-08-12T22:15:09Z",
+	"2017-07-19T03:21:51:897+0100",
+	"2019-05-29T08:41-04", // no seconds, 2 digit TZ offset
+	//   yyyy-mm-dd hh:mm:ss
+	"2014-04-26 17:24:37.3186369",
+	"2012-08-03 18:31:59.257000000",
+	"2014-04-26 17:24:37.123",
+	"2013-04-01 22:43",
+	"2013-04-01 22:43:22",
+	"2014-12-16 06:20:00 UTC",
+	"2014-12-16 06:20:00 GMT",
+	"2014-04-26 05:24:37 PM",
+	"2014-04-26 13:13:43 +0800",
+	"2014-04-26 13:13:43 +0800 +08",
+	"2014-04-26 13:13:44 +09:00",
+	"2012-08-03 18:31:59.257000000 +0000 UTC",
+	"2015-09-30 18:48:56.35272715 +0000 UTC",
+	"2015-02-18 00:12:00 +0000 GMT",
+	"2015-02-18 00:12:00 +0000 UTC",
+	"2015-02-08 03:02:00 +0300 MSK m=+0.000000001",
+	"2015-02-08 03:02:00.001 +0300 MSK m=+0.000000001",
+	"2017-07-19 03:21:51+00:00",
+	"2014-04-26",
+	"2014-04",
+	"2014",
+	"2014-05-11 08:20:13,787",
+	// yyyy-mm-dd-07:00
+	"2020-07-20+08:00",
+	// mm.dd.yy
+	"3.31.2014",
+	"03.31.2014",
+	"08.21.71",
+	"2014.03",
+	"2014.03.30",
+	//  yyyymmdd and similar
+	"20140601",
+	"20140722105203",
+	// yymmdd hh:mm:yy  mysql log
+	// 080313 05:21:55 mysqld started
+	"171113 14:14:20",
+	// unix seconds, ms, micro, nano
+	"1332151919",
+	"1384216367189",
+	"1384216367111222",
+	"1384216367111222333",
+}
+
+var (
+	timezone = ""
+)
+
+func main() {
+	flag.StringVar(&timezone, "timezone", "UTC", "Timezone aka `America/Los_Angeles` formatted time-zone")
+	flag.Parse()
+
+	if timezone != "" {
+		// NOTE:  This is very, very important to understand
+		// time-parsing in go
+		loc, err := time.LoadLocation(timezone)
+		if err != nil {
+			panic(err.Error())
+		}
+		time.Local = loc
+	}
+
+	table := termtables.CreateTable()
+
+	table.AddHeaders("Input", "Parsed, and Output as %v")
+	for _, dateExample := range examples {
+		t, err := dateparse.ParseLocal(dateExample)
+		if err != nil {
+			panic(err.Error())
+		}
+		table.AddRow(dateExample, fmt.Sprintf("%v", t))
+	}
+	fmt.Println(table.Render())
+}
+
+/*
++-------------------------------------------------------+-----------------------------------------+
+| Input                                                 | Parsed, and Output as %v                |
++-------------------------------------------------------+-----------------------------------------+
+| May 8, 2009 5:57:51 PM                                | 2009-05-08 17:57:51 +0000 UTC           |
+| oct 7, 1970                                           | 1970-10-07 00:00:00 +0000 UTC           |
+| oct 7, '70                                            | 1970-10-07 00:00:00 +0000 UTC           |
+| oct. 7, 1970                                          | 1970-10-07 00:00:00 +0000 UTC           |
+| oct. 7, 70                                            | 1970-10-07 00:00:00 +0000 UTC           |
+| Mon Jan  2 15:04:05 2006                              | 2006-01-02 15:04:05 +0000 UTC           |
+| Mon Jan  2 15:04:05 MST 2006                          | 2006-01-02 15:04:05 +0000 MST           |
+| Mon Jan 02 15:04:05 -0700 2006                        | 2006-01-02 15:04:05 -0700 -0700         |
+| Monday, 02-Jan-06 15:04:05 MST                        | 2006-01-02 15:04:05 +0000 MST           |
+| Mon, 02 Jan 2006 15:04:05 MST                         | 2006-01-02 15:04:05 +0000 MST           |
+| Tue, 11 Jul 2017 16:28:13 +0200 (CEST)                | 2017-07-11 16:28:13 +0200 +0200         |
+| Mon, 02 Jan 2006 15:04:05 -0700                       | 2006-01-02 15:04:05 -0700 -0700         |
+| Mon 30 Sep 2018 09:09:09 PM UTC                       | 2018-09-30 21:09:09 +0000 UTC           |
+| Mon Aug 10 15:44:11 UTC+0100 2015                     | 2015-08-10 15:44:11 +0000 UTC           |
+| Thu, 4 Jan 2018 17:53:36 +0000                        | 2018-01-04 17:53:36 +0000 UTC           |
+| Fri Jul 03 2015 18:04:07 GMT+0100 (GMT Daylight Time) | 2015-07-03 18:04:07 +0100 GMT           |
+| Sun, 3 Jan 2021 00:12:23 +0800 (GMT+08:00)            | 2021-01-03 00:12:23 +0800 +0800         |
+| September 17, 2012 10:09am                            | 2012-09-17 10:09:00 +0000 UTC           |
+| September 17, 2012 at 10:09am PST-08                  | 2012-09-17 10:09:00 -0800 PST           |
+| September 17, 2012, 10:10:09                          | 2012-09-17 10:10:09 +0000 UTC           |
+| October 7, 1970                                       | 1970-10-07 00:00:00 +0000 UTC           |
+| October 7th, 1970                                     | 1970-10-07 00:00:00 +0000 UTC           |
+| 12 Feb 2006, 19:17                                    | 2006-02-12 19:17:00 +0000 UTC           |
+| 12 Feb 2006 19:17                                     | 2006-02-12 19:17:00 +0000 UTC           |
+| 14 May 2019 19:11:40.164                              | 2019-05-14 19:11:40.164 +0000 UTC       |
+| 7 oct 70                                              | 1970-10-07 00:00:00 +0000 UTC           |
+| 7 oct 1970                                            | 1970-10-07 00:00:00 +0000 UTC           |
+| 03 February 2013                                      | 2013-02-03 00:00:00 +0000 UTC           |
+| 1 July 2013                                           | 2013-07-01 00:00:00 +0000 UTC           |
+| 2013-Feb-03                                           | 2013-02-03 00:00:00 +0000 UTC           |
+| 06/Jan/2008:15:04:05 -0700                            | 2008-01-06 15:04:05 -0700 -0700         |
+| 06/Jan/2008 15:04:05 -0700                            | 2008-01-06 15:04:05 -0700 -0700         |
+| 3/31/2014                                             | 2014-03-31 00:00:00 +0000 UTC           |
+| 03/31/2014                                            | 2014-03-31 00:00:00 +0000 UTC           |
+| 08/21/71                                              | 1971-08-21 00:00:00 +0000 UTC           |
+| 8/1/71                                                | 1971-08-01 00:00:00 +0000 UTC           |
+| 4/8/2014 22:05                                        | 2014-04-08 22:05:00 +0000 UTC           |
+| 04/08/2014 22:05                                      | 2014-04-08 22:05:00 +0000 UTC           |
+| 4/8/14 22:05                                          | 2014-04-08 22:05:00 +0000 UTC           |
+| 04/2/2014 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 8/8/1965 12:00:00 AM                                  | 1965-08-08 00:00:00 +0000 UTC           |
+| 8/8/1965 01:00:01 PM                                  | 1965-08-08 13:00:01 +0000 UTC           |
+| 8/8/1965 01:00 PM                                     | 1965-08-08 13:00:00 +0000 UTC           |
+| 8/8/1965 1:00 PM                                      | 1965-08-08 13:00:00 +0000 UTC           |
+| 8/8/1965 12:00 AM                                     | 1965-08-08 00:00:00 +0000 UTC           |
+| 4/02/2014 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 03/19/2012 10:11:59                                   | 2012-03-19 10:11:59 +0000 UTC           |
+| 03/19/2012 10:11:59.3186369                           | 2012-03-19 10:11:59.3186369 +0000 UTC   |
+| 2014/3/31                                             | 2014-03-31 00:00:00 +0000 UTC           |
+| 2014/03/31                                            | 2014-03-31 00:00:00 +0000 UTC           |
+| 2014/4/8 22:05                                        | 2014-04-08 22:05:00 +0000 UTC           |
+| 2014/04/08 22:05                                      | 2014-04-08 22:05:00 +0000 UTC           |
+| 2014/04/2 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 2014/4/02 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 2012/03/19 10:11:59                                   | 2012-03-19 10:11:59 +0000 UTC           |
+| 2012/03/19 10:11:59.3186369                           | 2012-03-19 10:11:59.3186369 +0000 UTC   |
+| 2014:3:31                                             | 2014-03-31 00:00:00 +0000 UTC           |
+| 2014:03:31                                            | 2014-03-31 00:00:00 +0000 UTC           |
+| 2014:4:8 22:05                                        | 2014-04-08 22:05:00 +0000 UTC           |
+| 2014:04:08 22:05                                      | 2014-04-08 22:05:00 +0000 UTC           |
+| 2014:04:2 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 2014:4:02 03:00:51                                    | 2014-04-02 03:00:51 +0000 UTC           |
+| 2012:03:19 10:11:59                                   | 2012-03-19 10:11:59 +0000 UTC           |
+| 2012:03:19 10:11:59.3186369                           | 2012-03-19 10:11:59.3186369 +0000 UTC   |
+| 2014年04月08日                                        | 2014-04-08 00:00:00 +0000 UTC           |
+| 2006-01-02T15:04:05+0000                              | 2006-01-02 15:04:05 +0000 UTC           |
+| 2009-08-12T22:15:09-07:00                             | 2009-08-12 22:15:09 -0700 -0700         |
+| 2009-08-12T22:15:09                                   | 2009-08-12 22:15:09 +0000 UTC           |
+| 2009-08-12T22:15:09.988                               | 2009-08-12 22:15:09.988 +0000 UTC       |
+| 2009-08-12T22:15:09Z                                  | 2009-08-12 22:15:09 +0000 UTC           |
+| 2017-07-19T03:21:51:897+0100                          | 2017-07-19 03:21:51.897 +0100 +0100     |
+| 2019-05-29T08:41-04                                   | 2019-05-29 08:41:00 -0400 -0400         |
+| 2014-04-26 17:24:37.3186369                           | 2014-04-26 17:24:37.3186369 +0000 UTC   |
+| 2012-08-03 18:31:59.257000000                         | 2012-08-03 18:31:59.257 +0000 UTC       |
+| 2014-04-26 17:24:37.123                               | 2014-04-26 17:24:37.123 +0000 UTC       |
+| 2013-04-01 22:43                                      | 2013-04-01 22:43:00 +0000 UTC           |
+| 2013-04-01 22:43:22                                   | 2013-04-01 22:43:22 +0000 UTC           |
+| 2014-12-16 06:20:00 UTC                               | 2014-12-16 06:20:00 +0000 UTC           |
+| 2014-12-16 06:20:00 GMT                               | 2014-12-16 06:20:00 +0000 UTC           |
+| 2014-04-26 05:24:37 PM                                | 2014-04-26 17:24:37 +0000 UTC           |
+| 2014-04-26 13:13:43 +0800                             | 2014-04-26 13:13:43 +0800 +0800         |
+| 2014-04-26 13:13:43 +0800 +08                         | 2014-04-26 13:13:43 +0800 +0800         |
+| 2014-04-26 13:13:44 +09:00                            | 2014-04-26 13:13:44 +0900 +0900         |
+| 2012-08-03 18:31:59.257000000 +0000 UTC               | 2012-08-03 18:31:59.257 +0000 UTC       |
+| 2015-09-30 18:48:56.35272715 +0000 UTC                | 2015-09-30 18:48:56.35272715 +0000 UTC  |
+| 2015-02-18 00:12:00 +0000 GMT                         | 2015-02-18 00:12:00 +0000 UTC           |
+| 2015-02-18 00:12:00 +0000 UTC                         | 2015-02-18 00:12:00 +0000 UTC           |
+| 2015-02-08 03:02:00 +0300 MSK m=+0.000000001          | 2015-02-08 03:02:00 +0300 +0300         |
+| 2015-02-08 03:02:00.001 +0300 MSK m=+0.000000001      | 2015-02-08 03:02:00.001 +0300 +0300     |
+| 2017-07-19 03:21:51+00:00                             | 2017-07-19 03:21:51 +0000 UTC           |
+| 2014-04-26                                            | 2014-04-26 00:00:00 +0000 UTC           |
+| 2014-04                                               | 2014-04-01 00:00:00 +0000 UTC           |
+| 2014                                                  | 2014-01-01 00:00:00 +0000 UTC           |
+| 2014-05-11 08:20:13,787                               | 2014-05-11 08:20:13.787 +0000 UTC       |
+| 2020-07-20+08:00                                      | 2020-07-20 00:00:00 +0800 +0800         |
+| 3.31.2014                                             | 2014-03-31 00:00:00 +0000 UTC           |
+| 03.31.2014                                            | 2014-03-31 00:00:00 +0000 UTC           |
+| 08.21.71                                              | 1971-08-21 00:00:00 +0000 UTC           |
+| 2014.03                                               | 2014-03-01 00:00:00 +0000 UTC           |
+| 2014.03.30                                            | 2014-03-30 00:00:00 +0000 UTC           |
+| 20140601                                              | 2014-06-01 00:00:00 +0000 UTC           |
+| 20140722105203                                        | 2014-07-22 10:52:03 +0000 UTC           |
+| 171113 14:14:20                                       | 2017-11-13 14:14:20 +0000 UTC           |
+| 1332151919                                            | 2012-03-19 10:11:59 +0000 UTC           |
+| 1384216367189                                         | 2013-11-12 00:32:47.189 +0000 UTC       |
+| 1384216367111222                                      | 2013-11-12 00:32:47.111222 +0000 UTC    |
+| 1384216367111222333                                   | 2013-11-12 00:32:47.111222333 +0000 UTC |
++-------------------------------------------------------+-----------------------------------------+
+*/
+
+```

vendor/github.com/go-shiori/dom/.gitignore

@@ -0,0 +1,2 @@
+# Exclude config
+/.vscode

vendor/github.com/go-shiori/dom/.travis.yml

@@ -0,0 +1,11 @@
+language: go
+sudo: true
+matrix:
+  include:
+    - go: 1.x
+      env: LATEST=true
+
+script:
+  - go get -v ./...
+  - go vet $(go list ./... | grep -v /vendor/)
+  - go test -v -race ./...

vendor/github.com/go-shiori/dom/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019-present Radhi Fadlillah
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/go-shiori/dom/README.md

@@ -0,0 +1,22 @@
+DOM
+===
+
+[![GoDoc](https://godoc.org/github.com/go-shiori/dom?status.png)](https://godoc.org/github.com/go-shiori/dom)
+[![Travis CI](https://travis-ci.org/go-shiori/dom.svg?branch=master)](https://travis-ci.org/go-shiori/dom)
+[![Go Report Card](https://goreportcard.com/badge/github.com/go-shiori/dom)](https://goreportcard.com/report/github.com/go-shiori/dom)
+[![Donate PayPal](https://img.shields.io/static/v1?label=donate&message=PayPal&color=00457C&logo=paypal)](https://www.paypal.me/RadhiFadlillah)
+[![Donate Ko-fi](https://img.shields.io/static/v1?label=donate&message=Ko-fi&color=F16061&logo=ko-fi)](https://ko-fi.com/radhifadlillah)
+
+DOM is a Go package for manipulating [HTML](https://godoc.org/golang.org/x/net/html) node. The methods that exist in this package has similar name and purpose as the DOM manipulation methods in JS, which make it useful when porting code from JS to Go. Currently it used in [`warc`](https://github.com/go-shiori/warc) and [`go-readability`](https://github.com/go-shiori/go-readability).
+
+## Installation
+
+To install this package, just run `go get` :
+
+```
+go get -u -v github.com/go-shiori/dom
+```
+
+## Licenses
+
+DOM is distributed under [MIT license](https://choosealicense.com/licenses/mit/), which means you can use and modify it however you want. However, if you make an enhancement for it, if possible, please send a pull request. If you like this project, please consider donating to me either via [PayPal](https://www.paypal.me/RadhiFadlillah) or [Ko-Fi](https://ko-fi.com/radhifadlillah).

vendor/github.com/go-shiori/dom/dom.go

@@ -0,0 +1,622 @@
+package dom
+
+import (
+	"bytes"
+	"regexp"
+	"strings"
+
+	"github.com/andybalholm/cascadia"
+	"golang.org/x/net/html"
+)
+
+var (
+	rxPunctuation      = regexp.MustCompile(`\s+([.?!,;])\s*(\S*)`)
+	rxTempNewline      = regexp.MustCompile(`\s*\|\\/\|\s*`)
+	rxDisplayNone      = regexp.MustCompile(`(?i)display:\s*none`)
+	rxVisibilityHidden = regexp.MustCompile(`(?i)visibility:\s*(:?hidden|collapse)`)
+)
+
+// QuerySelectorAll returns array of document's elements that match
+// the specified group of selectors.
+func QuerySelectorAll(doc *html.Node, selectors string) []*html.Node {
+	matcher, err := cascadia.ParseGroup(selectors)
+	if err != nil {
+		return nil
+	}
+
+	return cascadia.QueryAll(doc, matcher)
+}
+
+// QuerySelector returns the first document's element that match
+// the specified group of selectors.
+func QuerySelector(doc *html.Node, selectors string) *html.Node {
+	matcher, err := cascadia.ParseGroup(selectors)
+	if err != nil {
+		return nil
+	}
+
+	return cascadia.Query(doc, matcher)
+}
+
+// GetElementByID returns a Node object representing the element whose id
+// property matches the specified string.
+func GetElementByID(doc *html.Node, id string) *html.Node {
+	if id == "" {
+		return nil
+	}
+
+	var results []*html.Node
+	var finder func(*html.Node)
+
+	finder = func(node *html.Node) {
+		nodeID := GetAttribute(node, "id")
+		nodeID = strings.TrimSpace(nodeID)
+
+		if node.Type == html.ElementNode && nodeID == id {
+			results = append(results, node)
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	for child := doc.FirstChild; child != nil; child = child.NextSibling {
+		finder(child)
+
+		if len(results) > 0 {
+			return results[0]
+		}
+	}
+
+	return nil
+}
+
+// GetElementsByClassName returns an array of all child elements which
+// have all of the given class name(s).
+func GetElementsByClassName(doc *html.Node, classNames string) []*html.Node {
+	// Convert class name to map
+	classes := map[string]struct{}{}
+	for _, class := range strings.Fields(classNames) {
+		classes[class] = struct{}{}
+	}
+
+	nClasses := len(classes)
+	if nClasses == 0 {
+		return nil
+	}
+
+	// Create finder method
+	var results []*html.Node
+	var finder func(*html.Node)
+	allClassExist := func(node *html.Node) bool {
+		matchCount := 0
+		nodeClasses := GetAttribute(node, "class")
+		for _, nodeClass := range strings.Fields(nodeClasses) {
+			if _, exist := classes[nodeClass]; exist {
+				matchCount++
+			}
+		}
+
+		return matchCount == nClasses
+	}
+
+	finder = func(node *html.Node) {
+		if node.Type == html.ElementNode && allClassExist(node) {
+			results = append(results, node)
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	// Check all nodes
+	for child := doc.FirstChild; child != nil; child = child.NextSibling {
+		finder(child)
+	}
+
+	return results
+}
+
+// GetElementsByTagName returns a collection of all elements in the document with
+// the specified tag name, as an array of Node object.
+// The special tag "*" will represents all elements.
+func GetElementsByTagName(doc *html.Node, tagName string) []*html.Node {
+	var results []*html.Node
+	var finder func(*html.Node)
+
+	finder = func(node *html.Node) {
+		if node.Type == html.ElementNode && (tagName == "*" || node.Data == tagName) {
+			results = append(results, node)
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	for child := doc.FirstChild; child != nil; child = child.NextSibling {
+		finder(child)
+	}
+
+	return results
+}
+
+// CreateElement creates a new ElementNode with specified tag.
+func CreateElement(tagName string) *html.Node {
+	return &html.Node{
+		Type: html.ElementNode,
+		Data: tagName,
+	}
+}
+
+// CreateTextNode creates a new Text node.
+func CreateTextNode(data string) *html.Node {
+	return &html.Node{
+		Type: html.TextNode,
+		Data: data,
+	}
+}
+
+// TagName returns the tag name of a Node.
+// If it's not ElementNode, return empty string.
+func TagName(node *html.Node) string {
+	if node == nil {
+		return ""
+	}
+
+	if node.Type != html.ElementNode {
+		return ""
+	}
+
+	return node.Data
+}
+
+// GetAttribute returns the value of a specified attribute on
+// the element. If the given attribute does not exist, the value
+// returned will be an empty string.
+func GetAttribute(node *html.Node, attrName string) string {
+	for i := 0; i < len(node.Attr); i++ {
+		if node.Attr[i].Key == attrName {
+			return node.Attr[i].Val
+		}
+	}
+	return ""
+}
+
+// SetAttribute sets attribute for node. If attribute already exists,
+// it will be replaced.
+func SetAttribute(node *html.Node, attrName string, attrValue string) {
+	attrIdx := -1
+	for i := 0; i < len(node.Attr); i++ {
+		if node.Attr[i].Key == attrName {
+			attrIdx = i
+			break
+		}
+	}
+
+	if attrIdx >= 0 {
+		node.Attr[attrIdx].Val = attrValue
+	} else {
+		node.Attr = append(node.Attr, html.Attribute{
+			Key: attrName,
+			Val: attrValue,
+		})
+	}
+}
+
+// RemoveAttribute removes attribute with given name.
+func RemoveAttribute(node *html.Node, attrName string) {
+	attrIdx := -1
+	for i := 0; i < len(node.Attr); i++ {
+		if node.Attr[i].Key == attrName {
+			attrIdx = i
+			break
+		}
+	}
+
+	if attrIdx >= 0 {
+		a := node.Attr
+		a = append(a[:attrIdx], a[attrIdx+1:]...)
+		node.Attr = a
+	}
+}
+
+// HasAttribute returns a Boolean value indicating whether the
+// specified node has the specified attribute or not.
+func HasAttribute(node *html.Node, attrName string) bool {
+	for i := 0; i < len(node.Attr); i++ {
+		if node.Attr[i].Key == attrName {
+			return true
+		}
+	}
+	return false
+}
+
+// TextContent returns the text content of the specified node,
+// and all its descendants.
+func TextContent(node *html.Node) string {
+	var buffer bytes.Buffer
+	var finder func(*html.Node)
+
+	finder = func(n *html.Node) {
+		if n.Type == html.TextNode {
+			buffer.WriteString(n.Data)
+		}
+
+		for child := n.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(node)
+	return buffer.String()
+}
+
+// InnerText in JS used to capture text from an element while excluding text from hidden
+// children. A child is hidden if it's computed width is 0, whether because its CSS (e.g
+// `display: none`, `visibility: hidden`, etc), or if the child has `hidden` attribute.
+// Since we can't compute stylesheet, we only look at `hidden` attribute and inline style.
+//
+// Besides excluding text from hidden children, difference between this function and
+// `TextContent` is the latter will skip <br> tag while this function will preserve
+// <br> as newline.
+func InnerText(node *html.Node) string {
+	var buffer bytes.Buffer
+	var finder func(*html.Node)
+
+	finder = func(n *html.Node) {
+		switch n.Type {
+		case html.TextNode:
+			buffer.WriteString(" " + n.Data + " ")
+
+		case html.ElementNode:
+			if n.Data == "br" {
+				buffer.WriteString(`|\/|`)
+				return
+			}
+
+			if HasAttribute(n, "hidden") {
+				return
+			}
+
+			styleAttr := GetAttribute(n, "style")
+			if rxDisplayNone.MatchString(styleAttr) || rxVisibilityHidden.MatchString(styleAttr) {
+				return
+			}
+		}
+
+		for child := n.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(node)
+	text := buffer.String()
+	text = strings.Join(strings.Fields(text), " ")
+	text = rxPunctuation.ReplaceAllString(text, "$1 $2")
+	text = rxTempNewline.ReplaceAllString(text, "\n")
+	return text
+}
+
+// OuterHTML returns an HTML serialization of the element and its descendants.
+// The returned HTML value is escaped.
+func OuterHTML(node *html.Node) string {
+	if node == nil {
+		return ""
+	}
+
+	var buffer bytes.Buffer
+	err := html.Render(&buffer, node)
+	if err != nil {
+		return ""
+	}
+
+	return buffer.String()
+}
+
+// InnerHTML returns the HTML content (inner HTML) of an element.
+// The returned HTML value is escaped.
+func InnerHTML(node *html.Node) string {
+	var err error
+	var buffer bytes.Buffer
+
+	if node == nil {
+		return ""
+	}
+
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		err = html.Render(&buffer, child)
+		if err != nil {
+			return ""
+		}
+	}
+
+	return strings.TrimSpace(buffer.String())
+}
+
+// DocumentElement returns the Element that is the root element
+// of the document. Since we are working with HTML document,
+// the root will be <html> element for HTML documents).
+func DocumentElement(doc *html.Node) *html.Node {
+	if nodes := GetElementsByTagName(doc, "html"); len(nodes) > 0 {
+		return nodes[0]
+	}
+	return nil
+}
+
+// ID returns the value of the id attribute of the specified element.
+func ID(node *html.Node) string {
+	id := GetAttribute(node, "id")
+	id = strings.TrimSpace(id)
+	return id
+}
+
+// ClassName returns the value of the class attribute of
+// the specified element.
+func ClassName(node *html.Node) string {
+	className := GetAttribute(node, "class")
+	className = strings.TrimSpace(className)
+	className = strings.Join(strings.Fields(className), " ")
+	return className
+}
+
+// Children returns an HTMLCollection of the direct child elements of Node.
+func Children(node *html.Node) []*html.Node {
+	var children []*html.Node
+	if node == nil {
+		return nil
+	}
+
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			children = append(children, child)
+		}
+	}
+
+	return children
+}
+
+// ChildNodes returns list of a node's direct children.
+func ChildNodes(node *html.Node) []*html.Node {
+	var childNodes []*html.Node
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		childNodes = append(childNodes, child)
+	}
+	return childNodes
+}
+
+// FirstElementChild returns the object's first child Element,
+// or nil if there are no child elements.
+func FirstElementChild(node *html.Node) *html.Node {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			return child
+		}
+	}
+	return nil
+}
+
+// PreviousElementSibling returns the the Element immediately prior
+// to the specified one in its parent's children list, or null if
+// the specified element is the first one in the list.
+func PreviousElementSibling(node *html.Node) *html.Node {
+	for sibling := node.PrevSibling; sibling != nil; sibling = sibling.PrevSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}
+
+// NextElementSibling returns the Element immediately following
+// the specified one in its parent's children list, or nil if the
+// specified Element is the last one in the list.
+func NextElementSibling(node *html.Node) *html.Node {
+	for sibling := node.NextSibling; sibling != nil; sibling = sibling.NextSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}
+
+// AppendChild adds a node to the end of the list of children of a
+// specified parent node. If the given child is a reference to an
+// existing node in the document, AppendChild() moves it from its
+// current position to the new position.
+func AppendChild(node *html.Node, child *html.Node) {
+	// Make sure node is not void
+	if !IsVoidElement(node) {
+		DetachChild(child)
+		node.AppendChild(child)
+	}
+}
+
+// PrependChild works like AppendChild() except it adds a node to the
+// beginning of the list of children of a specified parent node.
+func PrependChild(node *html.Node, child *html.Node) {
+	// Make sure node is not void
+	if !IsVoidElement(node) {
+		DetachChild(child)
+		if node.FirstChild != nil {
+			node.InsertBefore(child, node.FirstChild)
+		} else {
+			node.AppendChild(child)
+		}
+	}
+}
+
+// ReplaceChild replaces a child node within the given (parent) node.
+// If the new child is already exist in document, ReplaceChild() will move it
+// from its current position to replace old child. Returns both the new and old child.
+//
+// TODO: I'm note sure but I *think* there are some issues here. Check later I guess.
+func ReplaceChild(parent *html.Node, newChild *html.Node, oldChild *html.Node) (*html.Node, *html.Node) {
+	// Make sure parent is specified and not void
+	if parent == nil && !IsVoidElement(parent) {
+		return newChild, oldChild
+	}
+
+	// Make sure the specified parent IS the parent of the old child
+	if oldChild.Parent != parent {
+		return newChild, oldChild
+	}
+
+	// Detach the new child
+	DetachChild(newChild)
+	parent.InsertBefore(newChild, oldChild)
+	parent.RemoveChild(oldChild)
+	return newChild, oldChild
+}
+
+// IncludeNode determines if node is included inside nodeList.
+func IncludeNode(nodeList []*html.Node, node *html.Node) bool {
+	for i := 0; i < len(nodeList); i++ {
+		if nodeList[i] == node {
+			return true
+		}
+	}
+	return false
+}
+
+// Clone returns a clone of the node and (if specified) its children.
+// However, it will be detached from the original's parents and siblings.
+func Clone(src *html.Node, deep bool) *html.Node {
+	clone := &html.Node{
+		Type:     src.Type,
+		DataAtom: src.DataAtom,
+		Data:     src.Data,
+		Attr:     append([]html.Attribute{}, src.Attr...),
+	}
+
+	if deep {
+		for child := src.FirstChild; child != nil; child = child.NextSibling {
+			clone.AppendChild(Clone(child, deep))
+		}
+	}
+
+	return clone
+}
+
+// GetAllNodesWithTag is wrapper for GetElementsByTagName()
+// which allow to get several tags at once.
+func GetAllNodesWithTag(node *html.Node, tagNames ...string) []*html.Node {
+	var result []*html.Node
+	for i := 0; i < len(tagNames); i++ {
+		result = append(result, GetElementsByTagName(node, tagNames[i])...)
+	}
+	return result
+}
+
+// ForEachNode iterates over a NodeList and runs fn on each node.
+func ForEachNode(nodeList []*html.Node, fn func(*html.Node, int)) {
+	for i := 0; i < len(nodeList); i++ {
+		fn(nodeList[i], i)
+	}
+}
+
+// RemoveNodes iterates over a NodeList, calls `filterFn` for each node
+// and removes node if function returned `true`. If function is not
+// passed, removes all the nodes in node list.
+func RemoveNodes(nodeList []*html.Node, filterFn func(*html.Node) bool) {
+	for i := len(nodeList) - 1; i >= 0; i-- {
+		node := nodeList[i]
+		parentNode := node.Parent
+		if parentNode != nil && (filterFn == nil || filterFn(node)) {
+			parentNode.RemoveChild(node)
+		}
+	}
+}
+
+// SetTextContent sets the text content of the specified node.
+func SetTextContent(node *html.Node, text string) {
+	if IsVoidElement(node) {
+		return
+	}
+
+	child := node.FirstChild
+	for child != nil {
+		nextSibling := child.NextSibling
+		node.RemoveChild(child)
+		child = nextSibling
+	}
+
+	node.AppendChild(&html.Node{
+		Type: html.TextNode,
+		Data: text,
+	})
+}
+
+// SetInnerHTML sets inner HTML of the specified node.
+func SetInnerHTML(node *html.Node, rawHTML string) {
+	// Parse raw HTML
+	parsedHTML, err := html.Parse(strings.NewReader(rawHTML))
+	if err != nil || parsedHTML == nil {
+		return
+	}
+
+	// Remove node's current children
+	child := node.FirstChild
+	for child != nil {
+		nextSibling := child.NextSibling
+		node.RemoveChild(child)
+		child = nextSibling
+	}
+
+	// Put content of parsed HTML to the node
+	if body := QuerySelector(parsedHTML, "body"); body != nil {
+		bodyChild := body.FirstChild
+		for bodyChild != nil {
+			nextSibling := bodyChild.NextSibling
+			AppendChild(node, bodyChild)
+			bodyChild = nextSibling
+		}
+	}
+}
+
+// IsVoidElement check whether a node can have any contents or not.
+// Return true if element is void (can't have any children).
+func IsVoidElement(n *html.Node) bool {
+	// If it's not element, it's void
+	if n.Type != html.ElementNode {
+		return true
+	}
+
+	// Check tag name
+	switch n.Data {
+	case "area", "base", "br", "col", "embed", "hr",
+		"img", "input", "keygen", "link", "meta",
+		"param", "source", "track", "wbr":
+		return true
+	default:
+		return false
+	}
+}
+
+func DetachChild(child *html.Node) {
+	if child.Parent != nil || child.PrevSibling != nil || child.NextSibling != nil {
+		if child.Parent != nil {
+			if child.Parent.FirstChild == child {
+				child.Parent.FirstChild = child.NextSibling
+			}
+
+			if child.Parent.LastChild == child {
+				child.Parent.LastChild = child.PrevSibling
+			}
+		}
+
+		if child.PrevSibling != nil {
+			child.PrevSibling.NextSibling = child.NextSibling
+		}
+
+		if child.NextSibling != nil {
+			child.NextSibling.PrevSibling = child.PrevSibling
+		}
+
+		child.Parent = nil
+		child.PrevSibling = nil
+		child.NextSibling = nil
+	}
+}

vendor/github.com/go-shiori/dom/parser.go

@@ -0,0 +1,61 @@
+package dom
+
+import (
+	"bytes"
+	"io"
+	"io/ioutil"
+
+	"github.com/gogs/chardet"
+	"golang.org/x/net/html"
+	"golang.org/x/net/html/charset"
+	xunicode "golang.org/x/text/encoding/unicode"
+	"golang.org/x/text/runes"
+	"golang.org/x/text/transform"
+	"golang.org/x/text/unicode/norm"
+)
+
+// FastParse parses html.Node from the specified reader without caring about
+// text encoding. It always assume that the input uses UTF-8 encoding.
+func FastParse(r io.Reader) (*html.Node, error) {
+	return html.Parse(r)
+}
+
+// Parse parses html.Node from the specified reader while converting the character
+// encoding into UTF-8. This function is useful to correctly parse web pages that
+// uses custom text encoding, e.g. web pages from Asian websites. However, since it
+// has to detect charset before parsing, this function is quite slow and expensive
+// so if you sure the reader uses valid UTF-8, just use FastParse.
+func Parse(r io.Reader) (*html.Node, error) {
+	// Split the reader using tee
+	content, err := ioutil.ReadAll(r)
+	if err != nil {
+		return nil, err
+	}
+
+	// Detect page encoding
+	res, err := chardet.NewHtmlDetector().DetectBest(content)
+	if err != nil {
+		return nil, err
+	}
+
+	pageEncoding, _ := charset.Lookup(res.Charset)
+	if pageEncoding == nil {
+		pageEncoding = xunicode.UTF8
+	}
+
+	// Parse HTML using the page encoding
+	r = bytes.NewReader(content)
+	r = transform.NewReader(r, pageEncoding.NewDecoder())
+	r = normalizeTextEncoding(r)
+	return html.Parse(r)
+}
+
+// normalizeTextEncoding convert text encoding from NFD to NFC.
+// It also remove soft hyphen since apparently it's useless in web.
+// See: https://web.archive.org/web/19990117011731/http://www.hut.fi/~jkorpela/shy.html
+func normalizeTextEncoding(r io.Reader) io.Reader {
+	fnSoftHyphen := func(r rune) bool { return r == '\u00AD' }
+	softHyphenSet := runes.Predicate(fnSoftHyphen)
+	transformer := transform.Chain(norm.NFD, runes.Remove(softHyphenSet), norm.NFC)
+	return transform.NewReader(r, transformer)
+}

vendor/github.com/go-shiori/go-readability/.gitattributes

@@ -0,0 +1 @@
+test-pages/* linguist-vendored

vendor/github.com/go-shiori/go-readability/.gitignore

@@ -0,0 +1,6 @@
+.vscode/
+js/*
+
+# Hidden script for development
+scripts/internal/
+.DS_Store

vendor/github.com/go-shiori/go-readability/.travis.yml

@@ -0,0 +1,16 @@
+language: go
+sudo: true
+matrix:
+  include:
+    - go: 1.x
+      env: LATEST=true
+    - go: "1.13"
+
+install:
+  - go get github.com/go-shiori/go-readability
+
+script:
+  - go get -t -v ./...
+  - diff -u <(echo -n) <(gofmt -d .)
+  - go vet $(go list ./... | grep -v /vendor/)
+  - go test -v -race ./...

vendor/github.com/go-shiori/go-readability/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019 Radhi Fadlillah
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/go-shiori/go-readability/Makefile

@@ -0,0 +1,10 @@
+generate:
+	@for name in internal/re2go/*.re; do \
+		RE_IN=$$name; \
+		RE_OUT=$$(echo $$name | sed 's/\.re/.go/'); \
+		re2go -W -F --input-encoding utf8 --utf8 --no-generation-date -i $$RE_IN -o $$RE_OUT; \
+		gofmt -w $$RE_OUT; \
+	done
+
+test: generate
+	go test -timeout 30s ./...