p1: /v2/scrape, /v1/fetch, auth, CLI, contract tests

Single Fetcher.Fetch on the Guard that holds DenyPrefixes. skipTlsVerification
is ignored. No 502. keys add writes the secret only to --out. Release binary
has no -test-allow-prefix.

vendor/github.com/JohannesKaufmann/dom/.gitignore

@@ -0,0 +1,22 @@
+# If you prefer the allow list template instead of the deny list, see community template:
+# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
+#
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Dependency directories (remove the comment below to include it)
+# vendor/
+
+# Go workspace file
+go.work
+go.work.sum

vendor/github.com/JohannesKaufmann/dom/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/JohannesKaufmann/dom/README.md

@@ -0,0 +1,277 @@
+# dom
+
+[![Go Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/dom.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/dom)
+
+Helper functions for "net/html" that make it easier to interact with `*html.Node`.
+
+🚀 [Getting Started](#getting-started) - 📚 [Documentation](#documentation) - 🧑‍💻 [Examples](/examples/)
+
+## Installation
+
+```bash
+go get -u github.com/JohannesKaufmann/dom
+```
+
+> [!NOTE]
+> This "dom" libary was developed for the needs of the [html-to-markdown](https://github.com/JohannesKaufmann/html-to-markdown) library.
+> That beeing said, please submit any functions that you need.
+
+## Getting Started
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func main() {
+	input := `
+	<ul>
+		<li><a href="github.com/JohannesKaufmann/dom">dom</a></li>
+		<li><a href="github.com/JohannesKaufmann/html-to-markdown">html-to-markdown</a></li>
+	</ul>
+	`
+
+	doc, err := html.Parse(strings.NewReader(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	// - - - //
+
+	firstLink := dom.FindFirstNode(doc, func(node *html.Node) bool {
+		return dom.NodeName(node) == "a"
+	})
+
+	fmt.Println("href:", dom.GetAttributeOr(firstLink, "href", ""))
+}
+```
+
+## Node vs Element
+
+The naming scheme in this library is:
+
+- "Node" means `*html.Node{}`
+  - This means _any_ node in the tree of nodes.
+- "Element" means `*html.Node{Type: html.ElementNode}`
+  - This means _only_ nodes with the type of `ElementNode`. For example `<p>`, `<span>`, `<a>`, ... but not `#text`, `<!--comment-->`, ...
+
+For most functions, there are two versions. For example:
+
+- `FirstChildNode()` and `FirstChildElement()`
+- `AllChildNodes()` and `AllChildElements()`
+- ...
+
+## Documentation
+
+[![Go Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/dom.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/dom)
+
+### Attributes & Content
+
+You can get the attributes of a node using `GetAttribute`, `GetAttributeOr` or the more specialized `GetClasses` that returns a slice of strings.
+
+For matching nodes, `HasID` and `HasClass` can be used.
+
+If you want to collect the #text of all the child nodes, you can call `CollectText`.
+
+```go
+name := dom.NodeName(node)
+// "h2"
+
+href := dom.GetAttributeOr(node, "href", "")
+// "github.com"
+
+isHeading := dom.HasClass(node, "repo__name")
+// `true`
+
+content := dom.CollectText(node)
+// "Lorem ipsum"
+```
+
+---
+
+### Children & Siblings
+
+You can already use `node.FirstChild` to get the first child _node_. For the convenience we added `FirstChildNode()` and `FirstChildElement()` which returns `*html.Node`.
+
+To get all direct children, use `AllChildNodes` and `AllChildElements` which returns `[]*html.Node`.
+
+- `PrevSiblingNode` and `PrevSiblingElement`
+
+- `NextSiblingNode` and `NextSiblingElement`
+
+### Find Nodes
+
+Searching for nodes deep in the tree is made easier with:
+
+```go
+firstParagraph := dom.FindFirstNode(doc, func(node *html.Node) bool {
+    return dom.NodeName(node) == "p"
+})
+// *html.Node
+
+
+allParagraphs := dom.FindAllNodes(doc, func(node *html.Node) bool {
+    return dom.NodeName(node) == "p"
+})
+// []*html.Node
+```
+
+- 🧑‍💻 [Example code, find](/examples/find/main.go)
+- 🧑‍💻 [Example code, selectors](/examples/selectors/main.go)
+
+---
+
+### Get next/previous neighbors
+
+What is special about this? The order!
+
+If you are somewhere in the DOM, you can call `GetNextNeighborNode` to get the next node, even if it is _further up_ the tree. The order is the same as you would see the elements in the DOM.
+
+```go
+node := startNode
+for node != nil {
+    fmt.Println(dom.NodeName(node))
+
+    node = dom.GetNextNeighborNode(node)
+}
+```
+
+If we start the `for` loop at the `<button>` and repeatedly call `GetNextNeighborNode` this would be the _order_ that the nodes are _visited_.
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─nav
+│ │ │ ├─p
+│ │ │ │ ├─#text "up"
+│ │ ├─main
+│ │ │ ├─button   *️⃣
+│ │ │ │ ├─span  0️⃣
+│ │ │ │ │ ├─#text "start"  1️⃣
+│ │ │ ├─div  2️⃣
+│ │ │ │ ├─h3  3️⃣
+│ │ │ │ │ ├─#text "heading"  4️⃣
+│ │ │ │ ├─p  5️⃣
+│ │ │ │ │ ├─#text "description"  6️⃣
+│ │ ├─footer  7️⃣
+│ │ │ ├─p  8️⃣
+│ │ │ │ ├─#text "down"  9️⃣
+```
+
+If you only want to visit the ElementNode's (and skip the `#text` Nodes) you can use `GetNextNeighborElement` instead.
+
+If you want to skip the children you can use `GetNextNeighborNodeExcludingOwnChild`. In the example above, when starting at the `<button>` the next node would be the `<div>`.
+
+The same functions also exist for the previous nodes, e.g. `GetPrevNeighborNode`.
+
+- 🧑‍💻 [Example code, next basics](/examples/next_basics/main.go)
+- 🧑‍💻 [Example code, next inside a loop](/examples/next_loop/main.go)
+
+---
+
+### Remove & Replace Node
+
+```go
+if dom.HasClass(node, "lang__old") {
+	newNode := &html.Node{
+		Type: html.TextNode,
+		Data: "🪦",
+	}
+	dom.ReplaceNode(node, newNode)
+}
+
+
+for _, node := range emptyTextNodes {
+	dom.RemoveNode(node)
+}
+```
+
+- 🧑‍💻 [Example code, remove and replace](/examples/remove_replace/main.go)
+
+### Unwrap Node
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─article   *️⃣
+│ │ │ ├─h3
+│ │ │ │ ├─#text "Heading"
+│ │ │ ├─p
+│ │ │ │ ├─#text "short description"
+```
+
+If we take the input above and run `UnwrapNode(articleNode)` we can "unwrap" the `<article>`. That means removing the `<article>` while _keeping_ the children (`<h3>` and `<p>`).
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─h3
+│ │ │ ├─#text "Heading"
+│ │ ├─p
+│ │ │ ├─#text "short description"
+```
+
+For the reverse you can use `WrapNode(existingNode, newNode)`.
+
+---
+
+### RenderRepresentation
+
+```go
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func main() {
+	input := `<a href="/about">Read More</a>`
+
+	doc, err := html.Parse(strings.NewReader(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	fmt.Println(dom.RenderRepresentation(doc))
+}
+```
+
+The tree representation helps to visualize the tree-structure of the DOM.
+And the `#text` nodes stand out.
+
+> [!TIP]
+> This function could be useful for debugging & testcases.
+> For example in [neighbors_test.go](/neighbors_test.go)
+
+```text
+#document
+├─html
+│ ├─head
+│ ├─body
+│ │ ├─a (href=/about)
+│ │ │ ├─#text "Read More"
+```
+
+While the normal "net/html" [`Render()`](https://pkg.go.dev/golang.org/x/net/html#Render) function would have produced this:
+
+```
+<html><head></head><body><a href="/about">Read More</a></body></html>
+```
+
+- 🧑‍💻 [Example code, dom representation](/examples/dom_representation/main.go)

vendor/github.com/JohannesKaufmann/dom/attr.go

@@ -0,0 +1,68 @@
+package dom
+
+import (
+	"bytes"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+func GetAttribute(node *html.Node, key string) (string, bool) {
+	for _, attr := range node.Attr {
+		if attr.Key == key {
+			return attr.Val, true
+		}
+	}
+	return "", false
+}
+func GetAttributeOr(node *html.Node, key string, fallback string) string {
+	for _, attr := range node.Attr {
+		if attr.Key == key {
+			return attr.Val
+		}
+	}
+	return fallback
+}
+
+func GetClasses(node *html.Node) []string {
+	val, found := GetAttribute(node, "class")
+	if !found {
+		return nil
+	}
+
+	return strings.Fields(val)
+}
+
+func HasID(node *html.Node, expectedID string) bool {
+	val, found := GetAttribute(node, "id")
+	if !found {
+		return false
+	}
+	return strings.TrimSpace(val) == expectedID
+}
+func HasClass(node *html.Node, expectedClass string) bool {
+	classes := GetClasses(node)
+	for _, class := range classes {
+		if class == expectedClass {
+			return true
+		}
+	}
+	return false
+}
+
+// - - - - //
+
+func collectText(n *html.Node, buf *bytes.Buffer) {
+	if n.Type == html.TextNode {
+		buf.WriteString(n.Data)
+	}
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		collectText(c, buf)
+	}
+}
+
+func CollectText(node *html.Node) string {
+	var buf bytes.Buffer
+	collectText(node, &buf)
+	return buf.String()
+}

vendor/github.com/JohannesKaufmann/dom/change.go

@@ -0,0 +1,49 @@
+package dom
+
+import "golang.org/x/net/html"
+
+func RemoveNode(node *html.Node) {
+	if node == nil || node.Parent == nil {
+		return
+	}
+
+	node.Parent.RemoveChild(node)
+}
+
+func ReplaceNode(node, newNode *html.Node) {
+	if node.Parent == nil || node == newNode {
+		return
+	}
+
+	node.Parent.InsertBefore(newNode, node)
+	node.Parent.RemoveChild(node)
+}
+
+func UnwrapNode(node *html.Node) {
+	if node == nil || node.Parent == nil {
+		return
+	}
+
+	// In each iteration, we once again grab the first child, since
+	// the previous first child was just removed.
+	for child := node.FirstChild; child != nil; child = node.FirstChild {
+		node.RemoveChild(child)
+		node.Parent.InsertBefore(child, node)
+	}
+
+	node.Parent.RemoveChild(node)
+}
+
+// WrapNode wraps the newNode around the existingNode.
+func WrapNode(existingNode, newNode *html.Node) *html.Node {
+	if existingNode == nil || existingNode.Parent == nil {
+		return existingNode
+	}
+
+	existingNode.Parent.InsertBefore(newNode, existingNode)
+	existingNode.Parent.RemoveChild(existingNode)
+
+	newNode.AppendChild(existingNode)
+
+	return newNode
+}

vendor/github.com/JohannesKaufmann/dom/dom.go

@@ -0,0 +1,88 @@
+// dom makes it easier to interact with the html document.
+//
+// Node = return all the nodes
+// Element = return all the nodes that are of type Element. This e.g. excludes #text nodes.
+package dom
+
+import "golang.org/x/net/html"
+
+// AllNodes recursively gets all the nodes in the tree.
+func AllNodes(startNode *html.Node) (allNodes []*html.Node) {
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		allNodes = append(allNodes, node)
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(startNode)
+
+	return allNodes
+}
+
+// - - - - - - - - - - - - - - - //
+
+func AllChildNodes(node *html.Node) (children []*html.Node) {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		children = append(children, child)
+	}
+	return children
+}
+
+// AllChildElements is similar to AllChildNodes but only returns
+// nodes of type `ElementNode`.
+func AllChildElements(node *html.Node) (children []*html.Node) {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			children = append(children, child)
+		}
+	}
+	return children
+}
+
+// - - - - - - - - - - - - - - - //
+
+func FirstChildNode(node *html.Node) *html.Node {
+	return node.FirstChild
+}
+func FirstChildElement(node *html.Node) *html.Node {
+	for child := node.FirstChild; child != nil; child = child.NextSibling {
+		if child.Type == html.ElementNode {
+			return child
+		}
+	}
+	return nil
+}
+
+// - - - - - - - - - - - - - - - //
+
+func PrevSiblingNode(node *html.Node) *html.Node {
+	return node.PrevSibling
+}
+func PrevSiblingElement(node *html.Node) *html.Node {
+	for sibling := node.PrevSibling; sibling != nil; sibling = sibling.PrevSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}
+
+// - - - - - - - - - - - - - - - //
+
+func NextSiblingNode(node *html.Node) *html.Node {
+	return node.NextSibling
+}
+
+// NextSiblingElement returns the element immediately following the passed-in node or nil.
+// In contrast to `node.NextSibling` this only returns the next `ElementNode`.
+func NextSiblingElement(node *html.Node) *html.Node {
+	for sibling := node.NextSibling; sibling != nil; sibling = sibling.NextSibling {
+		if sibling.Type == html.ElementNode {
+			return sibling
+		}
+	}
+	return nil
+}

vendor/github.com/JohannesKaufmann/dom/dom_representation.go

@@ -0,0 +1,64 @@
+package dom
+
+import (
+	"bytes"
+	"fmt"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+func writePipeChar(buf *bytes.Buffer, index int) {
+	if index == 0 {
+		return
+	}
+	buf.WriteString(strings.Repeat("│ ", index-1))
+	buf.WriteString("├─")
+}
+func writeNode(buf *bytes.Buffer, node *html.Node) {
+	name := NodeName(node)
+	buf.WriteString(name)
+
+	if len(node.Attr) != 0 {
+		buf.WriteString(" (")
+	}
+	for i, attr := range node.Attr {
+		buf.WriteString(fmt.Sprintf("%s=%q", attr.Key, attr.Val))
+
+		if i < len(node.Attr)-1 {
+			buf.WriteString(" ")
+		}
+	}
+	if len(node.Attr) != 0 {
+		buf.WriteString(")")
+	}
+
+	if name == "#text" {
+		buf.WriteString(fmt.Sprintf(" %q", node.Data))
+	}
+
+}
+
+// RenderRepresentation is useful for debugging.
+// It renders out the *structure* of the dom.
+func RenderRepresentation(startNode *html.Node) string {
+	var buf bytes.Buffer
+	var finder func(*html.Node, int)
+	finder = func(node *html.Node, index int) {
+		writePipeChar(&buf, index)
+		writeNode(&buf, node)
+		buf.WriteRune('\n')
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child, index+1)
+		}
+	}
+
+	if startNode.Parent == nil {
+		finder(startNode, 0)
+	} else {
+		finder(startNode, 1)
+	}
+
+	return strings.TrimSpace(buf.String())
+}

vendor/github.com/JohannesKaufmann/dom/find.go

@@ -0,0 +1,49 @@
+package dom
+
+import "golang.org/x/net/html"
+
+func ContainsNode(startNode *html.Node, matchFn func(node *html.Node) bool) bool {
+	return FindFirstNode(startNode, matchFn) != nil
+}
+
+func FindFirstNode(startNode *html.Node, matchFn func(node *html.Node) bool) *html.Node {
+	nextFunc := UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		func(node *html.Node) bool {
+			// We should not get higher up than the startNode...
+			return node == startNode
+		},
+	)
+
+	child := startNode.FirstChild
+	for child != nil {
+		if matchFn(child) {
+			return child
+		}
+
+		child = nextFunc(child)
+	}
+	return nil
+}
+
+func FindAllNodes(startNode *html.Node, matchFn func(node *html.Node) bool) (foundNodes []*html.Node) {
+	nextFunc := UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		func(node *html.Node) bool {
+			// We should not get higher up than the startNode...
+			return node == startNode
+		},
+	)
+
+	child := startNode.FirstChild
+	for child != nil {
+		if matchFn(child) {
+			foundNodes = append(foundNodes, child)
+		}
+
+		child = nextFunc(child)
+	}
+	return foundNodes
+}

vendor/github.com/JohannesKaufmann/dom/neighbors.go

@@ -0,0 +1,108 @@
+package dom
+
+import "golang.org/x/net/html"
+
+// Warning: It is not meant to be called directly and may change signature from release to release!
+func UNSTABLE_initGetNeighbor(
+	firstChildFunc func(node *html.Node) *html.Node,
+	prevNextFunc func(node *html.Node) *html.Node,
+	goUpUntilFunc func(node *html.Node) bool,
+) func(*html.Node) *html.Node {
+
+	return func(node *html.Node) *html.Node {
+		// First look at the children
+		if child := firstChildFunc(node); child != nil {
+			return child
+		}
+
+		// Otherwise my prev/next sibling
+		if sibling := prevNextFunc(node); sibling != nil {
+			return sibling
+		}
+
+		for {
+			// Finally, continously go upwards until we find an element with a sibling
+			node = node.Parent
+			if node == nil {
+				// We reached the top
+				return nil
+			}
+
+			if goUpUntilFunc(node) {
+				// Don't go too far up...
+				return nil
+			}
+
+			sibling := prevNextFunc(node)
+			if sibling != nil {
+				return sibling
+			}
+		}
+	}
+}
+
+// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //
+// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //
+
+var goUpForever = func(node *html.Node) bool { return false }
+var skipFirstChild = func(node *html.Node) *html.Node { return nil }
+
+func GetPrevNeighborNode(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		PrevSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborElement(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildElement,
+		PrevSiblingElement,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborNodeExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		PrevSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetPrevNeighborElementExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		PrevSiblingElement,
+		goUpForever,
+	)(node)
+}
+
+// - - - - - - - - //
+
+func GetNextNeighborNode(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildNode,
+		NextSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborElement(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		FirstChildElement,
+		NextSiblingElement,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborNodeExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		NextSiblingNode,
+		goUpForever,
+	)(node)
+}
+func GetNextNeighborElementExcludingOwnChild(node *html.Node) *html.Node {
+	return UNSTABLE_initGetNeighbor(
+		skipFirstChild,
+		NextSiblingElement,
+		goUpForever,
+	)(node)
+}

vendor/github.com/JohannesKaufmann/dom/tags.go

@@ -0,0 +1,147 @@
+package dom
+
+import "golang.org/x/net/html"
+
+// In order to stay consistent with v1 of the library, this follows
+// the naming scheme of goquery.
+// E.g. "#text", "div", ...
+func NodeName(node *html.Node) string {
+	if node == nil {
+		return ""
+	}
+
+	switch node.Type {
+	case html.ErrorNode:
+		return "#error"
+	case html.TextNode:
+		return "#text"
+	case html.DocumentNode:
+		return "#document"
+	case html.CommentNode:
+		return "#comment"
+
+	case html.DoctypeNode:
+		// E.g. for `<!DOCTYPE html>` it would be "html"
+		return node.Data
+	case html.ElementNode:
+		// E.g. "div" or "p"
+		return node.Data
+	}
+
+	return ""
+}
+
+func NameIsInlineNode(name string) bool {
+	switch name {
+	case
+		"#text",
+		"a",
+		"abbr",
+		"acronym",
+		"audio",
+		"b",
+		"bdi",
+		"bdo",
+		"big",
+		"br",
+		"button",
+		"canvas",
+		"cite",
+		"code",
+		"data",
+		"datalist",
+		"del",
+		"dfn",
+		"em",
+		"embed",
+		"i",
+		"iframe",
+		"img",
+		"input",
+		"ins",
+		"kbd",
+		"label",
+		"map",
+		"mark",
+		"meter",
+		"noscript",
+		"object",
+		"output",
+		"picture",
+		"progress",
+		"q",
+		"ruby",
+		"s",
+		"samp",
+		"script",
+		"select",
+		"slot",
+		"small",
+		"span",
+		"strong",
+		"sub",
+		"sup",
+		"svg",
+		"template",
+		"textarea",
+		"time",
+		"u",
+		"tt",
+		"var",
+		"video",
+		"wbr":
+		return true
+
+	default:
+		return false
+	}
+}
+
+func NameIsBlockNode(name string) bool {
+	switch name {
+	case
+		"address",
+		"article",
+		"aside",
+		"blockquote",
+		"details",
+		"dialog",
+		"dd",
+		"div",
+		"dl",
+		"dt",
+		"fieldset",
+		"figcaption",
+		"figure",
+		"footer",
+		"form",
+		"h1", "h2", "h3", "h4", "h5", "h6",
+		"header",
+		"hgroup",
+		"hr",
+		"li",
+		"main",
+		"nav",
+		"ol",
+		"p",
+		"pre",
+		"section",
+		"table",
+		"ul":
+
+		return true
+
+	default:
+		return false
+	}
+}
+
+func NameIsHeading(name string) bool {
+	switch name {
+	case
+		"h1", "h2", "h3", "h4", "h5", "h6":
+		return true
+	default:
+		return false
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/.gitignore

@@ -0,0 +1,27 @@
+
+# - - - - - General - - - - - #
+
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, build with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+.DS_Store
+
+
+
+# - - - - - Project Specific - - - - - #
+
+NOTES.md
+.tmp
+
+
+dist/

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/.goreleaser.yaml

@@ -0,0 +1,67 @@
+
+# The lines below are called `modelines`. See `:help modeline`
+# Feel free to remove those if you don't want/need to use them.
+# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
+# vim: set ts=2 sw=2 tw=0 fo=cnqoj
+
+version: 2
+
+before:
+  hooks:
+    - go mod tidy
+
+builds:
+  - env:
+      - CGO_ENABLED=0
+    goos:
+      - linux
+      - windows
+      - darwin
+
+    # Note: We only use goreleaser for the CLI,
+    # so we have to go into the "cli" directory.
+    dir: cli/html2markdown
+    binary: html2markdown
+
+archives:
+  - format: tar.gz
+    # this name template makes the OS and Arch compatible with the results of `uname`.
+    name_template: >-
+      {{ .ProjectName }}_
+      {{- title .Os }}_
+      {{- if eq .Arch "amd64" }}x86_64
+      {{- else if eq .Arch "386" }}i386
+      {{- else }}{{ .Arch }}{{ end }}
+      {{- if .Arm }}v{{ .Arm }}{{ end }}
+    # use zip for windows archives
+    format_overrides:
+      - goos: windows
+        format: zip
+
+brews:
+  - name: html2markdown
+    repository:
+      owner: JohannesKaufmann
+      name: homebrew-tap
+      token: "{{ .Env.TAP_GITHUB_TOKEN }}"
+
+nfpms:
+  - id: default
+    package_name: html2markdown
+
+    vendor: JohannesKaufmann
+    homepage: https://github.com/JohannesKaufmann/html-to-markdown
+    maintainer: Johannes Kaufmann <johannes@joina.de>
+    description: |
+      Convert HTML to Markdown. Even works with entire websites.
+    license: MIT
+
+    formats:
+      - deb
+
+changelog:
+  sort: asc
+  filters:
+    exclude:
+      - "^docs:"
+      - "^test:"

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/ESCAPING.md

@@ -0,0 +1,63 @@
+## Escaping
+
+Some characters have a special meaning in markdown. For example, the character "\*" can be used for lists, emphasis and dividers. By placing a backlash before that character (e.g. "\\\*") you can "escape" it. Then the character will render as a raw "\*" without the _"markdown meaning"_ applied.
+
+But why is "escaping" even necessary?
+
+<!-- prettier-ignore -->
+```md
+Paragraph 1
+-
+Paragraph 2
+```
+
+The markdown above doesn't seem that problematic. But "Paragraph 1" (with only one hyphen below) will be recognized as a _setext heading_.
+
+```html
+<h2>Paragraph 1</h2>
+<p>Paragraph 2</p>
+```
+
+A well-placed backslash character would prevent that...
+
+<!-- prettier-ignore -->
+```md
+Paragraph 1
+\-
+Paragraph 2
+```
+
+---
+
+How to configure escaping? Depending on the `EscapeMode` option, the markdown output is going to be different.
+
+```go
+conv := converter.NewConverter(
+    converter.WithEscapeMode("smart"), // default
+    converter.WithPlugins(
+        // ...
+    )
+)
+```
+
+Lets try it out with this HTML input:
+
+|          |                                                       |
+| -------- | ----------------------------------------------------- |
+| input    | `<p>fake **bold** and real <strong>bold</strong></p>` |
+|          |                                                       |
+|          | **With EscapeMode "smart"**                           |
+| output   | `fake \*\*bold\*\* and real **bold**`                 |
+| rendered | fake \*\*bold\*\* and real **bold**                   |
+|          |                                                       |
+|          | **With EscapeMode "disabled"**                        |
+| output   | `fake **bold** and real **bold**`                     |
+| rendered | fake **bold** and real **bold**                       |
+
+With **smart** escaping, we get some escape characters (the backlash "\\") but it renders correctly.
+
+With escaping **disabled**, the fake and real bold can't be distinguished in the markdown. That means it is both going to render as bold.
+
+---
+
+So now you know the purpose of escaping. However, if you encounter some content where the escaping breaks, you can manually disable it. But please also open an issue!

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2018 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/README.md

@@ -0,0 +1,399 @@
+# html-to-markdown
+
+A robust html-to-markdown converter that transforms HTML (even entire websites) into clean, readable Markdown. It supports complex formatting, customizable options, and plugins for full control over the conversion process.
+
+Use the fully extendable [Golang library](#golang-library) or a quick [CLI command](#cli---using-it-on-the-command-line). Alternatively, try the [Online Demo](https://html-to-markdown.com) or [REST API](https://html-to-markdown.com/api) to see it in action!
+
+Here are some _cool features_:
+
+- **Bold & Italic:** Supports bold and italic—even within single words.
+
+  ![](./.github/images/point_bold_italic.png)
+
+- **List:** Handles ordered and unordered lists with full nesting support.
+
+  ![](./.github/images/point_list.png)
+
+- **Blockquote:** Blockquotes can include other elements, with seamless support for nested quotes.
+
+  ![](./.github/images/point_blockquote.png)
+
+- **Inline Code & Code Block:** Correctly handles backticks and multi-line code blocks, preserving code structure.
+
+  ![](./.github/images/point_code.png)
+
+- **Link & Image:** Properly formats multi-line links, adding escapes for blank lines where needed.
+
+  ![](./.github/images/point_link_image.png)
+
+- **Smart Escaping:** Escapes special characters only when necessary, to avoid accidental Markdown rendering.
+  🗒️ [ESCAPING.md](/ESCAPING.md)
+
+  ![](./.github/images/point_escaping.png)
+
+- **Remove/Keep HTML:** Choose to strip or retain specific HTML tags for ultimate control over output.
+
+  ![](./.github/images/point_wrapper.png)
+
+- **Plugins:** Easily extend with plugins. Or create custom ones to enhance functionality.
+
+  ![](./.github/images/point_strikethrough.png)
+
+- **Table Plugin:** Converts tables with support for alignment, rowspan and colspan.
+
+  ![](./.github/images/point_table.png)
+
+---
+
+## Usage
+
+[💻 Golang library](#golang-library) | [📦 CLI](#cli---using-it-on-the-command-line) | [▶️ Hosted Demo](https://html-to-markdown.com) | [🌐 Hosted REST API](https://html-to-markdown.com/api)
+
+---
+
+## Golang Library
+
+### Installation
+
+```bash
+go get -u github.com/JohannesKaufmann/html-to-markdown/v2
+```
+
+_Or if you want a specific commit add the suffix `/v2@commithash`_
+
+> [!NOTE]  
+> This is the documentation for the v2 library. For the old version switch to the ["v1" branch](https://github.com/JohannesKaufmann/html-to-markdown/tree/v1).
+
+### Usage
+
+[![Go V2 Reference](https://pkg.go.dev/badge/github.com/JohannesKaufmann/html-to-markdown/v2.svg)](https://pkg.go.dev/github.com/JohannesKaufmann/html-to-markdown/v2)
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
+)
+
+func main() {
+	input := `<strong>Bold Text</strong>`
+
+	markdown, err := htmltomarkdown.ConvertString(input)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: **Bold Text**
+}
+```
+
+- 🧑‍💻 [Example code, basics](/examples/basics/main.go)
+
+Use `WithDomain` to convert _relative_ links to _absolute_ links:
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+)
+
+func main() {
+	input := `<img src="/assets/image.png" />`
+
+	markdown, err := htmltomarkdown.ConvertString(
+		input,
+		converter.WithDomain("https://example.com"),
+	)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: ![](https://example.com/assets/image.png)
+}
+```
+
+The function `htmltomarkdown.ConvertString()` is a _small wrapper_ around `converter.NewConverter()` and the _base_ and _commonmark_ plugins. If you want more control, use the following:
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark"
+)
+
+func main() {
+	input := `<strong>Bold Text</strong>`
+
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(
+				commonmark.WithStrongDelimiter("__"),
+				// ...additional configurations for the plugin
+			),
+
+			// ...additional plugins (e.g. table)
+		),
+	)
+
+	markdown, err := conv.ConvertString(input)
+	if err != nil {
+		log.Fatal(err)
+	}
+	fmt.Println(markdown)
+	// Output: __Bold Text__
+}
+```
+
+- 🧑‍💻 [Example code, options](/examples/options/main.go)
+
+> [!NOTE]  
+> If you use `NewConverter` directly make sure to also **register the commonmark and base plugin**.
+
+---
+
+### Collapse & Tag Type
+
+![](./.github/images/tag_type_renderer.png)
+
+You can specify how different HTML tags should be handled during conversion.
+
+- **Tag Types:** When _collapsing_ whitespace it is useful to know if a node is _block_ or _inline_.
+  - So if you have Web Components/Custom Elements remember to register the type using `TagType` or `RendererFor`.
+  - Additionally, you can _remove_ tags completely from the output.
+- **Pre-built Renderers:** There are several pre-built renderers available. For example:
+  - `RenderAsHTML` will render the node (including children) as HTML.
+  - `RenderAsHTMLWrapper` will render the node as HTML and render the children as markdown.
+
+> [!NOTE]  
+> By default, some tags are automatically removed (e.g. `<style>`). You can override existing configuration by using a different _priority_. For example, you could keep `<style>` tags by registering them with `PriorityEarly`.
+
+Here are the examples for the screenshot above:
+
+```go
+conv.Register.TagType("nav", converter.TagTypeRemove, converter.PriorityStandard)
+
+conv.Register.RendererFor("b", converter.TagTypeInline, base.RenderAsHTML, converter.PriorityEarly)
+
+conv.Register.RendererFor("article", converter.TagTypeBlock, base.RenderAsHTMLWrapper, converter.PriorityStandard)
+```
+
+### Plugins
+
+#### Published Plugins
+
+These are the plugins located in the [plugin folder](/plugin):
+
+| Name                  | Description                                                                                        |
+| --------------------- | -------------------------------------------------------------------------------------------------- |
+| Base                  | Implements basic shared functionality (e.g. removing nodes)                                        |
+| Commonmark            | Implements Markdown according to the [Commonmark Spec](https://spec.commonmark.org/)               |
+|                       |                                                                                                    |
+| GitHubFlavored        | _planned_                                                                                          |
+| TaskListItems         | _planned_                                                                                          |
+| Strikethrough         | Converts `<strike>`, `<s>`, and `<del>` to the `~~` syntax.                                        |
+| Table                 | Implements Tables according to the [GitHub Flavored Markdown Spec](https://github.github.com/gfm/) |
+|                       |                                                                                                    |
+| VimeoEmbed            | _planned_                                                                                          |
+| YoutubeEmbed          | _planned_                                                                                          |
+|                       |                                                                                                    |
+| ConfluenceCodeBlock   | _planned_                                                                                          |
+| ConfluenceAttachments | _planned_                                                                                          |
+
+> [!NOTE]  
+> Not all the plugins from v1 are already ported to v2. These will soon be implemented...
+
+These are the plugins in other repositories:
+
+| Name                         | Description         |
+| ---------------------------- | ------------------- |
+| \[Plugin Name\]\(Your Link\) | A short description |
+
+#### Writing Plugins
+
+You want to write custom logic?
+
+1. Write your logic and **register** it.
+
+   ![](./.github/images/autocomplete_register.png)
+   - 🧑‍💻 [Example code, register](/examples/register/main.go)
+
+2. _Optional:_ Package your logic into a **plugin** and publish it.
+   - 🗒️ [WRITING_PLUGINS.md](/WRITING_PLUGINS.md)
+
+---
+
+---
+
+## CLI - Using it on the command line
+
+Using the Golang library provides the most customization, while the CLI is the simplest way to get started.
+
+### Installation
+
+#### Homebrew Tap
+
+```bash
+brew install JohannesKaufmann/tap/html2markdown
+```
+
+#### Debian
+
+A `deb` package is available. See the [Setup Instructions](https://cloudsmith.io/~html-to-markdown/repos/stable/setup/#formats-deb).
+
+_Note: Support for other Linux distributions is tracked in [#119](https://github.com/JohannesKaufmann/html-to-markdown/issues/119)_
+
+#### Pre-compiled Binaries
+
+Download pre-compiled binaries for Linux, macOS or Windows from the [releases page](https://github.com/JohannesKaufmann/html-to-markdown/releases). Extract the archive and copy the executable to a location in your system PATH (e.g. `/usr/local/bin`).
+
+#### Installation via Go
+
+If you have Go installed, you can install the CLI directly using:
+
+```bash
+go install github.com/JohannesKaufmann/html-to-markdown/v2/cli/html2markdown@latest
+```
+
+This will download the source code and compile it into an executable in your Go binary directory (typically `$GOPATH/bin`).
+
+#### Build from Source
+
+Binaries are automatically built via [GoReleaser](https://goreleaser.com/) and attached to each [release](https://github.com/JohannesKaufmann/html-to-markdown/releases).
+
+To build locally (requires Go):
+
+```bash
+go build ./cli/html2markdown
+```
+
+### Version
+
+```bash
+html2markdown --version
+```
+
+> [!NOTE]  
+> Make sure that `--version` prints `2.X.X` as there is a different CLI for V2 of the converter.
+
+### Usage
+
+```bash
+$ echo "<strong>important</strong>" | html2markdown
+
+**important**
+```
+
+```text
+$ curl --no-progress-meter http://example.com | html2markdown
+
+# Example Domain
+
+This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.
+
+[More information...](https://www.iana.org/domains/example)
+```
+
+```bash
+$ html2markdown --input file.html --output file.md
+
+$ html2markdown --input "src/*.html" --output "dist/"
+```
+
+Use `--help` to learn about the configurations, for example:
+
+- `--domain="https://example.com"` to convert _relative_ links to _absolute_ links.
+- `--exclude-selector=".ad"` to exclude the html elements with `class="ad"` from the conversion.
+- `--include-selector="article"` to only include the `<article>` html elements in the conversion.
+- `--plugin-strikethrough` or `--plugin-table` to enable plugins.
+
+_(The cli does not support every option yet. Over time more customization will be added)_
+
+---
+
+---
+
+## FAQ
+
+### Character Encoding
+
+Regardless of which method you use (`ConvertString()`, `ConvertNode()`, etc.), the input is expected to _already_ be UTF-8 encoded. Under the hood, Go's `html.Parse()` is used to build the DOM and it requires UTF-8. It does not detect or convert other charsets.
+
+If your HTML uses a different charset (e.g. `ISO-8859-1`, `Windows-1252`), the replacement character `�` may appear in the output instead of the correct letters.
+
+This library does not handle charset detection or conversion. It is your responsibility to decode the HTML to UTF-8 before passing it in. When fetching HTML over HTTP, the `Content-Type` header tells you the charset. When reading from a file, you may need to detect it from `<meta>` tags or by inspecting the bytes. The [`golang.org/x/net/html/charset`](https://pkg.go.dev/golang.org/x/net/html/charset) package can help with these cases.
+
+### Extending with Plugins
+
+- Need your own logic? Write your own code and then **register** it.
+  - Don't like the **defaults** that the library uses? You can use `PriorityEarly` to run you logic _earlier_ than others.
+
+  - 🧑‍💻 [Example code, register](/examples/register/main.go)
+
+- If you believe that you logic could also benefit others, you can package it up into a **plugin**.
+  - 🗒️ [WRITING_PLUGINS.md](/WRITING_PLUGINS.md)
+
+### Bugs
+
+You found a bug?
+
+[Open an issue](https://github.com/JohannesKaufmann/html-to-markdown/issues/new/choose) with the HTML snippet that does not produce the expected results. Please, please, plase _submit the HTML snippet_ that caused the problem. Otherwise it is very difficult to reproduce and fix...
+
+### Security
+
+This library produces markdown that is readable and can be changed by humans.
+
+Once you convert this markdown back to HTML (e.g. using [goldmark](https://github.com/yuin/goldmark) or [blackfriday](https://github.com/russross/blackfriday)) you need to be careful of malicious content.
+
+This library does NOT sanitize untrusted content. Use an HTML sanitizer such as [bluemonday](https://github.com/microcosm-cc/bluemonday) before displaying the HTML in the browser.
+
+🗒️ [SECURITY.md](/SECURITY.md) if you find a security vulnerability
+
+### Goroutines
+
+You can use the `Converter` from (multiple) goroutines. Internally a mutex is used & there is a test to verify that behaviour.
+
+### Escaping & Backslash
+
+Some characters have a special meaning in markdown (e.g. "\*" for emphasis). The backslash `\` character is used to "escape" those characters. That is perfectly safe and won't be displayed in the final render.
+
+🗒️ [ESCAPING.md](/ESCAPING.md)
+
+### Contributing
+
+You want to contribute? Thats great to hear! There are many ways to help:
+
+Helping to answer questions, triaging issues, writing documentation, writing code, ...
+
+If you want to make a code change: Please first discuss the change you wish to make, by opening an issue. I'm also happy to guide you to where a change is most likely needed. There are also extensive tests (see below) so you can freely experiment 🧑‍🔬
+
+_Note: The outside API should not change because of backwards compatibility..._
+
+### Testing
+
+You don't have to be afraid of breaking the converter, since there are many "Golden File" tests:
+
+Add your problematic HTML snippet to one of the `.in.html` files in the `testdata` folders. Then run `go test -update` and have a look at which `.out.md` files changed in GIT.
+
+You can now change the internal logic and inspect what impact your change has by running `go test -update` again.
+
+_Note: Before submitting your change as a PR, make sure that you run those tests and check the files into GIT..._
+
+### License
+
+Unless otherwise specified, the project is licensed under the terms of the MIT license.
+
+🗒️ [LICENSE](/LICENSE)

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/SECURITY.md

@@ -0,0 +1,6 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+Please report (suspected) security vulnerabilities to johannes@joina.de with the subject _"Security html-to-markdown"_ and you will receive a response within 48 hours.
+

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/WRITING_PLUGINS.md

@@ -0,0 +1,5 @@
+# Writing Plugins
+
+Still in progress, check back later...
+
+TODO: info about writing plugins

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/collapse.go

@@ -0,0 +1,174 @@
+/*
+
+The function to collapse whitespace was adapted from the "turndown" library by Dom Christie,
+which was adapted from the "collapse-whitespace" library by Luc Thevenard.
+
+It was ported from Javascript to Golang by Johannes Kaufmann for the use in the "html-to-markdown" library.
+To increase performance the use of regex was replaced by custom code.
+
+https://github.com/wooorm/collapse-white-space
+https://github.com/mixmark-io/turndown
+https://github.com/JohannesKaufmann/html-to-markdown
+
+-----------
+
+MIT License
+
+Copyright (c) 2017 Dom Christie
+Copyright (c) 2014 Luc Thevenard <lucthevenard@gmail.com>
+Copyright (c) 2018 Johannes Kaufmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+*/
+
+// collapse can collapse whitespace in html elements.
+//
+// It is a port from the Javascript library "turndown" to Golang.
+package collapse
+
+import (
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func nextNode(prev *html.Node, current *html.Node, domFuncs *DomFuncs) *html.Node {
+	if (prev != nil && prev.Parent == current) || domFuncs.IsPreformattedNode(current) {
+		if current.NextSibling != nil {
+			return current.NextSibling
+		}
+
+		return current.Parent
+	}
+
+	if current.FirstChild != nil {
+		return current.FirstChild
+	}
+	if current.NextSibling != nil {
+		return current.NextSibling
+	}
+
+	return current.Parent
+}
+
+func removeNode(node *html.Node) *html.Node {
+	next := node.NextSibling
+	if next == nil {
+		next = node.Parent
+	}
+
+	node.Parent.RemoveChild(node)
+
+	return next
+
+}
+
+type DomFuncs struct {
+	IsBlockNode        func(node *html.Node) bool
+	IsVoidNode         func(node *html.Node) bool
+	IsPreformattedNode func(node *html.Node) bool
+}
+
+func fillDefaultDomFuncs(domFuncs *DomFuncs) *DomFuncs {
+	if domFuncs == nil {
+		domFuncs = &DomFuncs{}
+	}
+	if domFuncs.IsBlockNode == nil {
+		domFuncs.IsBlockNode = defaultIsBlockNode
+	}
+	if domFuncs.IsVoidNode == nil {
+		domFuncs.IsVoidNode = defaultIsVoidNode
+	}
+	if domFuncs.IsPreformattedNode == nil {
+		domFuncs.IsPreformattedNode = defaultIsPreformattedNode
+	}
+	return domFuncs
+
+}
+func Collapse(element *html.Node, domFuncs *DomFuncs) {
+	domFuncs = fillDefaultDomFuncs(domFuncs)
+	// - - - - - - - - - - - - - - - - - - //
+
+	if element.FirstChild == nil || domFuncs.IsPreformattedNode(element) {
+		return
+	}
+
+	var prevText *html.Node = nil
+	var keepLeadingWs = false
+
+	var prev *html.Node = nil
+	var node = nextNode(prev, element, domFuncs)
+
+	for node != element {
+		if node.Type == html.TextNode /* node.nodeType == 4 */ { // Node.TEXT_NODE or Node.CDATA_SECTION_NODE
+			var text = replaceAnyWhitespaceWithSpace(node.Data)
+
+			if (prevText == nil || strings.HasSuffix(prevText.Data, " ")) &&
+				!keepLeadingWs && text != "" && text[0] == ' ' {
+				text = text[1:]
+			}
+
+			// `text` might be empty at this point.
+			if text == "" {
+				node = removeNode(node)
+				continue
+			}
+
+			node.Data = text
+
+			prevText = node
+		} else if node.Type == html.ElementNode { // Node.ELEMENT_NODE
+			if domFuncs.IsBlockNode(node) || dom.NodeName(node) == "br" {
+				if prevText != nil {
+					prevText.Data = strings.TrimSuffix(prevText.Data, " ")
+				}
+
+				prevText = nil
+				keepLeadingWs = false
+			} else if domFuncs.IsVoidNode(node) || domFuncs.IsPreformattedNode(node) || dom.NodeName(node) == "code" {
+				// Avoid trimming space around non-block, non-BR void elements and inline PRE.
+				prevText = nil
+				keepLeadingWs = true
+			} else if prevText != nil {
+				// Drop protection if set previously.
+				keepLeadingWs = false
+			}
+		} else if node.Type == html.CommentNode {
+			// TODO: Is this enough to keep the comments? Does this cause other problems?
+		} else {
+			// E.g. DoctypeNode
+
+			node = removeNode(node)
+			continue
+		}
+
+		var nextNode = nextNode(prev, node, domFuncs)
+		prev = node
+		node = nextNode
+	}
+
+	if prevText != nil {
+		prevText.Data = strings.TrimSuffix(prevText.Data, " ")
+		if prevText.Data == "" {
+			removeNode(prevText)
+		}
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/is_node.go

@@ -0,0 +1,43 @@
+package collapse
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+var blockElements = []string{
+	"address", "article", "aside", "audio", "blockquote", "body", "canvas", "center", "dd", "dir", "div", "dl", "dt", "fieldset", "figcaption", "figure", "footer", "form", "frameset", "h1", "h2", "h3", "h4", "h5", "h6", "header", "hgroup", "hr", "html", "isindex", "li", "main", "menu", "nav", "noframes", "noscript", "ol", "output", "p", "pre", "section", "table", "tbody", "td", "tfoot", "th", "thead", "tr", "ul",
+}
+
+var voidElements = []string{
+	// Note: Compared to the javascript implementation, I removed "source"
+	"area", "base", "br", "col", "command", "embed", "hr", "img", "input", "keygen", "link", "meta", "param" /* "source, "*/, "track", "wbr",
+}
+
+var defaultIsBlockNode = func(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	for _, elem := range blockElements {
+		if elem == name {
+			return true
+		}
+	}
+	return false
+}
+var defaultIsVoidNode = func(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	for _, elem := range voidElements {
+		if elem == name {
+			return true
+		}
+	}
+	return false
+}
+var defaultIsPreformattedNode = func(node *html.Node) bool {
+	// Note: Originally in the javascript version, this just checked for "pre".
+	// I changed it, to also return true for "code"
+	name := dom.NodeName(node)
+
+	return name == "pre" || name == "code"
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/collapse/whitespace.go

@@ -0,0 +1,79 @@
+package collapse
+
+import (
+	"unsafe"
+)
+
+func byteSliceToString(b []byte) string {
+	/* #nosec G103 */
+	return *(*string)(unsafe.Pointer(&b))
+}
+
+func replaceAnyWhitespaceWithSpace(source string) string {
+	if source == "" {
+		return source
+	}
+
+	// Some performance optimizations:
+	// - If no replacement was done, we return the original slice and dont allocate.
+	// - We batch appends
+	var ret []byte
+	makeIfNeeded := func() {
+		if ret == nil {
+			ret = make([]byte, 0, len(source))
+		}
+	}
+
+	startNormal := 0
+	startMatch := -1
+	for i := 0; i < len(source); i++ {
+		isWhitespace := source[i] == ' ' || source[i] == '\r' || source[i] == '\n' || source[i] == '\t'
+
+		if startMatch == -1 && isWhitespace {
+			// Start of newlines
+			startMatch = i
+			continue
+		} else if startMatch != -1 && isWhitespace {
+			// Middle of newlines
+			continue
+		} else if startMatch != -1 {
+			// Character after the last newline character
+
+			count := i - startMatch
+			if count == 1 && source[startMatch] == ' ' {
+				// There was only one `isWhitespace` match & that is a space.
+				// So the replacement would be exactly the same...
+			} else {
+				makeIfNeeded()
+				ret = append(ret, source[startNormal:startMatch]...)
+				ret = append(ret, byte(' '))
+				startNormal = i
+			}
+
+			startMatch = -1
+		}
+	}
+
+	if startMatch == -1 && startNormal == 0 {
+		// a) no changes need to be done
+	} else if startMatch == -1 {
+		// b) Only the normal characters until the end still need to be added
+		makeIfNeeded()
+		ret = append(ret, source[startNormal:]...)
+	} else if ret == nil && len(source)-startMatch == 1 && source[startMatch] == ' ' {
+		// c) There is a match, but it is exactly the same as the replacement
+		//    If there is no new slice, we can skip the replacement.
+	} else {
+		// d) The match still needs to be replaced (and possible the previous normal characters be added)
+		makeIfNeeded()
+		ret = append(ret, source[startNormal:startMatch]...)
+		ret = append(ret, byte(' '))
+	}
+
+	if ret == nil {
+		// Huray, we did not do any allocations with make()
+		// and instead just return the original slice.
+		return source
+	}
+	return byteSliceToString(ret)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/convert.go

@@ -0,0 +1,54 @@
+package htmltomarkdown
+
+import (
+	"io"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark"
+	"golang.org/x/net/html"
+)
+
+// ConvertString converts a html-string to a markdown-string.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func ConvertString(htmlInput string, opts ...converter.ConvertOptionFunc) (string, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertString(htmlInput, opts...)
+}
+
+// ConvertReader converts the html from the reader to markdown.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func ConvertReader(r io.Reader, opts ...converter.ConvertOptionFunc) ([]byte, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertReader(r, opts...)
+}
+
+// ConvertNode converts a `*html.Node` to a markdown byte slice.
+//
+// If you have already parsed an HTML page using the `html.Parse()` function
+// from the "golang.org/x/net/html" package then you can pass this node
+// directly to the converter.
+func ConvertNode(doc *html.Node, opts ...converter.ConvertOptionFunc) ([]byte, error) {
+	conv := converter.NewConverter(
+		converter.WithPlugins(
+			base.NewBasePlugin(),
+			commonmark.NewCommonmarkPlugin(),
+		),
+	)
+
+	return conv.ConvertNode(doc, opts...)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/convert.go

@@ -0,0 +1,142 @@
+package converter
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"io"
+	"slices"
+	"strings"
+
+	"golang.org/x/net/html"
+)
+
+type convertOption struct {
+	domain  string
+	context context.Context
+}
+type ConvertOptionFunc func(o *convertOption)
+
+func WithContext(ctx context.Context) ConvertOptionFunc {
+	return func(o *convertOption) {
+		o.context = ctx
+	}
+}
+
+// WithDomain provides a base `domain` to the converter and
+// to the `AssembleAbsoluteURL` function.
+//
+// If a *relative* url is encountered (in an image or link) then the `domain` is used
+// to convert it to a *absolute* url.
+func WithDomain(domain string) ConvertOptionFunc {
+	return func(o *convertOption) {
+		o.domain = domain
+	}
+}
+
+func (conv *Converter) setError(err error) {
+	conv.m.Lock()
+	defer conv.m.Unlock()
+
+	conv.err = err
+}
+func (conv *Converter) getError() error {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	return conv.err
+}
+
+var errNoRenderHandlers = errors.New(`no render handlers are registered. did you forget to register the "commonmark" and "base" plugins?`)
+var errBasePluginMissing = errors.New(`you registered the "commonmark" plugin but the "base" plugin is also required`)
+
+// ConvertNode converts a `*html.Node` to a markdown byte slice.
+//
+// If you have already parsed an HTML page using the `html.Parse()` function
+// from the "golang.org/x/net/html" package then you can pass this node
+// directly to the converter.
+func (conv *Converter) ConvertNode(doc *html.Node, opts ...ConvertOptionFunc) ([]byte, error) {
+
+	if err := conv.getError(); err != nil {
+		// There can be errors while calling `Init` on the plugins (e.g. validation errors).
+		// Now is the first opportunity where we can return that error.
+		return nil, err
+	}
+
+	conv.m.Lock()
+	option := &convertOption{}
+	for _, fn := range opts {
+		fn(option)
+	}
+	conv.m.Unlock()
+
+	// If there are no render handlers registered this is
+	// usually a user error - since people want the Commonmark Plugin in 99% of cases.
+	if len(conv.getRenderHandlers()) == 0 {
+		return nil, errNoRenderHandlers
+	}
+
+	containsCommonmark := slices.Contains(conv.registeredPlugins, "commonmark")
+	containsBase := slices.Contains(conv.registeredPlugins, "base")
+	if containsCommonmark && !containsBase {
+		return nil, errBasePluginMissing
+	}
+
+	// - - - - - - - - - - - - - - - - - - - //
+
+	state := newGlobalState()
+
+	if option.context == nil {
+		option.context = context.Background()
+	}
+	ctx := option.context
+	ctx = provideDomain(ctx, option.domain)
+	ctx = provideAssembleAbsoluteURL(ctx, defaultAssembleAbsoluteURL)
+	ctx = state.provideGlobalState(ctx)
+
+	customCtx := newConverterContext(ctx, conv)
+
+	// - - - - - - - - - - - - - - - - - - - //
+
+	// Pre-Render
+	for _, handler := range conv.getPreRenderHandlers() {
+		handler.Value(customCtx, doc)
+	}
+
+	// Render
+	var buf bytes.Buffer
+	conv.handleRenderNode(customCtx, &buf, doc)
+
+	// Post-Render
+	result := buf.Bytes()
+	for _, handler := range conv.getPostRenderHandlers() {
+		result = handler.Value(customCtx, result)
+	}
+
+	return result, nil
+}
+
+// ConvertReader converts the html from the reader to markdown.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func (conv *Converter) ConvertReader(r io.Reader, opts ...ConvertOptionFunc) ([]byte, error) {
+	doc, err := html.Parse(r)
+	if err != nil {
+		return nil, err
+	}
+
+	return conv.ConvertNode(doc, opts...)
+}
+
+// ConvertString converts a html-string to a markdown-string.
+//
+// Under the hood `html.Parse()` is used to parse the HTML.
+func (conv *Converter) ConvertString(htmlInput string, opts ...ConvertOptionFunc) (string, error) {
+	r := strings.NewReader(htmlInput)
+	output, err := conv.ConvertReader(r, opts...)
+	if err != nil {
+		return "", err
+	}
+
+	return string(output), nil
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/converter.go

@@ -0,0 +1,72 @@
+package converter
+
+import "sync"
+
+type Converter struct {
+	m sync.RWMutex
+
+	err error
+
+	registeredPlugins []string
+
+	preRenderHandlers  prioritizedSlice[HandlePreRenderFunc]
+	renderHandlers     prioritizedSlice[HandleRenderFunc]
+	postRenderHandlers prioritizedSlice[HandlePostRenderFunc]
+
+	textTransformHandlers prioritizedSlice[HandleTextTransformFunc]
+
+	markdownChars    map[rune]interface{}
+	unEscapeHandlers prioritizedSlice[HandleUnEscapeFunc]
+
+	tagTypes map[string]prioritizedSlice[tagType]
+
+	escapeMode escapeMode
+
+	Register register
+}
+
+type converterOption = func(c *Converter) error
+
+func NewConverter(opts ...converterOption) *Converter {
+	conv := &Converter{
+		markdownChars: make(map[rune]interface{}),
+		tagTypes:      make(map[string]prioritizedSlice[tagType]),
+	}
+	conv.Register = register{conv}
+
+	for _, opt := range opts {
+		err := opt(conv)
+		if err != nil {
+			conv.setError(err)
+			break
+		}
+	}
+
+	return conv
+}
+
+type escapeMode string
+
+const (
+	EscapeModeDisabled escapeMode = "disabled"
+	EscapeModeSmart    escapeMode = "smart"
+)
+
+// WithEscapeMode changes the strictness of the "escaping".
+//
+// Some characters have a special meaning in markdown.
+// For example, the character "*" can be used for lists, emphasis and dividers.
+// By placing a backlash before that character (e.g. "\*") you can "escape" it.
+// Then the character will render as a raw "*" without the "markdown meaning" applied.
+//
+// Learn more in the documentation
+//
+//	"disabled" or "smart"
+//
+//	default: "smart"
+func WithEscapeMode(mode escapeMode) converterOption {
+	return func(c *Converter) error {
+		c.escapeMode = mode
+		return nil
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/ctx.go

@@ -0,0 +1,190 @@
+package converter
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+// func GetValue[K string, V any](ctx context.Context, key K) V {
+// 	val, _ := ctx.Value(key).(V)
+// 	return val
+// }
+// func SetValue[K string, V any](ctx context.Context, key K, val V) context.Context {
+// 	return context.WithValue(ctx, key, val)
+// }
+
+type ctxKey string
+
+const (
+	ctxKeyAssembleAbsoluteURL ctxKey = "AssembleAbsoluteURL"
+	ctxKeyDomain              ctxKey = "Domain"
+
+	ctxKeySetState    ctxKey = "SetState"
+	ctxKeyUpdateState ctxKey = "UpdateState"
+	ctxKeyGetState    ctxKey = "GetState"
+)
+
+func provideDomain(ctx context.Context, domain string) context.Context {
+	return context.WithValue(ctx, ctxKeyDomain, domain)
+}
+func GetDomain(ctx context.Context) string {
+	domain, ok := ctx.Value(ctxKeyDomain).(string)
+	if !ok {
+		fmt.Println("[warning] value ctxKeyDomain is different")
+		return ""
+	}
+
+	return domain
+}
+
+// - - - - - - - - - - - - - - - - - - - - - //
+
+type AssembleAbsoluteURLFunc func(tagName string, rawURL string, domain string) string
+
+func assembleAbsoluteURL(ctx context.Context, tagName string, rawURL string) string {
+	domain := GetDomain(ctx)
+
+	// TODO: since this gets passed down from the converter, it doesn't have to provided from the ctx anymore
+	fn, ok := ctx.Value(ctxKeyAssembleAbsoluteURL).(AssembleAbsoluteURLFunc)
+	if !ok {
+		fmt.Println("[warning] func ctxKeyAssembleAbsoluteURL is different")
+		return ""
+	}
+
+	return fn(tagName, rawURL, domain)
+}
+
+func provideAssembleAbsoluteURL(ctx context.Context, fn AssembleAbsoluteURLFunc) context.Context {
+	return context.WithValue(ctx, ctxKeyAssembleAbsoluteURL, fn)
+}
+
+// - - - - - - - - - - - - - - - - - - - - - //
+
+type SetStateFunc func(key string, val any)
+type UpdateStateFunc func(key string, fn func(any) any)
+type GetStateFunc func(key string) any
+
+type globalState struct {
+	data map[string]any
+}
+
+func newGlobalState() *globalState {
+
+	return &globalState{
+		data: make(map[string]any),
+	}
+}
+
+func (s *globalState) setState(key string, val any) {
+	s.data[key] = val
+}
+func (s *globalState) updateState(key string, fn func(any) any) {
+	s.data[key] = fn(s.data[key])
+}
+func (s *globalState) getState(key string) any {
+	return s.data[key]
+}
+
+func (s *globalState) provideGlobalState(ctx context.Context) context.Context {
+
+	var setState SetStateFunc = s.setState
+	var updateState UpdateStateFunc = s.updateState
+	var getState GetStateFunc = s.getState
+
+	ctx = context.WithValue(ctx, ctxKeySetState, setState)
+	ctx = context.WithValue(ctx, ctxKeyUpdateState, updateState)
+	ctx = context.WithValue(ctx, ctxKeyGetState, getState)
+
+	return ctx
+}
+
+func GetState[V any](ctx context.Context, key string) V {
+	fn := ctx.Value(ctxKeyGetState).(GetStateFunc)
+
+	val, _ := fn(key).(V)
+
+	return val
+}
+
+func SetState[V any](ctx context.Context, key string, val V) {
+	fn := ctx.Value(ctxKeySetState).(SetStateFunc)
+
+	fn(key, val)
+}
+
+func UpdateState[V any](ctx context.Context, key string, fn func(V) V) {
+	updater := ctx.Value(ctxKeyUpdateState).(UpdateStateFunc)
+
+	updater(key, func(val any) any {
+		value, ok := val.(V)
+		if !ok && val != nil {
+			// TODO: slog?
+			fmt.Println("[warning] val is different than V in UpdateState")
+		}
+
+		return fn(value)
+	})
+}
+
+// - - - - - - //
+
+// Context extends the normal context.Context with some additional
+// methods useful for the process of converting.
+type Context interface {
+	context.Context
+
+	AssembleAbsoluteURL(ctx Context, tagName string, rawURL string) string
+
+	GetTagType(tagName string) (tagType, bool)
+
+	RenderNodes(ctx Context, w Writer, nodes ...*html.Node)
+	RenderChildNodes(ctx Context, w Writer, n *html.Node)
+
+	EscapeContent(content []byte) []byte
+	UnEscapeContent(content []byte) []byte
+
+	WithValue(key any, val any) Context
+}
+
+type converterContext struct {
+	context.Context
+	conv *Converter
+}
+
+func newConverterContext(ctx context.Context, conv *Converter) Context {
+	return &converterContext{
+		Context: ctx,
+		conv:    conv,
+	}
+}
+
+func (c *converterContext) AssembleAbsoluteURL(ctx Context, tagName string, rawURL string) string {
+	return assembleAbsoluteURL(ctx, tagName, rawURL)
+}
+
+func (c *converterContext) RenderNodes(ctx Context, w Writer, nodes ...*html.Node) {
+	c.conv.handleRenderNodes(ctx, w, nodes...)
+}
+func (c *converterContext) RenderChildNodes(ctx Context, w Writer, n *html.Node) {
+	c.conv.handleRenderNodes(ctx, w, dom.AllChildNodes(n)...)
+}
+
+func (c *converterContext) GetTagType(tagName string) (tagType, bool) {
+	return c.conv.getTagType(tagName)
+}
+func (c *converterContext) EscapeContent(content []byte) []byte {
+	return c.conv.escapeContent(content)
+}
+func (c *converterContext) UnEscapeContent(content []byte) []byte {
+	return c.conv.unEscapeContent(content)
+}
+
+func (c *converterContext) WithValue(key any, val any) Context {
+	return &converterContext{
+		Context: context.WithValue(c.Context, key, val),
+		conv:    c.conv,
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/escape.go

@@ -0,0 +1,89 @@
+package converter
+
+import (
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+)
+
+const (
+	actionKeep   = iota
+	actionEscape = iota
+)
+
+// IMPORTANT: Only internally we assume it is only byte
+var placeholderByte byte = marker.BytesMarkerEscaping[0]
+
+func (conv *Converter) escapeContent(chars []byte) []byte {
+	if conv.escapeMode == EscapeModeDisabled {
+		return chars
+	}
+
+	newChars := make([]byte, 0, len(chars))
+	for index := 0; index < len(chars); index++ {
+		if chars[index] == '\u0000' {
+			// For security reasons, the Unicode character U+0000 must be replaced with the REPLACEMENT CHARACTER (U+FFFD).
+			newChars = append(newChars, []byte(string('\ufffd'))...)
+			continue
+		}
+
+		r, _ := utf8.DecodeRune(chars[index:])
+
+		isMarkdownChar := conv.checkIsEscapedChar(r)
+		if isMarkdownChar {
+			newChars = append(newChars, placeholderByte, chars[index])
+		} else {
+			newChars = append(newChars, chars[index])
+		}
+	}
+
+	return newChars
+}
+
+func (conv *Converter) unEscapeContent(chars []byte) []byte {
+	if conv.escapeMode == EscapeModeDisabled {
+		return chars
+	}
+
+	checkElements := func(index int) int {
+		for _, handler := range conv.getUnEscapeHandlers() {
+			if skip := handler.Value(chars, index); skip != -1 {
+				return skip
+			}
+		}
+
+		return -1
+	}
+
+	changes := make([]uint8, len(chars))
+	for index := 0; index < len(chars); index++ {
+
+		if chars[index] != placeholderByte {
+			continue
+		}
+		if index+1 >= len(chars) {
+			break
+		}
+
+		skip := checkElements(index + 1)
+		if skip == -1 {
+			continue
+		}
+		changes[index] = actionEscape
+		index += skip - 1
+	}
+
+	newChars := make([]byte, 0, len(chars))
+	for index, char := range chars {
+		if char != placeholderByte {
+			newChars = append(newChars, char)
+			continue
+		}
+
+		// What to do with this placeholder? Should we escape or not?
+		if changes[index] == actionEscape {
+			newChars = append(newChars, '\\')
+		}
+	}
+	return newChars
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/plugin.go

@@ -0,0 +1,22 @@
+package converter
+
+// Plugin can be used to extends functionality beyond what
+// is offered by commonmark.
+type Plugin interface {
+	// The public name of the plugin, e.g. "strikethrough"
+	Name() string
+
+	// Init is called to initialize the plugin. It can be used to
+	// *validate* the arguments and *register* the rules.
+	Init(conv *Converter) error
+}
+
+// WithPlugins can be used to add additional functionality to the converter.
+func WithPlugins(plugins ...Plugin) converterOption {
+	return func(c *Converter) error {
+		for _, plugin := range plugins {
+			c.Register.Plugin(plugin)
+		}
+		return nil
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/prioritized.go

@@ -0,0 +1,33 @@
+package converter
+
+import "sort"
+
+const (
+	// PriorityEarly means that the handler will be run **early** in the process.
+	// To run it even earlier you need to subtract from this number.
+	PriorityEarly = 100
+
+	// PriorityStandard is for handlers that don't need to be run in a particular order.
+	PriorityStandard = 500
+
+	// PriorityLate means that the handler will be run **late** in the process.
+	// To run it even later you need to add to this number.
+	PriorityLate = 1000
+)
+
+type prioritizedValue[V any] struct {
+	Value    V
+	Priority int
+}
+
+type prioritizedSlice[V any] []prioritizedValue[V]
+
+func (s prioritizedSlice[V]) Sort() {
+	sort.Slice(s, func(i, j int) bool {
+		return s[i].Priority < s[j].Priority
+	})
+}
+
+func prioritized[V any](v V, priority int) prioritizedValue[V] {
+	return prioritizedValue[V]{v, priority}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/register.go

@@ -0,0 +1,222 @@
+package converter
+
+import (
+	"errors"
+	"fmt"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+type register struct {
+	conv *Converter
+}
+
+func (r *register) Plugin(plugin Plugin) {
+	pluginName := plugin.Name()
+	if pluginName == "" {
+		r.conv.setError(errors.New("the plugin has no name"))
+		return
+	}
+
+	r.conv.m.Lock()
+	r.conv.registeredPlugins = append(r.conv.registeredPlugins, pluginName)
+	r.conv.m.Unlock()
+
+	err := plugin.Init(r.conv)
+	if err != nil {
+		r.conv.setError(fmt.Errorf("error while initializing %q plugin: %w", pluginName, err))
+		return
+	}
+}
+
+// - - - - - - - - - - - - - Pre-Render - - - - - - - - - - - - - //
+
+type HandlePreRenderFunc func(ctx Context, doc *html.Node)
+
+func (r *register) PreRenderer(fn HandlePreRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.preRenderHandlers = append(r.conv.preRenderHandlers, handler)
+}
+func (conv *Converter) getPreRenderHandlers() prioritizedSlice[HandlePreRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandlePreRenderFunc], len(conv.preRenderHandlers))
+	copy(handlers, conv.preRenderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Render - - - - - - - - - - - - - //
+
+// Writer is an interface that only conforms to the Write* methods of bytes.Buffer
+type Writer interface {
+	Write(p []byte) (n int, err error)
+	WriteByte(c byte) error
+	WriteRune(r rune) (n int, err error)
+	WriteString(s string) (n int, err error)
+}
+
+type HandleRenderFunc func(ctx Context, w Writer, n *html.Node) RenderStatus
+
+func (r *register) Renderer(fn HandleRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.renderHandlers = append(r.conv.renderHandlers, handler)
+}
+
+// RendererFor registers a renderer for a specific tag (e.g. "div").
+// It is a small wrapper around `TagType()` and `Renderer()`.
+func (r *register) RendererFor(tagName string, tagType tagType, renderFn HandleRenderFunc, priority int) {
+
+	// 1. we add the "tagType" to the map
+	r.TagType(tagName, tagType, priority)
+
+	// 2. we register the render function
+	r.Renderer(func(ctx Context, w Writer, n *html.Node) RenderStatus {
+		if dom.NodeName(n) == tagName {
+			return renderFn(ctx, w, n)
+		}
+		return RenderTryNext
+	}, priority)
+}
+
+func (conv *Converter) getRenderHandlers() prioritizedSlice[HandleRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleRenderFunc], len(conv.renderHandlers))
+	copy(handlers, conv.renderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Post Render - - - - - - - - - - - - - //
+
+type HandlePostRenderFunc func(ctx Context, content []byte) []byte
+
+func (r *register) PostRenderer(fn HandlePostRenderFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.postRenderHandlers = append(r.conv.postRenderHandlers, handler)
+}
+func (conv *Converter) getPostRenderHandlers() prioritizedSlice[HandlePostRenderFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandlePostRenderFunc], len(conv.postRenderHandlers))
+	copy(handlers, conv.postRenderHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Text - - - - - - - - - - - - - //
+
+type HandleTextTransformFunc func(ctx Context, content string) string
+
+func (r *register) TextTransformer(fn HandleTextTransformFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.textTransformHandlers = append(r.conv.textTransformHandlers, handler)
+}
+func (conv *Converter) getTextTransformHandlers() prioritizedSlice[HandleTextTransformFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleTextTransformFunc], len(conv.textTransformHandlers))
+	copy(handlers, conv.textTransformHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Escaping - - - - - - - - - - - - - //
+
+func (r *register) EscapedChar(chars ...rune) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	for _, char := range chars {
+		r.conv.markdownChars[char] = struct{}{}
+	}
+}
+func (conv *Converter) checkIsEscapedChar(r rune) bool {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	_, ok := conv.markdownChars[r]
+	return ok
+}
+
+type HandleUnEscapeFunc func(chars []byte, index int) int
+
+func (r *register) UnEscaper(fn HandleUnEscapeFunc, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	handler := prioritized(fn, priority)
+	r.conv.unEscapeHandlers = append(r.conv.unEscapeHandlers, handler)
+}
+func (conv *Converter) getUnEscapeHandlers() prioritizedSlice[HandleUnEscapeFunc] {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	handlers := make(prioritizedSlice[HandleUnEscapeFunc], len(conv.unEscapeHandlers))
+	copy(handlers, conv.unEscapeHandlers)
+	handlers.Sort()
+
+	return handlers
+}
+
+// - - - - - - - - - - - - - Tag Type - - - - - - - - - - - - - //
+
+type tagType string
+
+const (
+	TagTypeBlock  tagType = "block"
+	TagTypeInline tagType = "inline"
+
+	// TagTypeRemove will remove that node in the _PreRender_ phase with a high priority.
+	TagTypeRemove tagType = "remove"
+)
+
+func (r *register) TagType(tagName string, tagType tagType, priority int) {
+	r.conv.m.Lock()
+	defer r.conv.m.Unlock()
+
+	val := prioritized(tagType, priority)
+	r.conv.tagTypes[tagName] = append(r.conv.tagTypes[tagName], val)
+}
+func (conv *Converter) getTagType(tagName string) (tagType, bool) {
+	conv.m.RLock()
+	defer conv.m.RUnlock()
+
+	types, ok := conv.tagTypes[tagName]
+	if !ok || len(types) == 0 {
+
+		if dom.NameIsBlockNode(tagName) {
+			return TagTypeBlock, true
+		} else if dom.NameIsInlineNode(tagName) {
+			return TagTypeInline, true
+		}
+		return "", false
+	}
+
+	types.Sort()
+	firstType := types[0].Value
+
+	return firstType, true
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/render.go

@@ -0,0 +1,61 @@
+package converter
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func (conv *Converter) handleRenderNodes(ctx Context, w Writer, nodes ...*html.Node) {
+	for _, node := range nodes {
+		conv.handleRenderNode(ctx, w, node)
+	}
+}
+
+func (conv *Converter) handleRenderNode(ctx Context, w Writer, node *html.Node) RenderStatus {
+	name := dom.NodeName(node)
+
+	// - - A: the #text node - - //
+	if name == "#text" {
+		return conv.handleRenderText(ctx, w, node)
+	}
+
+	// - - B: the render handlers - - //
+	for _, handler := range conv.getRenderHandlers() {
+		status := handler.Value(ctx, w, node)
+		if status == RenderSuccess {
+			return status
+		}
+	}
+
+	// - - C: the fallback - - //
+	// If nothing works we fallback to this:
+	return conv.handleRenderFallback(ctx, w, node)
+}
+
+func (conv *Converter) handleRenderFallback(ctx Context, w Writer, node *html.Node) RenderStatus {
+
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == TagTypeBlock {
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+	}
+	ctx.RenderChildNodes(ctx, w, node)
+	if tagType == TagTypeBlock {
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+	}
+
+	return RenderSuccess
+}
+func (conv *Converter) handleRenderText(ctx Context, w Writer, node *html.Node) RenderStatus {
+	content := node.Data
+
+	for _, handler := range conv.getTextTransformHandlers() {
+		content = handler.Value(ctx, content)
+	}
+
+	w.WriteString(content)
+	return RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/status.go

@@ -0,0 +1,8 @@
+package converter
+
+type RenderStatus int
+
+const (
+	RenderTryNext RenderStatus = iota
+	RenderSuccess
+)

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/converter/url.go

@@ -0,0 +1,114 @@
+package converter
+
+import (
+	"net/url"
+	"strings"
+)
+
+var percentEncodingReplacer = strings.NewReplacer(
+	" ", "%20",
+	"[", "%5B",
+	"]", "%5D",
+	"(", "%28",
+	")", "%29",
+	"<", "%3C",
+	">", "%3E",
+)
+
+func parseBaseDomain(rawDomain string) *url.URL {
+	if rawDomain == "" {
+		return nil
+	}
+
+	u1, err := url.Parse(rawDomain)
+	if err == nil && u1.Host != "" {
+		// Yes, we got valid domain (probably with a http/https scheme)
+		return u1
+	}
+
+	u2, err := url.Parse("http://" + rawDomain)
+	if err == nil && u2.Host != "" {
+		// Yes, we got a valid domain (by choosing a fallback scheme)
+		return u2
+	}
+
+	return nil
+}
+func defaultAssembleAbsoluteURL(tagName string, rawURL string, domain string) string {
+	rawURL = strings.TrimSpace(rawURL)
+
+	if rawURL == "#" {
+		// Golangs url.Parse does not seem to distinguish between
+		// no fragment and an empty fragment.
+		return rawURL
+	}
+
+	// Increase the chance that the url will be parsed
+	rawURL = strings.ReplaceAll(rawURL, "\n", "%0A")
+	rawURL = strings.ReplaceAll(rawURL, "\t", "%09")
+
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		// We can't do anything with this url because it is invalid
+		return percentEncodingReplacer.Replace(rawURL)
+	}
+
+	if u.Scheme == "data" {
+		// This is a data uri (for example an inline base64 image)
+		return percentEncodingReplacer.Replace(rawURL)
+	}
+
+	// The default Query().Encode() encodes the query parameters "sorted by key".
+	// Instead we want to keep the original order, but still encode the parameters.
+	u.RawQuery = ParseAndEncodeQuery(u.RawQuery)
+
+	// For better compatibility (especially in regards to mailto links),
+	// instead of encoding a space with a "+" we use ""%20" to prevent
+	// e.g. the email reading "Hi+Johannes" instead of "Hi Johannes"
+	u.RawQuery = strings.ReplaceAll(u.RawQuery, "+", "%20")
+
+	if base := parseBaseDomain(domain); base != nil {
+		// If a "domain" is provided, we use that to convert relative links
+		// to absolute links.
+		u = base.ResolveReference(u)
+	}
+
+	return percentEncodingReplacer.Replace(u.String())
+}
+
+// - - - - //
+
+func decodeAndEncode(original string) string {
+	s, err := url.QueryUnescape(original)
+	if err != nil {
+		return original
+	}
+
+	return url.QueryEscape(s)
+}
+
+func ParseAndEncodeQuery(rawQuery string) string {
+	if rawQuery == "" {
+		return ""
+	}
+
+	rawParts := strings.Split(rawQuery, "&")
+	encodedParts := make([]string, len(rawParts))
+
+	for i, part := range rawParts {
+		splitted := strings.SplitN(part, "=", 2)
+
+		if len(splitted) == 1 {
+			// A: Just the key
+			encodedParts[i] = decodeAndEncode(splitted[0])
+		} else if splitted[1] == "" {
+			// B: The key and the equal sign
+			encodedParts[i] = decodeAndEncode(splitted[0]) + "="
+		} else {
+			// C: The key and the equal sign and the value
+			encodedParts[i] = decodeAndEncode(splitted[0]) + "=" + decodeAndEncode(splitted[1])
+		}
+	}
+
+	return strings.Join(encodedParts, "&")
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/add_space.go

@@ -0,0 +1,66 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func getFirstChildNode(startNode *html.Node, matchFn func(n *html.Node) bool) *html.Node {
+	node := startNode.FirstChild
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+		} else if matchFn(node) {
+			return node
+		} else {
+			return nil
+		}
+	}
+
+	return nil
+}
+
+func getLastChildNode(startNode *html.Node, matchFn func(n *html.Node) bool) *html.Node {
+	node := startNode.LastChild
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetPrevNeighborNode(node)
+		} else if matchFn(node) {
+			return node
+		} else {
+			return nil
+		}
+	}
+
+	return nil
+}
+func AddSpace(ctx context.Context, doc *html.Node, isOuterNode, isInnerNode func(*html.Node) bool) {
+	node := doc
+	for node != nil {
+		if isOuterNode(node) {
+			firstChild := getFirstChildNode(node, isInnerNode)
+			if firstChild != nil {
+				prev := getPrevTextNode(node)
+				if prev != nil {
+					prev.Data = prev.Data + " "
+				}
+			}
+
+			lastChild := getLastChildNode(node, isInnerNode)
+			if lastChild != nil {
+				next := getNextTextNode(node)
+				if next != nil {
+					next.Data = " " + next.Data
+				}
+			}
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/adjacent.go

@@ -0,0 +1,75 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func collectAdjacentNodes(node *html.Node, matchFn func(n *html.Node) bool) []*html.Node {
+	var collectedNodes []*html.Node
+
+	node = node.NextSibling
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+		} else if matchFn(node) {
+			collectedNodes = append(collectedNodes, node)
+			node = dom.GetNextNeighborNodeExcludingOwnChild(node)
+		} else {
+			// Return the collected nodes
+			return collectedNodes
+		}
+	}
+
+	return collectedNodes
+}
+
+func mergeChildren(destinationNode *html.Node, nodes ...*html.Node) {
+	for _, node := range nodes {
+		// We move all the children to the `destinationNode`.
+		children := dom.AllChildNodes(node)
+		for _, child := range children {
+			dom.RemoveNode(child)
+			destinationNode.AppendChild(child)
+		}
+		dom.RemoveNode(node)
+	}
+}
+
+func MergeAdjacent(doc *html.Node, matchFn func(*html.Node) bool) {
+	node := doc
+
+	for node != nil {
+		if matchFn(node) {
+			nextNodes := collectAdjacentNodes(node, matchFn)
+
+			mergeChildren(node, nextNodes...)
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}
+
+// - - - - - - - - //
+
+func MergeAdjacentTextNodes(n *html.Node) {
+	if n == nil {
+		return
+	}
+
+	var prev *html.Node
+	for c := n.FirstChild; c != nil; {
+		next := c.NextSibling
+		if c.Type == html.TextNode && prev != nil && prev.Type == html.TextNode {
+			// Combine adjacent text nodes
+			prev.Data += c.Data
+			n.RemoveChild(c)
+		} else {
+			MergeAdjacentTextNodes(c)
+			prev = c
+		}
+		c = next
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/alternatives.go

@@ -0,0 +1,114 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+// TODO: make this configurable via the options???
+func getMarkdownStructure(name string) string {
+	switch name {
+	case "#document", "html", "head", "body",
+		"blockquote", "ul", "ol", "li":
+		// A container block can also contain other blocks.
+		return "container_block"
+
+	// Note: "p" would also be part of "leaf_block"
+	case "hr", "pre",
+		"h1", "h2", "h3", "h4", "h5", "h6":
+		// Leaf blocks can contain inline content
+		// but NOT other blocks.
+		return "leaf_block"
+
+	case "#text", "span", "code",
+		"b", "strong", "i", "em",
+		"a", "img", "br":
+		return "inline"
+
+	case "div", "p":
+		// Since these are just placing newlines,
+		// we dont categorize them.
+		return ""
+
+	default:
+		return ""
+	}
+}
+
+func headingAlternative(ctx context.Context, node *html.Node) {
+	node.Data = "strong"
+
+	newChild := &html.Node{
+		Type: html.ElementNode,
+		Data: "br",
+	}
+	node.Parent.InsertBefore(newChild, node.NextSibling)
+}
+func blockquoteAlternative(ctx context.Context, node *html.Node) {
+	newBefore := &html.Node{Type: html.TextNode, Data: ` "`}
+	node.Parent.InsertBefore(newBefore, node)
+
+	node.Data = "span"
+
+	newAfter := &html.Node{Type: html.TextNode, Data: `" `}
+	node.Parent.InsertBefore(newAfter, node.NextSibling)
+}
+func preAlternative(ctx context.Context, node *html.Node) {
+	node.Data = "code"
+}
+func hrAlternative(ctx context.Context, node *html.Node) {
+	dom.RemoveNode(node)
+}
+
+// TODO: make this configurable via the options?
+var alternatives = map[string]func(ctx context.Context, node *html.Node){
+	"h1":         headingAlternative,
+	"h2":         headingAlternative,
+	"h3":         headingAlternative,
+	"h4":         headingAlternative,
+	"h5":         headingAlternative,
+	"h6":         headingAlternative,
+	"blockquote": blockquoteAlternative,
+	"pre":        preAlternative,
+	"hr":         hrAlternative,
+}
+
+func LeafBlockAlternatives(ctx context.Context, doc *html.Node) {
+	var finder func(node *html.Node, isInsideLeafBlock bool, isInsideInline bool)
+	finder = func(node *html.Node, isInsideLeafBlock bool, isInsideInline bool) {
+		name := dom.NodeName(node)
+
+		structure := getMarkdownStructure(name)
+		if (structure == "container_block" || structure == "leaf_block") && (isInsideLeafBlock || isInsideInline) {
+			// A block inside an inline OR a block inside a leaf-block
+			// is not valid markdown so cannot be rendered.
+			//
+			// For example, you cannot place a blockquote inside a heading.
+			//
+			// Instead of this weird output (## Heading > My Quote)
+			// we try to find alternatives (## Heading "My Quote")
+			fn, ok := alternatives[name]
+			if ok {
+				fn(ctx, node)
+			} else {
+				node.Data = "span"
+			}
+		}
+
+		// - - - - - - - - - - - - - - - - - - - - - - //
+
+		if structure == "leaf_block" {
+			isInsideLeafBlock = true
+		}
+		if structure == "inline" {
+			isInsideInline = true
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			defer finder(child, isInsideLeafBlock, isInsideInline)
+		}
+	}
+	finder(doc, false, false)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/domutils.go

@@ -0,0 +1,43 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func getNextTextNode(startNode *html.Node) *html.Node {
+	node := dom.GetNextNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		if node.Type == html.TextNode {
+			return node
+		}
+
+		if dom.NodeName(node) == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetNextNeighborNode(node)
+			continue
+		}
+
+		return nil
+	}
+	return nil
+}
+func getPrevTextNode(startNode *html.Node) *html.Node {
+	node := dom.GetPrevNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		if node.Type == html.TextNode {
+			return node
+		}
+
+		if dom.NodeName(node) == "span" {
+			// A span has no special meaning. So we just skip it...
+			node = dom.GetPrevNeighborNode(node)
+			continue
+		}
+
+		return nil
+	}
+	return nil
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/empty_code.go

@@ -0,0 +1,42 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func hasTextChildNodes(startNode *html.Node) bool {
+	var found bool
+
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		if node.Type == html.TextNode && node.Data != "" {
+			found = true
+			return
+		}
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(startNode)
+
+	return found
+}
+
+func RemoveEmptyCode(ctx context.Context, doc *html.Node) {
+	node := doc
+	for node != nil {
+		if dom.NodeName(node) == "code" && !hasTextChildNodes(node) {
+			next := dom.GetNextNeighborNodeExcludingOwnChild(node)
+
+			dom.RemoveNode(node)
+
+			node = next
+			continue
+		}
+
+		node = dom.GetNextNeighborNode(node)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/list_end_comment.go

@@ -0,0 +1,74 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+var ListEndCommentData = "THE END"
+
+func AddListEndComments(ctx context.Context, doc *html.Node) {
+	node := doc
+	for node != nil {
+		if nameIsList(node) && nextNameIsList(node) {
+			insertComment(node)
+		}
+
+		node = dom.GetNextNeighborElement(node)
+	}
+}
+
+func nameIsList(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "ul" || name == "ol"
+}
+
+func insertComment(listNode *html.Node) {
+	comment := &html.Node{
+		Type: html.CommentNode,
+		Data: ListEndCommentData,
+	}
+	listNode.Parent.InsertBefore(comment, listNode.NextSibling)
+}
+
+func nextNameIsList(startNode *html.Node) bool {
+	node := dom.GetNextNeighborNodeExcludingOwnChild(startNode)
+
+	for node != nil {
+		name := dom.NodeName(node)
+		if name == "ul" || name == "ol" {
+			return true
+		}
+		if name == "li" {
+			return false
+		}
+		if name == "#comment" && node.Data == ListEndCommentData {
+			return false
+		}
+
+		// If there is any text between two lists
+		// they are automatically not connected anymore.
+		if node.Type == html.TextNode {
+			return false
+		}
+
+		// - - - - //
+
+		if name == "hr" {
+			// A divider already seperates two lists...
+			return false
+		}
+
+		// TODO: RunContext.Render()
+		// -> get acess to keepRemoveMap
+
+		// TODO: look in the KeepRemoveMap?
+		// e.g. ul then script then ul
+
+		node = dom.GetNextNeighborNode(node)
+		continue
+	}
+	return false
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/list_items.go

@@ -0,0 +1,53 @@
+package domutils
+
+import (
+	"context"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+	"golang.org/x/net/html/atom"
+)
+
+// MoveListItems moves non-"li" nodes into the previous "li" nodes.
+func MoveListItems(ctx context.Context, n *html.Node) {
+	if n.Type == html.ElementNode && (n.Data == "ol" || n.Data == "ul") {
+		var previousLi *html.Node
+
+		// Collect children to avoid modifying the slice while iterating.
+		children := dom.AllChildNodes(n)
+
+		for _, child := range children {
+			if child.Type == html.ElementNode && child.Data == "li" {
+				previousLi = child
+			} else if child.Type == html.TextNode && strings.TrimSpace(child.Data) == "" {
+				// Skip the node, probably just formatting of code
+			} else {
+				// We expect that inside an "ol"/"ul" there are *only* "li" nodes.
+				// But sometimes that is not the case...
+
+				if previousLi != nil {
+					// There is a previous "li" node,
+					// so we move this content into the other "li" node.
+					n.RemoveChild(child)
+
+					previousLi.AppendChild(child)
+				} else {
+					// There is no previous "li" node,
+					// so we wrap this node with it's own "li" node.
+
+					newNode := &html.Node{
+						Type:     html.ElementNode,
+						DataAtom: atom.Li,
+						Data:     "li",
+					}
+					previousLi = dom.WrapNode(child, newNode)
+				}
+			}
+		}
+	}
+
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		MoveListItems(ctx, c)
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/redundant.go

@@ -0,0 +1,29 @@
+package domutils
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func RemoveRedundant(doc *html.Node, matchFn func(*html.Node, *html.Node) bool) {
+	for _, node := range dom.AllNodes(doc) {
+		if hasSameTypeAncestor(node, matchFn) {
+			dom.UnwrapNode(node)
+		}
+	}
+
+}
+
+func hasSameTypeAncestor(n *html.Node, matchFn func(*html.Node, *html.Node) bool) bool {
+	if !matchFn(n, n) {
+		return false
+	}
+
+	for p := n.Parent; p != nil; p = p.Parent {
+		if matchFn(n, p) {
+			return true
+		}
+	}
+
+	return false
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/span.go

@@ -0,0 +1,50 @@
+package domutils
+
+import (
+	"context"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func isFakeSpan(node *html.Node) bool {
+	name := dom.NodeName(node)
+	if name != "span" {
+		return false
+	}
+
+	var containsBlockNode = false
+
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		name := dom.NodeName(node)
+		if dom.NameIsBlockNode(name) {
+			containsBlockNode = true
+			return
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(node)
+
+	return containsBlockNode
+}
+
+// RenameFakeSpans renames all "span" nodes to "div" if
+// any block element is found as a child.
+func RenameFakeSpans(ctx context.Context, doc *html.Node) {
+	var finder func(node *html.Node)
+	finder = func(node *html.Node) {
+		if isFakeSpan(node) {
+			node.Data = "div"
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+
+	finder(doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils/swap.go

@@ -0,0 +1,51 @@
+package domutils
+
+import (
+	"context"
+	"slices"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"golang.org/x/net/html"
+)
+
+func swapTagsOfNodes(node1, node2 *html.Node) {
+	if node1.Type != html.ElementNode || node2.Type != html.ElementNode {
+		panic("swap only works with element nodes")
+	}
+
+	tempDataAtom := node1.DataAtom
+	tempData := node1.Data
+	tempAttr := node1.Attr
+
+	node1.DataAtom = node2.DataAtom
+	node1.Data = node2.Data
+	node1.Attr = node2.Attr
+
+	node2.DataAtom = tempDataAtom
+	node2.Data = tempData
+	node2.Attr = tempAttr
+}
+
+func isEmptyText(node *html.Node) bool {
+	return node.Type == html.TextNode && strings.TrimSpace(node.Data) == ""
+}
+func SwapTags(ctx context.Context, doc *html.Node, isOuterNode, isInnerNode func(*html.Node) bool) {
+	var finder func(*html.Node)
+	finder = func(node *html.Node) {
+		if isOuterNode(node) {
+			childs := dom.AllChildNodes(node)
+			childs = slices.DeleteFunc(childs, isEmptyText)
+
+			if len(childs) == 1 && isInnerNode(childs[0]) {
+				swapTagsOfNodes(node, childs[0])
+				return
+			}
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			finder(child)
+		}
+	}
+	finder(doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_backslash.go

@@ -0,0 +1,9 @@
+package escape
+
+func IsBackslash(chars []byte, index int) int {
+	if chars[index] != '\\' {
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_code.go

@@ -0,0 +1,45 @@
+package escape
+
+func IsFencedCode(chars []byte, index int) int {
+	if chars[index] != '`' && chars[index] != '~' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == ' ' || chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == '\n' {
+			break
+		}
+
+		return -1
+	}
+
+	count := 1
+	i := index + 1
+	for ; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == '`' || chars[i] == '~' {
+			count++
+			continue
+		}
+
+		break
+	}
+	if count < 3 {
+		return -1
+	}
+
+	return i - index
+}
+
+func IsInlineCode(chars []byte, index int) int {
+	if chars[index] != '`' {
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_divider.go

@@ -0,0 +1,47 @@
+package escape
+
+func IsDivider(chars []byte, index int) int {
+	if chars[index] != '-' && chars[index] != '_' && chars[index] != '*' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+
+		return -1
+	}
+
+	count := 1
+	lastChar := len(chars)
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == chars[index] {
+			count++
+			continue
+		}
+		if chars[i] == '\n' {
+			lastChar = i
+			break
+		}
+
+		return -1
+	}
+
+	if count >= 3 {
+		return lastChar - index
+	}
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_header.go

@@ -0,0 +1,77 @@
+package escape
+
+func IsAtxHeader(chars []byte, index int) int {
+	if chars[index] != '#' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		return -1
+	}
+
+	nPoundSigns := 1
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == '#' {
+			nPoundSigns++
+
+			if nPoundSigns > 6 {
+				return -1
+			}
+			continue
+		}
+
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' || chars[i] == '\t' || chars[i] == '\n' || chars[i] == '\r' {
+			// TODO: fix calculation with placeholder (maybe own for loop construct?)
+			// Returns the count of # that we encountered
+			return i - index
+		}
+
+		return -1
+	}
+	return 1
+}
+
+func IsSetextHeader(chars []byte, index int) int {
+	if chars[index] != '=' && chars[index] != '-' {
+		return -1
+	}
+
+	var newlineCount int
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte || chars[i] == ' ' {
+			continue
+		}
+
+		if chars[i] == '\n' {
+			newlineCount++
+			continue
+		}
+
+		if newlineCount == 0 {
+			// Without any newlines, this character is on the same line
+			// as the delimiter. So the delimiter is inside a normal text...
+			return -1
+		} else if newlineCount == 1 {
+			// The heading content is on the line above the delimiter
+			// which qualifies for a setext heading...
+			return 1
+		} else {
+			return -1
+		}
+
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_image.go

@@ -0,0 +1,36 @@
+package escape
+
+func IsImageOrLink(chars []byte, index int) int {
+	if chars[index] == '!' {
+		return isImageOrLinkStartExclamation(chars, index)
+	}
+	if chars[index] == '[' {
+		return isImageOrLinkStartBracket(chars, index)
+	}
+
+	return -1
+}
+
+func isImageOrLinkStartExclamation(chars []byte, index int) int {
+	nextIndex := index + 1
+	if nextIndex < len(chars) && chars[nextIndex] == '[' {
+		// It could be the start of an image
+		return 1
+	}
+
+	return -1
+}
+
+func isImageOrLinkStartBracket(chars []byte, index int) int {
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == '\n' {
+			return -1
+		}
+
+		if chars[i] == ']' {
+			return 1
+		}
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_italic_bold.go

@@ -0,0 +1,21 @@
+package escape
+
+import (
+	"unicode"
+)
+
+func IsItalicOrBold(chars []byte, index int) int {
+	if chars[index] != '*' && chars[index] != '_' {
+		return -1
+	}
+
+	next := getNextAsRune(chars, index)
+
+	nextIsWhitespace := unicode.IsSpace(next) || next == 0
+	if nextIsWhitespace {
+		// "not followed by Unicode whitespace"
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_list.go

@@ -0,0 +1,67 @@
+package escape
+
+import (
+	"unicode"
+)
+
+func IsUnorderedList(chars []byte, index int) int {
+	if chars[index] != '-' && chars[index] != '*' && chars[index] != '+' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return -1
+	}
+
+	next := getNext(chars, index)
+	if IsSpace(next) || next == 0 {
+		return 1
+	}
+
+	return -1
+}
+
+func IsOrderedList(chars []byte, index int) int {
+	if chars[index] != '.' && chars[index] != ')' {
+		return -1
+	}
+
+	// Directly before the dot needs to be a digit
+	prev := getPrevAsRune(chars, index)
+	if !unicode.IsDigit(prev) {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if IsDigit(chars[i]) {
+			continue
+		}
+
+		return -1
+	}
+
+	next := getNext(chars, index)
+	if IsSpace(next) || next == 0 {
+		return 1
+	}
+
+	return -1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/elem_quote.go

@@ -0,0 +1,23 @@
+package escape
+
+func IsBlockQuote(chars []byte, index int) int {
+	if chars[index] != '>' {
+		return -1
+	}
+
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == '\n' {
+			break
+		}
+		if chars[i] == placeholderByte {
+			continue
+		}
+		if chars[i] == ' ' {
+			continue
+		}
+
+		return -1
+	}
+
+	return 1
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/replacer.go

@@ -0,0 +1,8 @@
+package escape
+
+import "github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+
+var placeholderRune rune = marker.MarkerEscaping
+
+// IMPORTANT: Only internally we assume it is only byte
+var placeholderByte byte = marker.BytesMarkerEscaping[0]

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape/util.go

@@ -0,0 +1,70 @@
+package escape
+
+import "unicode/utf8"
+
+// TODO: move to markers package?
+
+func IsSpace(b byte) bool {
+	switch b {
+	case '\t', '\n', '\v', '\f', '\r', ' ', 0x85, 0xA0:
+		return true
+	}
+	return false
+}
+
+func IsDigit(b byte) bool {
+	switch b {
+	case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
+		return true
+	}
+	return false
+}
+
+func getPrev(chars []byte, index int) byte {
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return chars[i]
+	}
+	return 0
+}
+
+func getNext(chars []byte, index int) byte {
+	for i := index + 1; i < len(chars); i++ {
+		if chars[i] == placeholderByte {
+			continue
+		}
+		return chars[i]
+	}
+	return 0
+}
+
+func getPrevAsRune(chars []byte, index int) rune {
+	for i := index - 1; i >= 0; i-- {
+		if chars[i] == placeholderByte {
+			continue
+		}
+
+		r, _ := utf8.DecodeLastRune(chars[:i+1])
+
+		return r
+	}
+	return 0
+}
+func getNextAsRune(source []byte, index int) rune {
+	for i := index + 1; i < len(source); i++ {
+		if source[i] == placeholderByte {
+			continue
+		}
+
+		r, _ := utf8.DecodeRune(source[i:])
+		return r
+	}
+	return 0
+}
+
+// TODO: make public?
+func GetNextAsRune(source []byte, index int) rune {
+	return getNextAsRune(source, index)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/codefence.go

@@ -0,0 +1,61 @@
+package textutils
+
+import "strings"
+
+func CalculateCodeFenceOccurrences(fenceChar rune, content string) int {
+	var occurrences []int
+
+	var charsTogether int
+	for _, char := range content {
+		// We encountered a fence character, now count how many
+		// are directly afterwards
+		if char == fenceChar {
+			charsTogether++
+		} else if charsTogether != 0 {
+			occurrences = append(occurrences, charsTogether)
+			charsTogether = 0
+		}
+	}
+
+	// If the last element in the content was a fenceChar
+	if charsTogether != 0 {
+		occurrences = append(occurrences, charsTogether)
+	}
+
+	return findMax(occurrences)
+}
+
+// CalculateCodeFence can be passed the content of a code block and it returns
+// how many fence characters (` or ~) should be used.
+//
+// This is useful if the html content includes the same fence characters
+// for example ```
+// -> https://stackoverflow.com/a/49268657
+func CalculateCodeFence(fenceChar rune, content string) string {
+	repeat := CalculateCodeFenceOccurrences(fenceChar, content)
+
+	// The outer fence block always has to have
+	// at least one character more than any content inside
+	repeat++
+
+	// You have to have at least three fence characters
+	// to be recognized as a code block
+	if repeat < 3 {
+		repeat = 3
+	}
+
+	return strings.Repeat(string(fenceChar), repeat)
+}
+
+func findMax(a []int) (max int) {
+	for i, value := range a {
+		if i == 0 {
+			max = a[i]
+		}
+
+		if value > max {
+			max = value
+		}
+	}
+	return max
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/collapse_code.go

@@ -0,0 +1,31 @@
+package textutils
+
+import (
+	"bytes"
+)
+
+func CollapseInlineCodeContent(content []byte) []byte {
+	// TODO: what about other characters like the reset char? Maybe unicode.IsSpace?
+	content = bytes.ReplaceAll(content, []byte{'\n'}, []byte{' '})
+	content = bytes.ReplaceAll(content, []byte{'\t'}, []byte{' '})
+
+	content = bytes.TrimSpace(content)
+
+	newChars := make([]byte, 0, len(content))
+
+	var count int
+	for _, char := range content {
+		if char == ' ' {
+			count++
+		} else {
+			count = 0
+		}
+
+		if count > 1 {
+			continue
+		}
+		newChars = append(newChars, char)
+	}
+
+	return newChars
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/consecutive_newlines.go

@@ -0,0 +1,139 @@
+package textutils
+
+import (
+	"bytes"
+	"unicode/utf8"
+)
+
+func TrimUnnecessaryHardLineBreaks(content []byte) []byte {
+	content = bytes.ReplaceAll(content, []byte("  \n\n"), []byte("\n\n"))
+	content = bytes.ReplaceAll(content, []byte("  \n  \n"), []byte("\n\n"))
+	content = bytes.ReplaceAll(content, []byte("  \n \n"), []byte("\n\n"))
+	// out = bytes.ReplaceAll(out, []byte("\n  \n"), []byte("\n\n"))
+
+	return content
+}
+
+func TrimConsecutiveNewlines(input []byte) []byte {
+	var result []byte
+	newlineCount := 0
+	spaceBuffer := []byte{}
+
+	for i := 0; i < len(input); {
+		r, size := utf8.DecodeRune(input[i:])
+
+		if r == '\n' {
+			newlineCount++
+			if newlineCount <= 2 {
+				// Preserve up to 2 newlines, including preceding spaces
+				result = append(result, spaceBuffer...)
+				result = append(result, '\n')
+				spaceBuffer = spaceBuffer[:0] // Clear space buffer
+			} else {
+				// Skip additional newlines
+				spaceBuffer = spaceBuffer[:0] // Clear space buffer
+			}
+		} else if r == ' ' {
+			// Collect spaces into the space buffer
+			spaceBuffer = append(spaceBuffer, input[i:i+size]...)
+		} else {
+			// Reset newline count and append non-newline characters
+			newlineCount = 0
+			result = append(result, spaceBuffer...)
+			result = append(result, input[i:i+size]...)
+			spaceBuffer = spaceBuffer[:0] // Clear space buffer
+		}
+
+		i += size
+	}
+
+	// Append any trailing spaces
+	result = append(result, spaceBuffer...)
+
+	return result
+}
+
+/*
+func TrimConsecutiveNewlines(source []byte) []byte {
+	// Some performance optimizations:
+	// - If no replacement was done, we return the original slice and don't allocate.
+	// - We batch appends
+
+	var ret []byte
+
+	startNormal := 0
+	startMatch := -1
+
+	count := 0
+	// for i, b := range source {
+	for i := 0; i < len(source); i++ {
+		r, size := utf8.DecodeRune(source[i:])
+		_ = size
+
+		isNewline := r == '\n' // || r == marker.MarkerLineBreak
+		if isNewline {
+			count += 1
+		}
+
+		if startMatch == -1 && isNewline {
+			// Start of newlines
+			startMatch = i
+			i = i + size - 1
+			continue
+		} else if startMatch != -1 && isNewline {
+			// Middle of newlines
+			i = i + size - 1
+			continue
+		} else if startMatch != -1 {
+			// Character after the last newline character
+
+			if count > 2 {
+				if ret == nil {
+					ret = make([]byte, 0, len(source))
+				}
+
+				ret = append(ret, source[startNormal:startMatch]...)
+				ret = append(ret, '\n', '\n')
+				startNormal = i
+			}
+
+			startMatch = -1
+			count = 0
+		}
+	}
+
+	getStartEnd := func() (int, int, bool, bool) {
+		if startMatch == -1 && startNormal == 0 {
+			// a) no changes need to be done
+			return -1, -1, false, false
+		}
+
+		if count <= 2 {
+			// b) Only the normal characters still need to be added
+			return startNormal, len(source), true, false
+		}
+
+		// c) The match still needs to be replaced (and possible the previous normal characters be added)
+		return startNormal, startMatch, true, true
+	}
+
+	start, end, isKeepNeeded, isReplaceNeeded := getStartEnd()
+	if isKeepNeeded {
+		if ret == nil {
+			ret = make([]byte, 0, len(source))
+		}
+
+		ret = append(ret, source[start:end]...)
+		if isReplaceNeeded {
+			ret = append(ret, '\n', '\n')
+		}
+	}
+
+	if ret == nil {
+		// Hurray, we did not do any allocations with make()
+		// and instead just return the original slice.
+		return source
+	}
+	return ret
+}
+*/

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/delimiter.go

@@ -0,0 +1,37 @@
+package textutils
+
+import (
+	"bytes"
+)
+
+// DelimiterForEveryLine puts the delimiter not just at the start and end of the string
+// but if the text is divided on multiple lines, puts the delimiters on every line with content.
+//
+// Otherwise the bold/italic delimiters won't be recognized if it contains new line characters.
+func DelimiterForEveryLine(text []byte, delimiter []byte) []byte {
+	var buf bytes.Buffer
+
+	lines := bytes.Split(text, []byte("\n"))
+	for i, line := range lines {
+		leftExtra, trimmed, rightExtra := SurroundingSpaces(line)
+
+		if trimmed == nil {
+			// For empty lines, we don't need a delimiter
+			buf.Write(leftExtra)
+			buf.Write(rightExtra)
+		} else {
+			buf.Write(leftExtra)
+			buf.Write(delimiter)
+			buf.Write(trimmed)
+			buf.Write(delimiter)
+			buf.Write(rightExtra)
+		}
+
+		// To join the lines again, add a newlines character
+		if i < len(lines)-1 {
+			buf.WriteRune('\n')
+		}
+	}
+
+	return buf.Bytes()
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/escape_multiline.go

@@ -0,0 +1,56 @@
+package textutils
+
+import (
+	"bytes"
+	"unicode"
+)
+
+var (
+	doubleSpace = []byte{' ', ' '}
+
+	newlineBreak              = []byte{'\n'}
+	hardLineBreak             = []byte{' ', ' ', '\n'}
+	escapedNoContentLineBreak = []byte{'\\', '\n'}
+)
+
+// EscapeMultiLine deals with multiline content inside a link or a heading.
+func EscapeMultiLine(content []byte) []byte {
+	parts := bytes.Split(content, newlineBreak)
+	if len(parts) == 1 {
+		return content
+	}
+
+	output := make([]byte, 0, len(content))
+	for i := range parts {
+		trimmedLeft := bytes.TrimLeftFunc(parts[i], unicode.IsSpace)
+
+		if len(trimmedLeft) == 0 {
+			// A blank line would interrupt the link.
+			// So we need to escape the line
+			output = append(output, escapedNoContentLineBreak...)
+			continue
+		}
+
+		isLast := i == len(parts)-1
+		if isLast {
+			// For the last line we don't need to add any "\n" anymore
+			output = append(output, trimmedLeft...)
+			continue
+		}
+
+		// Now decide what ending we want:
+		if bytes.HasSuffix(trimmedLeft, doubleSpace) {
+			// We already have "  " so adding a "\n" is enough
+			output = append(output, trimmedLeft...)
+			output = append(output, newlineBreak...)
+			continue
+		} else {
+			// We *prefer* having a hard-line-break "  \n"
+			output = append(output, trimmedLeft...)
+			output = append(output, hardLineBreak...)
+			continue
+		}
+	}
+
+	return output
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/prefix_lines.go

@@ -0,0 +1,16 @@
+package textutils
+
+func PrefixLines(source []byte, repl []byte) []byte {
+	newSlice := make([]byte, 0, len(source))
+
+	newSlice = append(newSlice, repl...)
+	for _, b := range source {
+		newSlice = append(newSlice, b)
+		if b == '\n' {
+			newSlice = append(newSlice, repl...)
+		}
+
+	}
+
+	return newSlice
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/quote.go

@@ -0,0 +1,39 @@
+package textutils
+
+import "bytes"
+
+const (
+	DOUBLE_QUOTE = '"'
+	SINGLE_QUOTE = '\''
+)
+
+func SurroundBy(content []byte, chars []byte) []byte {
+	content = append(chars, content...)
+	content = append(content, chars...)
+	return content
+}
+func SurroundByQuotes(content []byte) []byte {
+	if len(content) == 0 {
+		return nil
+	}
+
+	containsDoubleQuote := bytes.ContainsRune(content, DOUBLE_QUOTE)
+	containsSingleQuote := bytes.ContainsRune(content, SINGLE_QUOTE)
+
+	if containsDoubleQuote && containsSingleQuote {
+		// Escape all quotes
+		content = bytes.ReplaceAll(content, []byte(`"`), []byte(`\"`))
+
+		// Surround the content by double quotes
+		return SurroundBy(content, []byte(`"`))
+	}
+	if containsDoubleQuote {
+		// Since it contains double quotes (but no single quotes)
+		// we can surround it by single quotes
+		return SurroundBy(content, []byte(`'`))
+	}
+
+	// It may contain single quotes, but definitely no double quotes,
+	// so we can safely surround it by double quotes.
+	return SurroundBy(content, []byte(`"`))
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils/surrounding_spaces.go

@@ -0,0 +1,58 @@
+/*
+
+The logic to handle whitespace around delimiters was initially developed
+in the fork from "anyproto" by Roman Khafizianov and Mikhail.
+
+The changes were then merged upstream by Johannes Kaufmann.
+
+https://github.com/anyproto/html-to-markdown
+https://github.com/JohannesKaufmann/html-to-markdown
+
+-----------
+
+MIT License
+
+Copyright (c) 2018 Johannes Kaufmann
+Copyright (c) 2020 Roman Khafizianov
+Copyright (c) 2023 Mikhail
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+*/
+
+package textutils
+
+import (
+	"bytes"
+	"unicode"
+)
+
+func SurroundingSpaces(content []byte) ([]byte, []byte, []byte) {
+	rightTrimmed := bytes.TrimRightFunc(content, func(r rune) bool {
+		return unicode.IsSpace(r)
+	})
+	rightExtra := content[len(rightTrimmed):]
+
+	trimmed := bytes.TrimLeftFunc(rightTrimmed, func(r rune) bool {
+		return unicode.IsSpace(r)
+	})
+	leftExtra := content[0 : len(rightTrimmed)-len(trimmed)]
+
+	return leftExtra, trimmed, rightExtra
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/marker/marker.go

@@ -0,0 +1,34 @@
+package marker
+
+import (
+	"bytes"
+)
+
+const (
+	// For simplicity we are using a rune that is one byte wide. A character
+	// that is not used widely (apart from cli's) is the bell character (7).
+	MarkerEscaping rune = '\a'
+
+	// - - - - //
+
+	// Marker0                rune = '\uF000' // 61440
+	// Marker1                rune = '\uF001' // 61441
+	MarkerCodeBlockNewline rune = '\uF002' // 61442
+)
+
+var (
+	BytesMarkerEscaping = []byte{7}
+
+	BytesMarkerCodeBlockNewline = []byte{239, 128, 130}
+)
+
+func init() {
+	checkRuneAndByteSlice(MarkerEscaping, BytesMarkerEscaping)
+	checkRuneAndByteSlice(MarkerCodeBlockNewline, BytesMarkerCodeBlockNewline)
+}
+
+func checkRuneAndByteSlice(r rune, b []byte) {
+	if !bytes.Equal([]byte(string(r)), b) {
+		panic("the rune and byte slice do not represent the same character")
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base/base.go

@@ -0,0 +1,138 @@
+package base
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/collapse"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+
+	"golang.org/x/net/html"
+)
+
+type base struct{}
+
+// NewBasePlugin registers a bunch of stuff that is not necessarily related to commonmark,
+// like removing nodes, trimming whitespace, collapsing whitespace, ...
+func NewBasePlugin() converter.Plugin {
+	base := base{}
+	return &base
+}
+
+func (s *base) Name() string {
+	return "base"
+}
+func (b *base) Init(conv *converter.Converter) error {
+	conv.Register.TagType("#comment", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("head", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("script", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("style", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("link", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("meta", converter.TagTypeRemove, converter.PriorityStandard)
+
+	conv.Register.TagType("iframe", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("noscript", converter.TagTypeRemove, converter.PriorityStandard)
+
+	conv.Register.TagType("input", converter.TagTypeRemove, converter.PriorityStandard)
+	conv.Register.TagType("textarea", converter.TagTypeRemove, converter.PriorityStandard)
+
+	// "tr" is not in the `IsBlockNode` list,
+	// but we want to treat is as a block anyway.
+	// conv.Register.TagStrategy("tr", converter.StrategyMarkdownBlock, converter.PriorityStandard)
+	// conv.Register.TagType("tr", converter.BlockTagType, converter.PriorityStandard)
+
+	conv.Register.PreRenderer(b.preRenderRemove, converter.PriorityEarly)
+	// Note: The priority is low, so that collapse runs _after_ all the other functions
+	conv.Register.PreRenderer(b.preRenderCollapse, converter.PriorityLate)
+
+	conv.Register.TextTransformer(b.handleTextTransform, converter.PriorityStandard)
+
+	conv.Register.PostRenderer(b.postRenderTrimContent, converter.PriorityStandard)
+	conv.Register.PostRenderer(b.postRenderUnescapeContent, converter.PriorityStandard+20)
+
+	return nil
+}
+
+func (b *base) preRenderRemove(ctx converter.Context, doc *html.Node) {
+	var finder func(node *html.Node)
+	finder = func(node *html.Node) {
+		name := dom.NodeName(node)
+
+		if tagType, _ := ctx.GetTagType(name); tagType == converter.TagTypeRemove {
+			dom.RemoveNode(node)
+			return
+		}
+
+		for child := node.FirstChild; child != nil; child = child.NextSibling {
+			// Because we are sometimes removing a node, this causes problems
+			// with the for loop. Using `defer` is a cool trick!
+			// https://gist.github.com/loopthrough/17da0f416054401fec355d338727c46e
+			defer finder(child)
+		}
+	}
+	finder(doc)
+
+	// - - - - - - - //
+
+	// After removing elements (see above) it can happen that we have
+	// two #text nodes right next to each other. This would cause problems
+	// with the collapse so we merge them together.
+	domutils.MergeAdjacentTextNodes(doc)
+}
+
+func (b *base) preRenderCollapse(ctx converter.Context, doc *html.Node) {
+	collapse.Collapse(doc, &collapse.DomFuncs{
+		IsBlockNode: func(node *html.Node) bool {
+			tagName := dom.NodeName(node)
+			tagType, ok := ctx.GetTagType(tagName)
+			if ok {
+				return tagType == converter.TagTypeBlock
+			}
+
+			return dom.NameIsBlockNode(tagName)
+		},
+	})
+}
+
+var characterEntityReplacer = strings.NewReplacer(
+	// We are not using `html.EscapeString` because we
+	// care about fewer characters
+	"<", "&lt;",
+	">", "&gt;",
+
+	// Note: We are not escaping "&" as "&amp;" anymore.
+	// In most cases the "&" is completely fine.
+	// https://github.com/JohannesKaufmann/html-to-markdown/issues/178
+)
+
+func (b *base) handleTextTransform(ctx converter.Context, content string) string {
+
+	// TODO: similar to UnEscapers also only escape if nessesary.
+	//       "<" only if not followed by space
+	//       "&" only if character entity
+	content = characterEntityReplacer.Replace(content)
+
+	// TODO: reduce conversion between types
+	content = string(ctx.EscapeContent([]byte(content)))
+
+	return content
+}
+
+func (b *base) postRenderTrimContent(ctx converter.Context, result []byte) []byte {
+	// Remove whitespace from the beginning & end
+	result = bytes.TrimSpace(result)
+
+	// Remove too many newlines
+	result = textutils.TrimConsecutiveNewlines(result)
+	result = textutils.TrimUnnecessaryHardLineBreaks(result)
+
+	return result
+}
+func (b *base) postRenderUnescapeContent(ctx converter.Context, result []byte) []byte {
+	result = ctx.UnEscapeContent(result)
+	return result
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base/renderers.go

@@ -0,0 +1,62 @@
+package base
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+// RenderAsHTML will render the node as HTML using `html.Render()`
+// Newlines will be inserted depending on the configured `TagType`.
+//
+// As an example, you could do such a combination:
+//
+//	"A text with <strong>bold</strong> and *italic* text"`
+func RenderAsHTML(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+	_ = html.Render(w, node) // TODO: what to do with error?
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}
+
+// RenderAsHTMLWrapper will render the node as HTML
+// and render the children as markdown.
+func RenderAsHTMLWrapper(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	name := dom.NodeName(node)
+
+	w.WriteString("<")
+	w.WriteString(name)
+	// TODO: also render the attributes?
+	w.WriteString(">\n\n")
+
+	ctx.RenderChildNodes(ctx, w, node)
+
+	w.WriteString("\n\n</")
+	w.WriteString(name)
+	w.WriteString(">")
+	return converter.RenderSuccess
+}
+
+// RenderAsPlaintextWrapper will keep the children of this node as markdown.
+func RenderAsPlaintextWrapper(ctx converter.Context, w converter.Writer, node *html.Node) converter.RenderStatus {
+	tagName := dom.NodeName(node)
+	tagType, _ := ctx.GetTagType(tagName)
+
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+	ctx.RenderChildNodes(ctx, w, node)
+	if tagType == converter.TagTypeBlock {
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/commonmark.go

@@ -0,0 +1,206 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/escape"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+type commonmark struct {
+	config
+}
+
+type OptionFunc = func(config *config)
+
+// _ or *
+//
+// default: *
+func WithEmDelimiter(delimiter string) OptionFunc {
+	return func(config *config) {
+		config.EmDelimiter = delimiter
+	}
+}
+
+// ** or __
+//
+// default: **
+func WithStrongDelimiter(delimiter string) OptionFunc {
+	return func(config *config) {
+		config.StrongDelimiter = delimiter
+	}
+}
+
+// Any Thematic break
+//
+// default: "* * *"
+func WithHorizontalRule(rule string) OptionFunc {
+	return func(config *config) {
+		config.HorizontalRule = rule
+	}
+}
+
+// "-", "+", or "*"
+//
+// default: "-"
+func WithBulletListMarker(marker string) OptionFunc {
+	return func(config *config) {
+		config.BulletListMarker = marker
+	}
+}
+func WithListEndComment(enabled bool) OptionFunc {
+	return func(config *config) {
+		config.DisableListEndComment = !enabled
+	}
+}
+
+// ``` or ~~~
+//
+// default: ```
+func WithCodeBlockFence(fence string) OptionFunc {
+	return func(config *config) {
+		config.CodeBlockFence = fence
+	}
+}
+
+// "setext" or "atx"
+//
+// default: "atx"
+func WithHeadingStyle(style headingStyle) OptionFunc {
+	return func(config *config) {
+		config.HeadingStyle = style
+	}
+}
+
+// WithLinkEmptyHrefBehavior configures how links with *empty hrefs* are rendered.
+// Take for example:
+//
+//	<a href="">the link content</a>
+//
+// LinkBehaviorRenderAsLink would result in "[the link content]()"
+//
+// LinkBehaviorSkipLink would result in "the link content"
+func WithLinkEmptyHrefBehavior(behavior linkRenderingBehavior) OptionFunc {
+	return func(config *config) {
+		config.LinkEmptyHrefBehavior = behavior
+	}
+}
+
+// WithLinkEmptyContentBehavior configures how links *without content* are rendered.
+// Take for example:
+//
+//	<a href="/page"></a>
+//
+// LinkBehaviorRenderAsLink would result in "[](/page)"
+//
+// LinkBehaviorSkipLink would result in an empty string.
+func WithLinkEmptyContentBehavior(behavior linkRenderingBehavior) OptionFunc {
+	return func(config *config) {
+		config.LinkEmptyContentBehavior = behavior
+	}
+}
+
+// TODO: allow changing the link style once the render logic is implemented
+//
+// "inlined" or "referenced_index" or "referenced_short"
+//
+// default: inlined
+// func WithLinkStyle(style linkStyle) OptionFunc {
+// 	return func(config *config) {
+// 		config.LinkStyle = style
+// 	}
+// }
+
+// NewCommonmarkPlugin registers the markdown syntax of commonmark.
+func NewCommonmarkPlugin(opts ...OptionFunc) converter.Plugin {
+
+	cfg := &config{}
+	for _, opt := range opts {
+		opt(cfg)
+	}
+
+	cm := commonmark{
+		config: fillInDefaultConfig(cfg),
+	}
+
+	return &cm
+}
+
+func (s *commonmark) Name() string {
+	return "commonmark"
+}
+func (cm *commonmark) Init(conv *converter.Converter) error {
+	if err := validateConfig(&cm.config); err != nil {
+		return err
+	}
+
+	// - - - - - - - - //
+
+	conv.Register.PreRenderer(cm.handlePreRender, converter.PriorityStandard)
+
+	// Note: Should run after "collapse" & also after "remove"
+	conv.Register.PreRenderer(func(ctx converter.Context, doc *html.Node) {
+		if cm.DisableListEndComment {
+			// Early return if the feature is unwanted
+			return
+		}
+
+		domutils.AddListEndComments(ctx, doc)
+	}, converter.PriorityLate+100)
+
+	conv.Register.EscapedChar(
+		'\\',
+		'*', '_', '-', '+',
+		'.', '>', '|',
+		'$',
+		'#', '=',
+		'[', ']', '(', ')',
+		'!',
+		'~', '`', '"', '\'',
+	)
+	conv.Register.UnEscaper(escape.IsItalicOrBold, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsBlockQuote, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsAtxHeader, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsSetextHeader, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsDivider, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsOrderedList, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsUnorderedList, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsImageOrLink, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsFencedCode, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsInlineCode, converter.PriorityStandard)
+	conv.Register.UnEscaper(escape.IsBackslash, converter.PriorityStandard)
+
+	conv.Register.Renderer(cm.handleRender, converter.PriorityStandard)
+
+	conv.Register.TextTransformer(cm.handleTextTransform, converter.PriorityLate)
+
+	conv.Register.PostRenderer(cm.handlePostRenderCodeBlockNewline, converter.PriorityLate)
+
+	return nil
+}
+
+func (cm commonmark) handlePostRenderCodeBlockNewline(ctx converter.Context, content []byte) []byte {
+	return bytes.ReplaceAll(
+		content,
+		[]byte(string(marker.BytesMarkerCodeBlockNewline)),
+		[]byte("\n"),
+	)
+}
+
+func (cm commonmark) handleTextTransform(ctx converter.Context, content string) string {
+
+	if isEnabled, ok := ctx.Value("is_inside_link").(bool); ok && isEnabled {
+		content = strings.Replace(content, string(marker.MarkerEscaping)+`]`, `\]`, -1)
+	}
+	// if isEnabled, ok := ctx.Value("is_inside_heading").(bool); ok && isEnabled {
+	// 	// The "#" character would be completely removed, if at the _end_
+	// 	// of the heading content. So always escape it inside headings.
+	// 	content = strings.Replace(content, string(marker.MarkerEscaping)+`#`, `\#`, -1)
+	// }
+
+	return content
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/handle_pre_render.go

@@ -0,0 +1,94 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"golang.org/x/net/html"
+)
+
+func nameIsBold(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "strong" || name == "b"
+}
+func nameIsItalic(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "em" || name == "i"
+}
+
+func nameIsBoldOrItalic(node *html.Node) bool {
+	return nameIsBold(node) || nameIsItalic(node)
+}
+func nameIsBothBoldOrItalic(a, b *html.Node) bool {
+	if nameIsBold(a) && nameIsBold(b) {
+		return true
+	}
+	if nameIsItalic(a) && nameIsItalic(b) {
+		return true
+	}
+
+	return false
+}
+
+func nameIsPre(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "pre"
+}
+func nameIsInlineCode(node *html.Node) bool {
+	name := dom.NodeName(node)
+	return name == "code" || name == "var" || name == "samp" || name == "kbd" || name == "tt"
+}
+
+func nameIsLink(node *html.Node) bool {
+	return dom.NodeName(node) == "a"
+}
+
+func nameIsBothLink(a, b *html.Node) bool {
+	return dom.NodeName(a) == "a" && dom.NodeName(b) == "a"
+}
+
+func nameIsHeading(node *html.Node) bool {
+	name := dom.NodeName(node)
+
+	if name == "h1" || name == "h2" || name == "h3" || name == "h4" || name == "h5" || name == "h6" {
+		return true
+	}
+	return false
+}
+
+// func nameIsBlockquote(node *html.Node) bool {
+// 	return dom.NodeName(node) == "blockquote"
+// }
+
+func (c *commonmark) handlePreRender(ctx converter.Context, doc *html.Node) {
+	domutils.RenameFakeSpans(ctx, doc)
+
+	// domutils.SplitUp(ctx, doc, nameIsBoldOrItalic, nameIsLink, atom.Span)
+
+	// domutils.SplitUp(ctx, doc, nameIsLink, nameIsHeading, atom.Div)
+	// domutils.SplitUp(ctx, doc, nameIsLink, nameIsBlockquote, atom.Div)
+
+	// - - - Bold / Italic - - - //
+	domutils.RemoveRedundant(doc, nameIsBothBoldOrItalic)
+	domutils.MergeAdjacent(doc, nameIsBoldOrItalic)
+
+	// domutils.MovePunctuation(ctx, doc, nameIsBoldOrItalic)
+
+	// - - - Code - - - //
+	domutils.RemoveEmptyCode(ctx, doc)
+	domutils.SwapTags(ctx, doc, nameIsInlineCode, nameIsPre)
+	domutils.MergeAdjacent(doc, nameIsInlineCode)
+
+	domutils.AddSpace(ctx, doc, nameIsBoldOrItalic, nameIsInlineCode)
+
+	// - - - Link - - - //
+	domutils.RemoveRedundant(doc, nameIsBothLink)
+	domutils.SwapTags(ctx, doc, nameIsBoldOrItalic, nameIsLink)
+
+	// - - - Heading - - - //
+	domutils.SwapTags(ctx, doc, nameIsLink, nameIsHeading)
+	domutils.LeafBlockAlternatives(ctx, doc)
+
+	// - - - List - - - //
+	domutils.MoveListItems(ctx, doc)
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/handle_render.go

@@ -0,0 +1,47 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) handleRender(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	name := dom.NodeName(n)
+
+	switch name {
+	case "strong", "b",
+		"em", "i":
+		return c.renderBoldItalic(ctx, w, n)
+	case "hr":
+		return c.renderDivider(ctx, w, n)
+	case "br":
+		return c.renderBreak(ctx, w, n)
+	case "ul", "ol":
+		return c.renderListContainer(ctx, w, n)
+
+	case "pre":
+		return c.renderBlockCode(ctx, w, n)
+	case "code",
+		"var", "samp", "kbd", "tt":
+		return c.renderInlineCode(ctx, w, n)
+
+	case "blockquote":
+		return c.renderBlockquote(ctx, w, n)
+
+	case "h1", "h2", "h3", "h4", "h5", "h6":
+		return c.renderHeading(ctx, w, n)
+
+	case "img":
+		return c.renderImage(ctx, w, n)
+
+	case "a":
+		return c.renderLink(ctx, w, n)
+
+	case "#comment":
+		return c.renderComment(ctx, w, n)
+	}
+
+	return converter.RenderTryNext
+
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/options.go

@@ -0,0 +1,130 @@
+package commonmark
+
+type linkStyle string
+
+const (
+	// For example:
+	//
+	//  [view more](/about.html)
+	LinkStyleInlined linkStyle = "inlined"
+
+	LinkStyleReferencedIndex linkStyle = "referenced_index"
+	LinkStyleReferencedShort linkStyle = "referenced_short"
+)
+
+type headingStyle string
+
+const (
+	// HeadingStyleATX is the heading style of prefixing the heading with "#" signs indicating the level. For example:
+	//
+	//  ## Heading
+	HeadingStyleATX headingStyle = "atx"
+
+	// HeadingStyleSetext is the heading style of putting "=" or "-" on the followed line. For example:
+	//
+	//  Heading
+	//  -------
+	HeadingStyleSetext headingStyle = "setext"
+)
+
+type linkRenderingBehavior string
+
+const (
+	// LinkBehaviorRender renders the element as a link
+	LinkBehaviorRender linkRenderingBehavior = "render"
+	// LinkBehaviorSkip skips link rendering and falls back to the other rules (e.g. paragraph)
+	LinkBehaviorSkip linkRenderingBehavior = "skip"
+)
+
+// config to customize the output. You can change stuff like
+// the character that is used for strong text.
+type config struct {
+	// _ or *
+	//
+	// default: *
+	EmDelimiter string
+
+	// ** or __
+	//
+	// default: **
+	StrongDelimiter string
+
+	// Any Thematic break
+	//
+	// default: "* * *"
+	HorizontalRule string
+
+	// "-", "+", or "*"
+	//
+	// default: "-"
+	BulletListMarker string
+
+	DisableListEndComment bool
+
+	// "indented" or "fenced"
+	//
+	// default: "indented"
+	// TODO: CodeBlockStyle string
+
+	// ``` or ~~~
+	//
+	// default: ```
+	CodeBlockFence string
+
+	// "setext" or "atx"
+	//
+	// default: "atx"
+	HeadingStyle headingStyle
+
+	// TODO: LineBreakStyle string "hard" or "soft"
+
+	// "inlined" or "referenced_index" or "referenced_short"
+	//
+	// default: inlined
+	LinkStyle linkStyle
+
+	LinkEmptyHrefBehavior    linkRenderingBehavior
+	LinkEmptyContentBehavior linkRenderingBehavior
+}
+
+func fillInDefaultConfig(cfg *config) config {
+	if cfg.EmDelimiter == "" {
+		// The new default is now "*" (instead of "_") as that works better inside words.
+		cfg.EmDelimiter = "*"
+	}
+	if cfg.StrongDelimiter == "" {
+		cfg.StrongDelimiter = "**"
+	}
+
+	if cfg.HorizontalRule == "" {
+		cfg.HorizontalRule = "* * *"
+	}
+
+	if cfg.BulletListMarker == "" {
+		cfg.BulletListMarker = "-"
+	}
+
+	// TODO: also check for spelling mistakes in "indented"
+	// if opt.CodeBlockStyle == "" {
+	// 	opt.CodeBlockStyle = "indented"
+	// }
+	if cfg.CodeBlockFence == "" {
+		cfg.CodeBlockFence = "```"
+	}
+
+	if cfg.HeadingStyle == "" {
+		cfg.HeadingStyle = "atx"
+	}
+
+	if cfg.LinkEmptyHrefBehavior == "" {
+		cfg.LinkEmptyHrefBehavior = LinkBehaviorRender
+	}
+	if cfg.LinkEmptyContentBehavior == "" {
+		cfg.LinkEmptyContentBehavior = LinkBehaviorRender
+	}
+	if cfg.LinkStyle == "" {
+		cfg.LinkStyle = LinkStyleInlined
+	}
+
+	return *cfg
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_blockquote.go

@@ -0,0 +1,32 @@
+package commonmark
+
+import (
+	"bytes"
+
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderBlockquote(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+
+	content := buf.Bytes()
+	content = bytes.TrimSpace(content)
+	if content == nil {
+		return converter.RenderSuccess
+	}
+
+	content = textutils.TrimConsecutiveNewlines(content)
+	content = textutils.TrimUnnecessaryHardLineBreaks(content)
+	content = textutils.PrefixLines(content, []byte{'>', ' '})
+
+	w.WriteRune('\n')
+	w.WriteRune('\n')
+	w.Write(content)
+	w.WriteRune('\n')
+	w.WriteRune('\n')
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_bold_italic.go

@@ -0,0 +1,39 @@
+package commonmark
+
+import (
+	"bytes"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func (c commonmark) getDelimiter(n *html.Node) []byte {
+	name := dom.NodeName(n)
+	if name == "strong" || name == "b" {
+		return []byte(c.StrongDelimiter)
+	} else if name == "em" || name == "i" {
+		return []byte(c.EmDelimiter)
+	} else {
+		return nil
+	}
+}
+func (c commonmark) renderBoldItalic(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+
+	// Depending on the options & whether it is bold or italic there
+	// is going to be a different delimiter.
+	delimiter := c.getDelimiter(n)
+	content := buf.Bytes()
+
+	// If there is a newline character between the start and end delimiter
+	// the delimiters won't be recognized. Either we remove all newline characters
+	// OR on _every_ line we put start & end delimiters.
+	content = textutils.DelimiterForEveryLine(content, delimiter)
+
+	w.Write(content)
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_break.go

@@ -0,0 +1,12 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderBreak(_ converter.Context, w converter.Writer, _ *html.Node) converter.RenderStatus {
+	// Render a "hard line break"
+	w.WriteString("  \n")
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_code.go

@@ -0,0 +1,149 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderInlineCode(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	// TODO: configure delimeter in options?
+	fenceChar := '`'
+
+	codeContent, _ := getCodeWithoutTags(n)
+
+	// TODO: debug flag?
+	if len(codeContent) == 0 {
+		// fmt.Println("expected an empty inline code to be already removed")
+		// panic("expected an empty inline code to be already removed")
+	}
+	// TODO: configurable function to decide if inline or block?
+	if bytes.Contains(codeContent, []byte("\n")) {
+		// fmt.Println("inline code contains newlines")
+		// return c.renderBlockCode(ctx, w, n, render)
+	}
+
+	if bytes.TrimSpace(codeContent) == nil {
+		// No stripping occurs if the code span contains _only_ spaces:
+		w.WriteRune(fenceChar)
+		w.Write(codeContent)
+		w.WriteRune(fenceChar)
+		return converter.RenderSuccess
+	}
+
+	// Newlines in the text aren't great, since this is inline code and not a code block.
+	// Newlines will be stripped anyway in the browser, but it won't be recognized as code
+	// from the markdown parser when there is more than one newline.
+	codeContent = textutils.CollapseInlineCodeContent(codeContent)
+
+	code := string(codeContent)
+
+	maxCount := textutils.CalculateCodeFenceOccurrences(fenceChar, code)
+	maxCount++
+
+	fence := strings.Repeat(string(fenceChar), maxCount)
+
+	// Code contains a backtick as first character
+	if strings.HasPrefix(code, "`") {
+		code = " " + code
+	}
+	// Code contains a backtick as last character
+	if strings.HasSuffix(code, "`") {
+		code = code + " "
+	}
+
+	w.WriteString(fence)
+	w.WriteString(code)
+	w.WriteString(fence)
+
+	return converter.RenderSuccess
+}
+func (c *commonmark) renderBlockCode(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	code, infoString := getCodeWithoutTags(n)
+
+	if bytes.HasSuffix(code, []byte("\n")) {
+		code = code[:len(code)-1]
+	}
+
+	fenceChar, _ := utf8.DecodeRuneInString(c.CodeBlockFence)
+	fence := textutils.CalculateCodeFence(fenceChar, string(code))
+
+	// We want to keep the original content inside the code block untouched.
+	// Because multiple newlines would be trimmed, we temporarily replace it with another character.
+	code = bytes.ReplaceAll(code, []byte("\n"), marker.BytesMarkerCodeBlockNewline)
+
+	w.WriteString("\n\n")
+	w.WriteString(fence)
+	w.WriteString(infoString)
+	w.WriteRune('\n')
+	w.Write(code)
+	w.WriteRune('\n')
+	w.WriteString(fence)
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}
+
+func getCodeLanguage(n *html.Node) string {
+	class := dom.GetAttributeOr(n, "class", "")
+
+	parts := strings.Split(class, " ")
+	for _, part := range parts {
+		if !strings.Contains(part, "language-") && !strings.Contains(part, "lang-") {
+			continue
+		}
+
+		part = strings.Replace(part, "language-", "", 1)
+		part = strings.Replace(part, "lang-", "", 1)
+
+		return part
+	}
+
+	return ""
+}
+func getCodeWithoutTags(startNode *html.Node) ([]byte, string) {
+	var buf bytes.Buffer
+	var infoString string
+
+	var f func(*html.Node)
+	f = func(n *html.Node) {
+		if n.Type == html.ElementNode && (n.Data == "code" || n.Data == "pre") {
+
+			// TODO: what if multiple elements have an info string?
+			if infoString == "" {
+				infoString = getCodeLanguage(n)
+			}
+		}
+
+		// - - - //
+
+		if n.Type == html.ElementNode && (n.Data == "style" || n.Data == "script" || n.Data == "textarea") {
+			return
+		}
+		if n.Type == html.ElementNode && (n.Data == "br" || n.Data == "div") {
+			buf.WriteString("\n")
+		}
+
+		if n.Type == html.TextNode {
+			// if strings.TrimSpace(n.Data) == "" && strings.Contains(n.Data, "\n") {
+			// 	buf.WriteString("\n")
+			// }
+			buf.WriteString(n.Data)
+			return
+		}
+
+		for c := n.FirstChild; c != nil; c = c.NextSibling {
+			f(c)
+		}
+	}
+
+	f(startNode)
+
+	return buf.Bytes(), infoString
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_comment.go

@@ -0,0 +1,25 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/domutils"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderComment(_ converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+
+	if n.Data == domutils.ListEndCommentData {
+		// We definitely want to render the list end comments
+		// that were just added
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+		_ = html.Render(w, n)
+		w.WriteRune('\n')
+		w.WriteRune('\n')
+		return converter.RenderSuccess
+
+	}
+
+	// Fallback to the normal settings for comments
+	return converter.RenderTryNext
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_divider.go

@@ -0,0 +1,15 @@
+package commonmark
+
+import (
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"golang.org/x/net/html"
+)
+
+func (c *commonmark) renderDivider(_ converter.Context, w converter.Writer, _ *html.Node) converter.RenderStatus {
+
+	w.WriteString("\n\n")
+	w.WriteString(c.HorizontalRule)
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_heading.go

@@ -0,0 +1,150 @@
+package commonmark
+
+import (
+	"bytes"
+	"regexp"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+// TODO: remove regex
+var multipleSpacesR = regexp.MustCompile(`  +`)
+
+func (r *commonmark) setextUnderline(level int, width int) []byte {
+	line := "-"
+	if level == 1 {
+		line = "="
+	}
+
+	return bytes.Repeat([]byte(line), width)
+}
+func (r *commonmark) atxPrefix(level int) []byte {
+	return bytes.Repeat([]byte("#"), level)
+}
+
+func getHeadingLevel(name string) int {
+	switch name {
+	case "h1":
+		return 1
+	case "h2":
+		return 2
+	case "h3":
+		return 3
+	case "h4":
+		return 4
+	case "h5":
+		return 5
+	case "h6":
+		return 6
+	default:
+		return 6
+	}
+}
+func runeCount(chars []rune) (count int) {
+	for _, char := range chars {
+		if char == marker.MarkerEscaping {
+			continue
+		}
+		count++
+	}
+	return
+}
+func getUnderlineWidth(content []byte, minVal int) int {
+	var width int
+
+	parts := bytes.Split(content, []byte("\n"))
+	for _, part := range parts {
+		// Count how wide the line should be,
+		// while using RuneCount to correctly count ä, ö, ...
+		//
+		// TODO: optimize function w := utf8.RuneCount(part)
+		w := runeCount([]rune(string(part)))
+		if w > width {
+			width = w
+		}
+	}
+
+	// Technically the minimum value is only one character,
+	// but one dash could easily trigger a heading.
+	if width < minVal {
+		return minVal
+	}
+
+	return width
+}
+
+func escapePoundSignAtEnd(s []byte) []byte {
+	// -1 #
+	// -2 placeholder
+	// -3 maybe \
+
+	if s[len(s)-1] != '#' {
+		// We don't have a # at the end,
+		// so there is no work to do...
+		return s
+	}
+	if len(s) >= 3 && s[len(s)-3] == '\\' {
+		// It is already escaped,
+		// so there is no work to do...
+		return s
+	}
+
+	// Because we have a # at the end,
+	// we should manually force the escaping
+	// by overriding the placeholder.
+	s[len(s)-2] = '\\'
+
+	return s
+}
+
+func (c *commonmark) renderHeading(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	// ctx = context.WithValue(ctx, "is_inside_heading", true)
+
+	level := getHeadingLevel(dom.NodeName(n))
+
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+	content := buf.Bytes()
+
+	if len(bytes.TrimSpace(content)) == 0 {
+		return converter.RenderSuccess
+	}
+
+	if c.HeadingStyle == HeadingStyleSetext && level < 3 {
+		// Note: We don't want to use `TrimUnnecessaryHardLineBreaks` here,
+		// since `EscapeMultiLine` also takes care of newlines.
+		content = textutils.TrimConsecutiveNewlines(content)
+		content = textutils.EscapeMultiLine(content)
+
+		width := getUnderlineWidth(content, 3)
+		underline := c.setextUnderline(level, width)
+
+		w.WriteString("\n\n")
+		w.Write(content)
+		w.WriteRune('\n')
+		w.Write(underline)
+		w.WriteString("\n\n")
+	} else {
+		content = bytes.ReplaceAll(content, []byte("\n"), []byte(" "))
+		content = bytes.ReplaceAll(content, []byte("\r"), []byte(" "))
+		// Replace multiple spaces by one space.
+		content = multipleSpacesR.ReplaceAll(content, []byte(" "))
+
+		content = bytes.TrimSpace(content)
+
+		// A # sign at the end would be removed otherwise
+		content = escapePoundSignAtEnd(content)
+
+		w.WriteString("\n\n")
+		w.Write(c.atxPrefix(level))
+		w.WriteRune(' ')
+		w.Write(content)
+		w.WriteString("\n\n")
+	}
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_image.go

@@ -0,0 +1,63 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+func escapeAlt(altString string) string {
+	alt := []byte(altString)
+
+	var buf bytes.Buffer
+	for i := range alt {
+		if alt[i] == '[' || alt[i] == ']' {
+			prevIndex := i - 1
+			if prevIndex < 0 || alt[prevIndex] != '\\' {
+				buf.WriteRune('\\')
+			}
+		}
+		buf.WriteByte(alt[i])
+	}
+
+	return buf.String()
+}
+
+func (c *commonmark) renderImage(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	src := dom.GetAttributeOr(n, "src", "")
+	src = strings.TrimSpace(src)
+	if src == "" {
+		return converter.RenderTryNext
+	}
+
+	src = ctx.AssembleAbsoluteURL(ctx, "img", src)
+
+	title := dom.GetAttributeOr(n, "title", "")
+	title = strings.ReplaceAll(title, "\n", " ")
+
+	alt := dom.GetAttributeOr(n, "alt", "")
+	alt = strings.ReplaceAll(alt, "\n", " ")
+
+	// The alt description will be placed between two square brackets `[alt]`
+	// so make sure that those characters are escaped.
+	alt = escapeAlt(alt)
+
+	w.WriteRune('!')
+	w.WriteRune('[')
+	w.WriteString(alt)
+	w.WriteRune(']')
+	w.WriteRune('(')
+	w.WriteString(src)
+	if title != "" {
+		// The destination and title must be seperated by a space
+		w.WriteRune(' ')
+		w.Write(textutils.SurroundByQuotes([]byte(title)))
+	}
+	w.WriteRune(')')
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_link.go

@@ -0,0 +1,103 @@
+package commonmark
+
+import (
+	"bytes"
+	"strings"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"golang.org/x/net/html"
+)
+
+// link in commonmark contains
+// - the link text (the visible text)
+// - a link destination (the URI that is the link destination)
+// - an optional link title
+type link struct {
+	*html.Node
+
+	before  []byte
+	content []byte
+	after   []byte
+
+	href  string
+	title string
+}
+
+func (c *commonmark) renderLinkInlined(w converter.Writer, l *link) converter.RenderStatus {
+
+	w.Write(l.before)
+	w.WriteRune('[')
+	w.Write(l.content)
+	w.WriteRune(']')
+	w.WriteRune('(')
+	w.WriteString(l.href)
+	if l.title != "" {
+		// The destination and title must be separated by a space
+		w.WriteRune(' ')
+		w.Write(textutils.SurroundByQuotes([]byte(l.title)))
+	}
+	w.WriteRune(')')
+	w.Write(l.after)
+
+	return converter.RenderSuccess
+}
+
+func (c *commonmark) renderLink(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	ctx = ctx.WithValue("is_inside_link", true)
+
+	href := dom.GetAttributeOr(n, "href", "")
+
+	href = strings.TrimSpace(href)
+	href = ctx.AssembleAbsoluteURL(ctx, "a", href)
+
+	if href == "" && c.config.LinkEmptyHrefBehavior == LinkBehaviorSkip {
+		// There is *no href* for the link. Now we have two options:
+		// Continue rendering as a link OR skip to let other renderers take over.
+		return converter.RenderTryNext
+	}
+
+	title := dom.GetAttributeOr(n, "title", "")
+	title = strings.ReplaceAll(title, "\n", " ")
+
+	l := &link{
+		Node:  n,
+		href:  href,
+		title: title,
+	}
+
+	var buf bytes.Buffer
+	ctx.RenderChildNodes(ctx, &buf, n)
+	content := buf.Bytes()
+
+	if len(bytes.TrimSpace(content)) == 0 && c.config.LinkEmptyContentBehavior == LinkBehaviorSkip {
+		// There is *no content* inside the link. Now we have two options:
+		// Continue rendering as a link OR skip to let other renderers take over.
+		return converter.RenderTryNext
+	}
+
+	if l.href == "" {
+		// A link without href is valid, like e.g. [text]()
+		// But a title would make it invalid.
+		l.title = ""
+	}
+
+	leftExtra, trimmed, rightExtra := textutils.SurroundingSpaces(content)
+
+	// Note: We don't want to use `TrimUnnecessaryHardLineBreaks` here,
+	// since `EscapeMultiLine` also takes care of newlines.
+	trimmed = textutils.TrimConsecutiveNewlines(trimmed)
+	trimmed = textutils.EscapeMultiLine(trimmed)
+
+	l.before = leftExtra
+	l.content = trimmed
+	l.after = rightExtra
+
+	switch c.LinkStyle {
+	case LinkStyleInlined:
+		return c.renderLinkInlined(w, l)
+	default:
+		return converter.RenderTryNext
+	}
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/render_list.go

@@ -0,0 +1,111 @@
+package commonmark
+
+import (
+	"bytes"
+	"fmt"
+	"strconv"
+	"unicode/utf8"
+
+	"github.com/JohannesKaufmann/dom"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/internal/textutils"
+	"github.com/JohannesKaufmann/html-to-markdown/v2/marker"
+	"golang.org/x/net/html"
+)
+
+func getStartAt(node *html.Node) int {
+	startVal := dom.GetAttributeOr(node, "start", "1")
+	startAt, err := strconv.Atoi(startVal)
+	if err != nil {
+		startAt = 1
+	}
+
+	return startAt
+}
+
+func (c commonmark) getPrefixFunc(n *html.Node, sliceLength int) func(int) string {
+	startAt := getStartAt(n)
+
+	return func(sliceIndex int) string {
+		if n.Data == "ul" {
+			return c.BulletListMarker + " "
+		}
+
+		currentIndex := startAt + sliceIndex
+		lastIndex := startAt + sliceLength - 1
+		maxLength := utf8.RuneCountInString(strconv.Itoa(lastIndex))
+
+		// Pad the numbers so that all prefix numbers in the list take up the same space
+		// `%02d.` -> "01. "
+		format := `%0` + strconv.Itoa(maxLength) + `d. `
+		return fmt.Sprintf(format, currentIndex)
+	}
+}
+
+func renderMultiLineListItem(w converter.Writer, content []byte, indentCount int) {
+	lines := bytes.Split(content, []byte("\n"))
+	indent := bytes.Repeat([]byte(" "), indentCount)
+
+	indentedCodeBlockNewline := append(marker.BytesMarkerCodeBlockNewline, indent...)
+
+	for i := range lines {
+		// Add indent to code block newlines
+		line := bytes.ReplaceAll(lines[i], marker.BytesMarkerCodeBlockNewline, indentedCodeBlockNewline)
+
+		if i != 0 {
+			// The first line is already indented through the prefix,
+			// all other lines need the correct amount of spaces.
+			w.Write(indent)
+		}
+		w.Write(line)
+
+		if i < len(lines)-1 {
+			w.WriteRune('\n')
+		}
+	}
+}
+
+func (c commonmark) renderListContainer(ctx converter.Context, w converter.Writer, n *html.Node) converter.RenderStatus {
+	children := dom.AllChildNodes(n)
+	items := make([][]byte, 0, len(children))
+
+	for _, child := range children {
+		var buf bytes.Buffer
+		ctx.RenderNodes(ctx, &buf, child)
+
+		content := buf.Bytes()
+		content = bytes.TrimSpace(content)
+		if content == nil {
+			continue
+		}
+
+		items = append(items, content)
+	}
+
+	if len(items) == 0 {
+		return converter.RenderSuccess
+	}
+
+	getPrefix := c.getPrefixFunc(n, len(items))
+	indentCount := utf8.RuneCountInString(getPrefix(0))
+
+	w.WriteString("\n\n")
+	for i, item := range items {
+		w.WriteString(getPrefix(i))
+
+		item = textutils.TrimConsecutiveNewlines(item)
+		item = textutils.TrimUnnecessaryHardLineBreaks(item)
+		item = ctx.UnEscapeContent(item)
+
+		// An item might have different lines that each
+		// must be indented with the correct count of spaces.
+		renderMultiLineListItem(w, item, indentCount)
+
+		if i < len(items)-1 {
+			w.WriteRune('\n')
+		}
+	}
+	w.WriteString("\n\n")
+
+	return converter.RenderSuccess
+}

vendor/github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark/validation.go

@@ -0,0 +1,101 @@
+package commonmark
+
+import (
+	"fmt"
+	"strings"
+)
+
+func contains(values []string, searchVal string) bool {
+	for _, val := range values {
+		if val == searchVal {
+			return true
+		}
+	}
+	return false
+}
+
+// TODO: should this be with the commonmark package? Or more general?
+// TODO: Maybe make it an interface? And also have a GetPluginName function?
+type ValidateConfigError struct {
+	Key   string
+	Value string
+
+	// By default is "Key:Value" but can be
+	// overriden to e.g. "--key=value"
+	KeyWithValue string
+
+	patternDescription string
+}
+
+func (e *ValidateConfigError) setDefaultKeyWithValue() {
+	e.KeyWithValue = fmt.Sprintf("%s:%q", e.Key, e.Value)
+}
+func (e *ValidateConfigError) Error() string {
+	if e.KeyWithValue == "" {
+		e.setDefaultKeyWithValue()
+	}
+
+	return fmt.Sprintf("invalid value for %s must be %s", e.KeyWithValue, e.patternDescription)
+}
+
+func validateConfig(cfg *config) error {
+	if strings.Count(cfg.EmDelimiter, "_") != 1 && strings.Count(cfg.EmDelimiter, "*") != 1 {
+		return &ValidateConfigError{
+			Key:                "EmDelimiter",
+			Value:              cfg.EmDelimiter,
+			patternDescription: `exactly 1 character of "*" or "_"`,
+		}
+	}
+	if strings.Count(cfg.StrongDelimiter, "_") != 2 && strings.Count(cfg.StrongDelimiter, "*") != 2 {
+		return &ValidateConfigError{
+			Key:                "StrongDelimiter",
+			Value:              cfg.StrongDelimiter,
+			patternDescription: `exactly 2 characters of "**" or "__"`,
+		}
+	}
+
+	if strings.Count(cfg.HorizontalRule, "*") < 3 &&
+		strings.Count(cfg.HorizontalRule, "_") < 3 &&
+		strings.Count(cfg.HorizontalRule, "-") < 3 {
+		return &ValidateConfigError{
+			Key:                "HorizontalRule",
+			Value:              cfg.HorizontalRule,
+			patternDescription: `at least 3 characters of "*", "_" or "-"`,
+		}
+	}
+
+	if !contains([]string{"-", "+", "*"}, cfg.BulletListMarker) {
+		return &ValidateConfigError{
+			Key:                "BulletListMarker",
+			Value:              cfg.BulletListMarker,
+			patternDescription: `one of "-", "+" or "*"`,
+		}
+	}
+
+	if !contains([]string{"```", "~~~"}, cfg.CodeBlockFence) {
+		return &ValidateConfigError{
+			Key:                "CodeBlockFence",
+			Value:              cfg.CodeBlockFence,
+			patternDescription: "one of \"```\" or \"~~~\"",
+		}
+	}
+
+	if !contains([]string{"atx", "setext"}, string(cfg.HeadingStyle)) {
+		return &ValidateConfigError{
+			Key:                "HeadingStyle",
+			Value:              string(cfg.HeadingStyle),
+			patternDescription: `one of "atx" or "setext"`,
+		}
+	}
+
+	possibleLinkStyles := []string{string(LinkStyleInlined), string(LinkStyleReferencedIndex), string(LinkStyleReferencedShort)}
+	if !contains(possibleLinkStyles, string(cfg.LinkStyle)) {
+		return &ValidateConfigError{
+			Key:                "LinkStyle",
+			Value:              string(cfg.LinkStyle),
+			patternDescription: `one of "inlined", "referenced_index" or "referenced_short"`,
+		}
+	}
+
+	return nil
+}