quarto is 100x slower than rmarkdown for long outputs

[idavydov]

I have:

• searched the issue tracker for similar issues
• installed the latest version of Quarto CLI
• formatted my issue following the Bug Reports guide

Bug description

Quarto render is very slow when the cat output is long.

Steps to reproduce

1. Create minimal.qmd file:

---
title: "Minimal reprex"
format: html
---

```{r}
cat(strrep("x", 100000))
```

2. time quarto render minimal.qmd
3. For comparison you can create minimal.Rmd:

---
title: "Minimal reprex"
output: html_document
---

```{r}
cat(strrep("x", 100000))
```

Actual behavior

$ time quarto render minimal.qmd > /dev/null 2>&1
quarto render minimal.qmd > /dev/null 2>&1  35.41s user 0.88s system 99% cpu 36.399 total
$ time Rscript -e 'rmarkdown::render("minimal.Rmd")' > /dev/null 2>&1
Rscript -e 'rmarkdown::render("minimal.Rmd")' > /dev/null 2>&1  0.34s user 0.08s system 86% cpu 0.487 total

Expected behavior

Quarto should run in <1s. The difference gets even worse for longer outputs.

Your environment

Tested on linux and on Mac OS X (m2)

Quarto check output

$ quarto check
Quarto 1.8.27
[✓] Checking environment information...
      Quarto cache location: /Users/davydovi/Library/Caches/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.6.3: OK
      Dart Sass version 1.87.0: OK
      Deno version 2.3.1: OK
      Typst version 0.13.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.8.27
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /Library/TeX/texbin
      Version: 2024

[✓] Checking Chrome Headless....................OK
      Using: Chrome found on system
      Path: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
      Source: MacOS known location

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.14.2
      Path: /opt/homebrew/opt/python@3.14/bin/python3.14
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.5.1
      Path: /opt/homebrew/Cellar/r/4.5.1/lib/R
      LibPaths:
        - /opt/homebrew/lib/R/4.5/site-library
        - /opt/homebrew/Cellar/r/4.5.1/lib/R/library
      knitr: 1.50
      rmarkdown: 2.29

[✓] Checking Knitr engine render......OK

[cscheid]

Thanks for the report. We agree with you that this performance is simply unacceptable.

I suspect this is coming from one of our attempts to maintain Markdown compatibility across the Pandoc changes that cause syntax drift.

(The following isn't an excuse, but rather an explanation of how this snuck past us) I think we're getting bit by the curse of accidental O(n^2) behavior. If the performance bug were any worse, then much smaller documents would have caused a bigger problem and it would have been "too loud". O(n^2) is precisely at the worst situation. Bad enough to eventually become an issue (like you're seeing), but not so bad that it's instantly obvious (and so it sneaks past our review).

With that said, thank you for a very clean reproduction. It should be enough for us to diagnose and fix.

[cscheid]

And, just to illustrate how pernicious these issues can be, try the following on RMarkdown:

---
title: "Minimal reprex"
output: html_document
---

```{r}
cat(strrep("x \n", 100000))
```

You'll find that RMarkdown now also takes upwards of 20s. (Again, none of this is an excuse but rather an illustration of how obnoxious performance issues can be to track down.)

[cscheid]

I've isolated this specific performance issue in Quarto to somewhere inside the pre-combined-figures-theorems-etc filter.

[cscheid]

It's src/resources/filters/quarto-pre/engine-escape.lua.

[idavydov]

Very interesting, thanks @cscheid .

Interestingly enough, on my machine cat(strrep("x \n", 100000)) completes in <1sec with rmarkdown::render(), but litedown seems to struggle. @yihui FYI this code this seem to choke litedown.

[yihui]

@idavydov Wow, that's really interesting. Thanks for letting me know! I just figured out the root cause and fixed it in xfun::record().

install.packages('xfun', repos = 'https://yihui.r-universe.dev')

Now:

> system.time(litedown::fuse('test.Rmd'))
   user  system elapsed 
  1.214   0.012   1.249 
> system.time(rmarkdown::render('test.Rmd', quiet = TRUE))
   user  system elapsed 
  0.548   0.046   0.641 

litedown is still slower than rmarkdown (due to my least favorite string manipulation hacks to bring the commonmark baby a little closer to pandoc), but I guess it's no longer unacceptably slow now.

[cscheid]

It's specifically

      el.text = el.text:gsub("(\n?[^`\n]+`+)" .. patterns.engine_escape, "%1%2")

We're all trying to find the guy, etc etc

fc6e1b1b7d (Carlos Scheidegger  2024-04-11 11:18:05 -0700  29)       el.text = e
l.text:gsub("(\n?[^`\n]+`+)" .. patterns.engine_escape, "%1%2")

I've confirmed that the ? I added on that commit is responsible for the performance regression.

[cscheid]

PR is up. Thank you for the report!