Talk.Escape Character Proposal

Theres already a Add No Wiki Escape Proposal, but this only discusses the use with nowiki and preformatted markup. This proposal could be a general discussion taking into consideration all markup elements as well as the exact definition of whitespace at the beginning of lines - they should have no meaning in creole and therefore not escaping anything.

-- ChristophSauer 2007-02-22

Gregor Hagedorn I dislike the use of backslash for this, since it will have to be escaped extremely frequently when a Wiki is used in the documenting or support of a Windows environment. Pathnames would then look like (Problem: How do I escape the triple-brace inside triple-brace text? O, the tilde seems to work, but is that Creole? - another argument why this proposal is essential!)

Look at: N:{{{\}}}Users{{{\}}}MailHelp ...

I propose to use tilde, which would be reasonable safe in my experience.

Sorry, I took the liberty to move your discussion here -- we use separate pages for talking. Hope you don't mind.

It's a normal practice to mark variables, formulas, code snippets and paths in the code. Given your example, you woudl write:

Look at: {{{N:\Users\MailHelp}}}...

You can put a triple closing curly brace in preformatted block without any problems in a real Creole parser -- see Add No Wiki Escape Proposal. You can also include a triple curly brace at the end of a nowiki span -- and open a new span immediately, if you need to put something more there.

As opposed to tilde, triple curly braces and backslashes never appear in normal text. Non-normal text, like paths, code, etc. has to be ascaped anyways -- otherwise it would be impossible to create any markup at all without a risk of conflicts. Tilde can be traditionally used to separate number ranges (especially with negative numbers), show approximate dates, etc. It was also proposed for subscript in Creole additions.

-- RadomirDopieralski, 2007-02-26

A bit selfish, maybe... but tilde is one of the most difficult characters to get on Italian keyboards.

-- Michele Tomaiuolo, 2007-02-26

Yes, it's difficult on German keyboards too, but this character is for EdgeCases - in the seldom cases you need it, it is acceptable (IMHO).

-- Christoph Sauer, 2007-02-26

I still have trouble understanding what is the scope of a single escape character in the source. It seems to be fuzzy at least. For example, if I write {{foo}}}}}, how many closing curly braces will be escaped? The idea that the escape character applies to single "element of markup" is evil, because it requires the users to understand the inner workings of the parser -- the "element of markup" will vary depending on thei mplementation too. Another objection I have is that I tested the escape character here on the JSPWiki a little, and I must say it doesn't always work -- it only works for the markup that somebody thought to include. This is a big problem in my opinion. -- RadomirDopieralski, 2007-02-27

In my opinion, a single character should be escaped. Obviously, this can provoke a whole markup element to be escaped.

For example, \[[this]] shouldn't be a link. It would be better to escape all brackets, but it's up to the user.

It should be decided also how escaping will work in nowiki sections. It could work just for closing braces. For example, Creole specs could require to remove a tilde or backslash from all sequences like these ones: \\\} ~~~} This way, collisions with included text would be reduced to the minimum.

-- Michele Tomaiuolo, 2007-02-27

I definitely prefer the one character approach -- at least it is predictable. As for escaping in pre, what's wrong with AddNoWikiEscapeProposal? Do we revoke it if this proposal passes?

I have another thing to consider. Can each developer think about it for a while and describe how he would go about implementing such an escape character in the parser he is developing?

-- RadomirDopieralski, 2007-02-27

Think its fairly straight forward for mine, once see a \ followed by creole markup character output the markup character directly and advance two characters.

The only problem may have is with dealing with it inside nowiki & link markup. At the moment my parser just searches for }}} , ~]] and | . So perhaps would have to use a regexp to make sure the preceeding character isn't a \.

-- JaredWilliams, 2007-02-28

As the code is short, I'll post it here.

    var $regex = '/[\\\]([\W_])/';

    function process(&$matches)
    {
        if ($matches[1] == ' ') $matches[1] = '\\';
        return $this->wiki->addToken(
            $this->rule,
            array('text' => $matches[1])
        );
    }

Explanation:

1. It uses backslash as escape char.
2. Two backslashes are handled as a linebreak - by another rule which fires before this one.

I made two other assumptions:

1. Backslash before an alphanumeric char does not fire this rule (untouched).
2. Backslash followed by space is translated as a backslash (space is removed - this is just for completeness).

Very very simple.

-- Michele Tomaiuolo, 2007-02-28

Michele, Jared, this is a very nice and clean implementation, but it's not what is proposed on EscapeCharacterProposal, is it?

-- Radomir Dopieralski, 2007-02-28

In a sense, it is, as long as the proposal is not yet well defined. Replacing backslash for tilde would take 2 minutes (but... I still hope to keep backslash).

I'm not sure if the general escaping should be applied in nowiki sections (both inline and block).

Probably applying the general rule is the easiest solution to understand and explain, even if users will have to pay attention to backslashes in the nowiki section.

Escaping should always have one-character scope. I agree.

-- Michele Tomaiuolo, 2007-02-28

Seems to me that we now have several decisions to make to create a proper proposal:

• What should be the scope of the escape character:
  • Single character
  • Single "piece" of markup
  • Single document element (in DOM sense)
  • Other
• Where should be the escape character recognized:
  • Anywhere
  • Anywhere but pre blocks
  • Anywhere but pre blocks and nowiki
  • Only where commonly needed
• What characters should it escape:
  • Any charaters
  • Only non-alphanumeric characters
  • Only characters used in markup
  • Only characters that would otherwise be interpreted as markup
  • Other
• What character should be used for escaping:
  • Backslash \
  • Tilde }}} ** Percent {{{%
  • Other
• Other issues, like how to obtain the character itself and resolve conflicts with exisitng markup:
  • Change markup for forced newline?
• Do we need it at all? (answered when the proposal is ready)

What do you think? Did I omit something or add something that is not important?

-- Radomir Dopieralski, 2007-02-27

Re-organizing contrasting alternatives was badly needed, Radomir. Great! In fact, I left many options open in the proposal on purpose, to discuss them.

Anyway, my own answers to your questions: ACBA.

As for nowiki (block and inline), we need a consistent way to break the closing sequence and rebuild it in output. It's a different need from general escaping - though it would also be a solution.

As for what to escape, the mechanism should work in other wikis too, where more chars are potentially used for markup.

-- Michele Tomaiuolo, 2007-02-28