Why not program our video text terminals/terminal emulators to use something JSON or XML on the backend instead of ANSI escape sequences?

Question

Backstory (You can skip)

Awhile back I was developing a console toolkit for displaying debug messages and the like: It gives me colour coding, blinking, underlines, bold, italic, etc.

While developing this library, I quickly learned that nesting with ANSI escape sequences was impossible, but assumed that good reasons existed why this was the case.

Of course working with other document types, nesting is more or less trivial:

<foo> the <bar> quick </bar> brown </foo>

or with JSON

{
        type=foo,
        text=[
                "the",
                {type=bar, text="quick",},
                "brown",
        ]
}

But with Ansi, its something like this:

\e[1m the \e[2m quick \e[1m brown

giving an output like this:

Basically meaning that you would need to manually track the formatting, and explicitly construct escape sequences to represent all output moving forward. You can obviously make due, but it complicates things. Before complaining about this, I'd like to clarify what reasons exist that would necessitate an escape sequence model over say, a structured document style.

Questions:

Is it purely due to legacy reasons why displaying text on our video terminals is done with ANSI escape sequences and not another framework such as JSON, Yaml, XML, or something else?

Is ANSI escape sequences in video terminal, simply an old technology similar to say, X11 that sticks around solely due to how embedded it is within the computing paradigm?

If not, why don't developers switch from an escape sequence style to something that would support nesting?

Are there any proposals to do away with ANSI escapes in terminals and replace it with something else?

In short: backwards compatibility. [ANSI escape codes](https://en.wikipedia.org/wiki/ANSI_escape_code) is a widely accepted standard, much older than JSON, XML or even HTML. If devs today wants to displaying fancy coloured text or tables using a hierarchical description language, they can utilize a web browser for this. — Doc Brown, Jun 13 '19 at 13:33
@Steve: does not convince me. If "efficiency" would be important here, why does the world *use* a (mostly) hierarchical description language (HTML) for displaying text and other things over the web? — Doc Brown, Jun 13 '19 at 13:37
The thing you seem to be missing is that terminals were originally not emulator programs, but dumb, cheap hardware that allowed users to interact with a computer. Often, timeshare systems would have multiple terminals. Terminals started out as teletypes that just printed the output on paper (this line-oriented user interface led to the "ed" editor). Later terminals were CRT screens that could render a character matrix, and this is where ANSI escape codes originated. E.g. "bright" colors cause the CRT to render that cell twice per screen refresh, causing it to glow more brightly. — amon, Jun 13 '19 at 14:16
I'm with Doc Brown. It's an old standard, and you follow it because it's the standard. If the standard doesn't meet your needs, you use something else like a web browser. We can cogitate over the relative merits of the standard, but it is what it is. — Robert Harvey, Jun 13 '19 at 14:29
@DocBrown, I mean efficiency in a very broad sense (including the complexity of parsers and the complexity of the code which describes the console output), and the reason HTML is used is because that has always been capable of far more complex features than a text terminal. Most old text terminal applications differ from websites in being rather faster to load and render. — Steve, Jun 13 '19 at 14:32
We did mention this is an *old* standard, right? The creators of the standard were *not* thinking about nesting when they created the standard; they merely needed a simple way to mark text for highlighting. — Robert Harvey, Jun 13 '19 at 14:36
@Steve: I agree that efficiency was probably a major concern at the time when those old text terminal applications you are talking of were created. However, that was IMHO not the OPs question. My understanding of that question is: "why is this ANSI escape style still so popular today?", and I don't buy "still efficiency" as an answer for this question. — Doc Brown, Jun 13 '19 at 14:47
@DocBrown OP here, and yes, you are exactly right on what I was asking. — Anon, Jun 13 '19 at 14:51
It wasn't so much for efficiency, as for cost: how much functionality can I deliver while keeping the cost of the terminal below $5000? (Yes, they weren't cheap). — Michael Kay, Jun 13 '19 at 14:56
@DocBrown, I suspect overall simplicity and speed does remain a significant reason why these old technologies are in use. After all, if you want the hierarchical model of HTML, then you'd just use the web browser rather than the console. If the question is why nobody has since developed an all-singing-all-dancing visual framework for console output, it's probably because we have those things and more already with GUIs. A simple hierarchical/xml type parser, which substitutes end tags for the escape sequence of the previous/outer state, would also probably not be very hard to implement. — Steve, Jun 13 '19 at 15:23
If your real question is "why is this ANSI escape style still so popular today," my answer is "it isn't." — Robert Harvey, Jun 13 '19 at 15:37
@RobertHarvey, in what way do you think the two answers to this question fall into the category of "*answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise*"? — David Arno, Jun 13 '19 at 16:31
@RobertHarvey, but you voted to close this question because "*answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise*". How on earth could you make that decision without evaluating the answers? We all expect gnat to vote to close every question without a second thought, but to have a moderator knee-jerk vote close a question without bothering to read the answers makes a mockery of rules of this site. — David Arno, Jun 13 '19 at 16:42
@RobertHarvey, [done](https://softwareengineering.meta.stackexchange.com/questions/8958/what-can-be-done-about-the-misuse-of-the-primarily-opinion-based-close-vote). — David Arno, Jun 14 '19 at 07:49

score 6 · Answer 1 · answered Jun 13 '19 at 13:29

6

Your question is sort of like saying assembly is difficult to work with, so computers should use higher level languages instead. The ANSI format is the right level of abstraction for working with terminal hardware. You have to do the state tracking somewhere, and it's best to not do it in hardware that needs to be as cheap as possible. Even in modern hardware, some library at some point has to do the state tracking level of abstraction.

When you want something easier for a human to use, you use higher-level abstractions that have been around forever. Even something like nroff is a big improvement, and I'm sure you can find libraries and/or utilities for all the markup formats you described.

answered Jun 13 '19 at 13:29

Karl Bielefeldt

146,727
38
279
479

`Your question is sort of like saying assembly is difficult to work with, so computers should use higher level languages instead.` Two issues with this statement: In one sense, we still all work with assembly, but rather our assembly is abstracted out of higher level code. We should rely on say, `c++` generated assembly rather than writing straight assembly, for the sake of time, optimization, and stability. Also, computers don't use terminals, human's do, so it isn't obvious to me why `ANSI` means good for computers while `JSON` is bad for computers, when tracking happens regardless. – Anon Jun 13 '19 at 14:17
2

@Akiva It's actually a brilliant metaphor. Are we using x86 compatible instruction sets because it is so awesome? No, it's an objectively awful assembler language/instruction set for modern computers. We just keep using the x86 architecture because it has a ginormous ecosystem of software that assumes it. Similarly, we keep using ANSI-compatible emulators because we keep using software that needs ANSI escape codes. But unlike with instruction sets, there's no strong reason to switch away from terminals. As Karl mentions, there already are abstraction layers for formatting like nroff. – amon Jun 13 '19 at 15:13
Salient point, but now you have me wondering whether the myth of the awful x86 instruction sets actually pasts muster. – Anon Jun 13 '19 at 15:31
2

@Akiva: by googling a little bit around, I found [this C#/.NET lib](https://github.com/Athari/CsConsoleFormat), which seems to support exactly what Karl Bielefeld suggested for your case: a XAML renderer for ANSI terminals. Since it is compatible to .Net standard, it should work on Linux terminals as well (but I did not try it out by myself). – Doc Brown Jun 15 '19 at 06:22
@DocBrown: *That* is *very* cool. – Robert Harvey Jun 15 '19 at 23:28

score 4 · Accepted Answer · answered Jun 13 '19 at 13:30

4

There is a simple reason. When you are typing you can only input a character at a time.

Something that is processing typed text has to deal with what in effect is invalid markup, half a json blob or half an xml document. You can't display "error" until the typing has finished and then display the result. You have to have a format where partial documents are valid and already input data wont change due to additions.

answered Jun 13 '19 at 13:30

Ewan

70,664
5
76
161

So why not use a buffer then? Or conversely, have the interpreter make due with partial documents until a fuller representation is given? Given the partial information ` quick` you can at the very least, assume, ` quick ` until you get ` quick brown `. – Anon Jun 13 '19 at 14:04
1

that would require you to reprocess the whole string when a character was added. Given that some escape sequences move the cursor that might be problematic. What it's doing is processing each character (or escape sequence) as a stand alone command. Much like a type writer, is the bold key down or up? I don't go back to the start of the document and work my way through counting open closures. I just look at the key – Ewan Jun 13 '19 at 14:32
Hmmm, that is an interesting way to look at terminals, effectively being a virtual bufferless typewriter, which of course would require this style. The only lingering question for me then is, what necessitates us today to have such a device today? What functionality do I gain from this? – Anon Jun 13 '19 at 14:57
2

Well, you can't get away from the fact that its design comes from an age before the mouse. The way we use computers has drastically changed and perhaps it has no value anymore. However, an emacs or vi user would disagree.. perhaps violently! – Ewan Jun 13 '19 at 15:01
1

@Akiva, the main remaining "necessity" for consoles is embedded systems that cannot support the overhead of anything more, and also integration with legacy systems and tools. When I say legacy, I don't necessarily mean defunct, but I mean old systems that are widely available, whose workings are well understood, and whose simplicity and refinement is often still very adequate. For example, the Windows command line interpreter and batch files. – Steve Jun 14 '19 at 13:01

Why not program our video text terminals/terminal emulators to use something JSON or XML on the backend instead of ANSI escape sequences?

Backstory (You can skip)

Questions:

2 Answers2