-2

I have seen tokens like this:

 var message = "Hello, {Name}";

and like this:

 var message = "Hello, ${name}";

and like this:

 var message = "Hello, @NAME";

and a few other styles over the years.

Is there a common or preferred token format, style or character used in creating a set of tokens?

I have tried to used constants, like so,

"The width is WIDTH"

but I ran into problems when I had a token phrase that was contained in another token phrase.

For example, I had WIDTH and PERCENT_WIDTH as tokens. If the string contained both tokens one token could be replaced by the other in the first run through.

So if the token replacement value was 50 and the string was:

"The width of the item is WIDTH. That is PERCENT_WIDTH of the total."

then I ended up with,

"The width of the item is 50. That is PERCENT_50 of the total."

It wouldn't find "PERCENT_WIDTH" because that token string was modified.

So if I replaced the "PERCENT_WIDTH" token first that would solve that problem but as I added more tokens that would increase the chances for errors. So now I'm trying to figure out a recommended token format.

I'm using regex to find and replace matches.

There appears to be no "tokens" topic tag.

UPDATE

For those reading later on. Adding a symbol in front of a reserved token name prevents the problems I mentioned above.

So for example,

"WIDTH, PERCENT_WIDTH" ends up as "50, PERCENT_50" if you replace WIDTH first.

But if you prepend a character or symbol (glyph whatever) to the token replacement order doesn't matter.

"$WIDTH, $PERCENT_WIDTH" ends up as "50, 50%" no matter what order you replace the tokens

UPDATE 2
I realized last night there might be a case where using the symbol and the phrase like this, $WIDTH, would break but I can't remember it now. I just remember thinking that I need to use brackets to enclose the string. Wait I think I'm remembering it.

So if you have $WIDTH and $WIDTH_PERCENT then you'd run into the same problem above.

So if the token replacement value was 50 and the string was:

"The width of the item is $WIDTH. That is $WIDTH_PERCENT of the total."

then you'd end up with,

"The width of the item is 50. That is 50_PERCENT of the total."

So now, I'm going to try to use {WIDTH} and see if that causes any issues.

1.21 gigawatts
  • 1,209
  • 1
  • 10
  • 22
  • 4
    There isn't an agreed upon style. As for your issue, don't try to make something overly general - try to fit it to your specific problem. You can add delimiters, or some kind of a special indicator character, then match those too. E.g. `"The width of the item is {WIDTH}. That is {PERCENT_WIDTH} of the total."` or whatever. `{WIDTH}` won't match `{PERCENT_WIDTH}`. This can work well if you know that these special characters won't appear in the data itself; if they do (if you need to support escaping them), it becomes more complicated. – Filip Milovanović Oct 15 '20 at 05:46
  • Yes thanks. Using a special indicator characters fixes the errors above and brackets would do the same. – 1.21 gigawatts Oct 15 '20 at 16:54
  • 2
    Anyone with more than 3 functioning brain cells will be able to easily recognize any of those three options you presented. – whatsisname Oct 15 '20 at 20:22
  • Burnout or late night coding is common in software development where sometimes people have less than 3 functioning brain cells. I appreciate the comment! :) – 1.21 gigawatts Oct 16 '20 at 00:16
  • You just showed 4 different ones which indicates there's no agreement. – user253751 Oct 16 '20 at 17:11
  • Well, I don't know everything dude. – 1.21 gigawatts Oct 16 '20 at 19:29
  • Also, I just found a 2nd case where 2 token types would fail. Using a simple string would fail if there is another token that contains that phrase. Using a symbol at the front of the token string also fails if another token has the same sub string. I've updated the answer to show this. – 1.21 gigawatts Oct 16 '20 at 19:36
  • @1.21gigawatts: let's get something straight, are you talking about tokens for computer science, which means the academic study where you are communicating with other people, or are you trying to decide a interpolation system for a practical implementation and dealing with all of the edge cases and practical problems that go along with it? Because your initial question points to the first and your updates indicate the second. – whatsisname Oct 16 '20 at 19:50
  • @whatsisname Where I'm from computer science is "the study of algorithmic processes and computational machines. As a discipline, computer science spans a range of topics from theoretical studies of algorithms, computation and information to the practical issues of implementing computing systems in hardware and software." and since there didn't seem to be any defined implementation mentioned as of yet I'm taking notes on what I've encountered. – 1.21 gigawatts Oct 16 '20 at 20:26

1 Answers1

6

No.

This is called string interpolation or substitution. Your language may define a syntax, your framework or tooling might define a syntax, and you can always define your own syntax and rules.

This is a varied topic, ranging from injections fields in Word documents, to simply including a users name on a web page. There is no one size fits all solution, and and the closest thing to a universal approach is probably C’s sprintf, which isn’t all that universal.

jmoreno
  • 10,640
  • 1
  • 31
  • 48