Ah, one quick Markdown lesson I learned recently.

If you've got Markdown inside of "real" block-level html tags, it's not supposed to be rendered.

<div>This is **some Markdown**.</div>

That shouldn't be bold. It renders as...

Unexpected, but (with a hat tip to this SO answer), apparently correct behavior according to The Grubes:

The only restrictions are that block-level HTML elements — e.g. <div>, <table>, <pre>, <p>, etc. — must be separated from surrounding content by blank lines, and the start and end tags of the block should not be indented with tabs or spaces. Markdown is smart enough not to add extra (unwanted) <p> tags around HTML block-level tags.

...

Note that Markdown formatting syntax is not processed within block-level HTML tags. E.g., you can’t use Markdown-style emphasis inside an HTML block.

Welcome to the land of unintended consequences. It looks like the deal was "stop adding all these danged <p> tags around my inline block-level html; it'd duping the breaks," and we got, "stop ALL markdown rendering in block tags" instead.

There is, however, a non-standard workaround, as described by Python-Markdown's docs:

The markdown attribute can be assigned one of three values: "1", "block", or "span".

Note: The expressions “block-level” and “span-level” as used in this document refer to an element’s designation according to the HTML specification. Whereas the "span" and "block" values assigned to the markdown attribute refer to the Markdown parser’s behavior.

markdown="1"

When the markdown attribute is set to "1", then the parser will use the default behavior for that specific tag.

So if you have

<div markdown="1">This is **some Markdown**.</div>

you should have

This is some Markdown.

... but I need to add that to MarkUpDown if I want to support it. And since it's not like someone is going to add that that doesn't want it, I guess I should!

Labels: gruber, html, markdown, markupdown

Here's a nice command template for Pandoc which, in spite of its foibles, is still a very cool library.

It turns out to be quite simple to convert a docx to markdown. The following example is from the Pandoc demos site.

pandoc -s example30.docx -t markdown -o example35.md

However the generated markdown from the above command has a few issues.

The lines are only 80 characters long. I do not know why an 80-character line length is the default but I do not like it. This is fortunately quite easy to fix with the option –no-wrap.

Links do not use the reference style. I prefer the reference style links because it makes the text less cluttered by moving the link it self to the bottom of the file. This is also easy to fix with the option –reference-links.

With the two options added the command looks like this.

pandoc -s example30.docx --no-wrap --reference-links -t markdown -o example35.md

Now the generated markdown is very readable and close to what I would write myself. 

Okay, I might quibble, well, vehemently deny "quite simple" and "very readable" are accurate. I just tried it on a complicated outline that was making Word throw up, you know, a doc where you try to add another line to your outline and Word gives up, not matching up indent or numbering with the rest of the content. 

I wanted to move to Markdown so I could define precisely where each bullet level was and who belonged to it. It's painful that we're in 2021 and we haven't come to the realization that you have to use some sort of markup code for complex document authoring. No rich text editor will ever achieve true WY[intend]IWYG.

Pandoc results were... not great. Here's a snippet:

3.  Unchecked and Enabled when instructor has access for none of the selected questions (I-I/F) -- TC 697849
iii. 
iv. User perms none \<\<\< Impossible! User owns some in this situation.
    1.  
    2.  
    3.  
```{=html}
<!-- -->
```
e)  Transfer Ownership is visible -- TC 694132
f)  Remove is visible -- TC 694132

Yuck. It didn't even convert bulleted lists with a), b), c) to a list in Markdown at all. To be clear, I don't have empty 1., 2., 3. in the original. Those are hidden in Word's DOM somewhere. Ugh.

So now I'm in VIm setting things right.

But it's still a good command template:

pandoc -s in.docx --no-wrap --reference-links -t markdown -o out.md

Labels: markdown, noteToSelf, pandoc, Word

See previous post about too many bad devs running around. It's not brand new. They are and have been everywhere.

From newscientist.com:

Parking meters, cash registers and a professional wrestling video game have fallen foul of a computer glitch related to the Y2K bug.

…

Programmers wanting to avoid the Y2K bug had two broad options: entirely rewrite their code, or adopt a quick fix called “windowing”, which would treat all dates from 00 to 20, as from the 2000s, rather than the 1900s. An estimated 80 per cent of computers fixed in 1999 used the quicker, cheaper option.

“Windowing, even during Y2K, was the worst of all possible solutions because it kicked the problem down the road,” says Dylan Mulvin at the London School of Economics.

Because of course they did.

Utility company bills have reportedly been produced with the erroneous date 1920, while tens of thousands of parking meters in New York City have declined credit card transactions because of the date glitch.

Would non-windowing require a deeper refactor? Yes. Should they have done that then? Maybe?

The real question is if the developers let their management know the limitations of the fix, and if the managers told the customers.

Because if there's one thing that's lacking as much as good developer instincts in software development, it's managers who can spell "success".

From the same article:

“Fixing bugs in old legacy systems is a nightmare: it’s spaghetti and nobody who wrote it is still around,” says Paul Lomax, who handled the Y2K bug for Vodafone.

"Old legacy systems" you say? That's where code is spaghetti? Lolls.

What you should say is, "any mature codebase". There are always stupid edge conditions captured in that code, which makes it look less like a useless textbook example that faints at the first site of The Real World (c) MTV and more like something that's doing its job.

There is no clean code in production. I mean, there is, and if you handle your method signatures correctly, there's more in your code than elsewhere, but, um, there's not much idealistically pure, ivory tower type sample code making people money. And if there is, it's about to let them down in some serious-yet-unexpected way.

Related..., if you wrote the preceding blog entry in Markdown:

Adding support for a strikethrough syntax is a longstanding request for Markdown, but I omitted it by design. For one thing, there is no good punctuation to represent strikethrough. Tildes don’t look like strikethrough at all — tildes are squiggly, but a strike is a straight line. 

You know what doesn't look like a strikethrough at all, John? <strike></strike> ;^D

Labels: developers, development, fail, markdown

Until recently, MarkUpDown had allowed you to paste copied HTML snippets into Markdown with all of the html styling retained using the not-so-discoverable shortcuts alt-v and, with quote, alt-shift-v).

This pasted those snippets into your Markdown with all of the HTML's CSS, making for the potentially neat situation where your quote was visibly very different from the rest of your page, preserving the look & feel of the page which provided your quote.

I quickly found that preserving colors could make for ugly pastes, and wrote some code to remove those styles, but otherwise, quotes would appear to be very similar to the original.

So not exact, but you can see the resemblance with the original:

But the code was also very ugly. There was so much CSS it started to ruin the whole "shorthand" nature of Markdown.

<p style="word-spacing: 0.05em; line-height: 1.6em; position: relative; z-index: 1; font-family: ‘Source Sans Pro’, ‘Helvetica Neue’, Arial, sans-serif; font-size: 15px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration- ">Vue (pronounced /vjuː/, like<span>&nbsp;</span><strong style="font-weight: 600; ">view</strong>) is a<span>&nbsp;</span><strong style="font-weight: 600; ">progressive framework</strong><span>&nbsp;</span>for building user interfaces. Unlike other monolithic frameworks, Vue is designed from the ground up to be incrementally adoptable. The core library is focused on the view layer only, and is easy to pick up and integrate with other libraries or existing projects. On the other hand, Vue is also perfectly capable of powering sophisticated Single-Page Applications when used in combination with<span>&nbsp;</span><a href="https://vuejs.org/v2/guide/single-file-components.html" style="text-decoration: none; font-weight: 600;">modern tooling</a><span>&nbsp;</span>and<span>&nbsp;</span><a href="https://github.com/vuejs/awesome-vue#components–libraries" target="_blank" rel="noopener" style="text-decoration: none; font-weight: 600;">supporting libraries</a>.</p>

This html code was provided by Microsoft’s HTML Clipboard Format. The CSS dump above is, honestly, essentially exactly what I get from the clipboard’s garbled mess. Browsers seem to translate many classes into inline styles, and the clipboard format doesn’t do anything to normalize it.

(To the HTML Clipboard Format’s credit, however, you also get the original link, so that "From VueJS.org" link preceeding the quote is a "free" part of the "paste html as quote" action in MarkUpDown.)

Here’s the first p tag from that html again with some linebreaks…

<p
 style="word-spacing: 0.05em; line-height: 1.6em; position: relative; z-index: 1;

 font-family: ‘Source Sans Pro’, ‘Helvetica Neue’, Arial, sans-serif; font-size: 15px;
 font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal;
 font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px;
 text-transform: none; white-space: normal; widows: 2; -webkit-text-stroke-width: 0px;

 text-decoration-style: initial; text-decoration- ">

Good grief, right? You don’t want all that trash in your otherwise pretty clean Markdown, do you?

The first thing I tried was to pull all the styling out into classes that I’d put just north of the blockquote, making something like this…

</style> > <p class="q233941–321">Vue (pronounced /vjuː/, like …

I also started trying to normalize the classes and pull out shared settings when they overlapped so there wouldn’t be as much text before the blockquote. That was fun.

The classes route is better, at least once you get to the blockquote, but that’s still a lot of markup to inject into a markup shorthand. I didn’t like it.

So for now I’ve removed inline styles and classes from pastes, which I’m about to release. I don’t feel great about, but you do retain the markup, most importantly bold, italics, links, even images, which is a heck of a lot better than treating HTML as plain text. But it isn’t nearly as pretty to look at now.

I’m planning on bringing styles back under the "opt-in" settings under the "Beta Features" section of settings, but it’s hard to justify the overhead I’ve already spent on it. The Markdown stays cleaner now. That’s ultimately an improvement for a Markdown editor.

Oh well. Thus is the life of a utility author.

Labels: html, markdown, markupdown, style

There's a reason John Gruber's initial spec for Markdown didn't approach the level of complexity the CommonMark people wanted: You end up with exceptionally wordy jive like what we find for list items.

From commonmark.org:

And let M be the marker 1., and N = 2. Then rule #1 says that the following is an ordered list item with start number 1, and the same contents as Ls:

Example 215Try It

[example removed -mfn]

The most important thing to notice is that the position of the text after the list marker determines how much indentation is needed in subsequent blocks in the list item. If the list marker takes up two spaces, and there are three spaces between the list marker and the next non-whitespace character, then blocks must be indented five spaces in order to fall under the list item.

Phew. That was fun, wasn't it?

I was in the CommonMark spec today for two reasons. First, I fairly recently (well, months ago) swapped my main Markdown engine to the .NET CommonMark renderer (don't worry; table support and other features are done outside of CommonMark). Second, I wanted to know why only two of the following were creating nested lists, though my old renderer would create nested lists from all three.

1. Testing ye olde bullet points
    2. Still testing
    3. Testing

1. Testing ye olde bullet points
    * Still testing
    * Testing

1. Testing ye olde bullet points
    1. Still Testing
    2. Testing

Curious? You've probably guessed, but here's the rendered HTML, first as HTML, then the HTML source.

<ol>
<li>Testing ye olde bullet points
2. Still testing
3. Testing</li>
</ol>
<ol>
<li>Testing ye olde bullet points
<ul>
<li>Still testing</li>
<li>Testing</li>
</ul>
</li>
</ol>
<ol>
<li>Testing ye olde bullet points
<ol>
<li>Still Testing</li>
<li>Testing</li>
</ol>
</li>
</ol>

See what happened there? If your nested list starts with a 2., as in the first example, though that'd be a bulleted list without the nest...

2. See? It's still...
4. a list.

2. See? It's still...
3. a list.

... it's not when nested.

Note that CommonMark is paying attention to the starting number of your ordered list. See how the list, above, starts with 2., though the next bullet is a 3., even though I used 4. in the raw Markdown?

Why is the nesting ignored in the first example? The spec tells us:

From commonmark.org:

Since it is well established Markdown practice to allow lists to interrupt paragraphs inside list items, the principle of uniformity requires us to allow this outside list items as well. (reStructuredText takes a different approach, requiring blank lines before lists even inside other list items.)

In order to solve of unwanted lists in paragraphs with hard-wrapped numerals, we allow only lists starting with 1 to interrupt paragraphs. Thus,

Example 265Try It

The number of windows in my house is
14.  The number of doors is 6.

<p>The number of windows in my house is
14.  The number of doors is 6.</p>

We may still get an unintended result in cases like

Example 266Try It

The number of windows in my house is
1.  The number of doors is 6.

<p>The number of windows in my house is</p>
<ol>
<li>The number of doors is 6.</li>
</ol>

but this rule should prevent most spurious list captures. [emphasis mine -mfn]

Make sense? 1. is special. If your nest doesn't start with 1., CommonMark thinks you're in the same "paragraph", give or take. You're not going to get that nest.

Which means MarkUpDown needs to capture when someone has a bullet from a numbered list, hits return, gets the next number automatically inserted...

1. They enter this first bullet.
2. Hit return and get a two. Then hit tab...

... which leaves them with...

1. They enter this first bullet.
    2. Hit return and get a two. Then hit tab...

... which doesn't render as a nested list. What they likely want is...

1. They enter this first bullet.
    1. Hit return and get a two. Then hit tab...

I'd hoped to get in automatic renumbering of ordered lists anyhow, but we'll start with this simpler change first... if it looks nested, we're going to reset the number to 1.

I realize this is a pretty boring post if you're not a Markdown aficionado. I've got some more interesting drafts around here somewhars. I'll try to find time to post them, but this one bugged me enough to write up.

Labels: CommonMark, markdown, markupdown

From Microsoft's download page for their RTF Specification:

That's right. You can't read the RTF specification in an RTF editor. You need something that reads .doc or .docx first. (Admittedly, this is from March of 2008, so this is still Ballmer's Microsoft.)

I'm guessing someone would say that OOXML is open (extensions aside), and that I could write that first.

That looks like fun. The pdf displayed below -- part 1 of edition 4 which does not supersede and refers back to edition 1, which is not included -- is over 5000 pages.

Raw XML can be human-readable, but there's no guarantee. UTF can be just as obfuscating as binary. And security through obfuscation is really all we ever had or have. It's all open source once you get to the zeroes and ones.

At least the OOXML format is given in a pdf. 🙄

I know, LibreOffice et all do a decent job with .doc. I can read the spec "for free". It's actually quite interesting, and quickly lets you know that some people still expected to read rtf docs' source when the format was created -- in this way, it's not wholly unlike Markdown.

Unlike most clear text files, an RTF file does not have to contain any carriage return/line feed pairs (CRLFs) and CRLFs should be ignored by RTF readers except that they can act as control word delimiters. RTF files are more readable when CRLFs occur at major group boundaries.

Labels: formats, markdown, rtf

Let's talk HTML Fragments in your clipboard when coding on Windows. I've created what's arguably the best Markdown editor on Win10 (if I do say so myself), and wanted it to be more intelligent when I was pasting content that comes from web browsers. That is, in the original version of MarkUpDown, if you copied the text I've got highlighted, below, from DaringFireball...

... and you pasted it into MarkUpDown as a quotation (Ctrl-Shift-V), you'd get unstyled text, like this...

> Mossberg:
> For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.
> Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.
> These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, they stop trying.

I mean, that's okay-ish. Unfortunately, you'd have to go put in an additional line after each line so that it's not all scrunched into one paragraph. And you lose that Gruber was block-quoting Mossberg. And you lose that "they stop trying" was originally in italics. That stinks.

Isn't Markdown really supposed to be shorthand for html? Can't we handle pasting html better than that?

Yes, yes we can...

TL;DR

Go pull the HtmlFragmentHelper, a working, in-progress library for turning HTML Fragment strings into view models with these properties...

public class HtmlFragmentViewModel
{
    public string Version = "";
    public int StartHtml = int.MinValue;
    public int EndHtml = int.MinValue;
    public int StartFragment = int.MinValue;
    public int EndFragment = int.MinValue;
    public string SourceUrl = "";
    public string FragmentSourceRaw = "";
    public string Error = "";

    //...
}

There are also several convenience methods to help you get, say, the top- and second-level domains for the HTML Fragment's source URL (example.com from http://blog.example.com/user/blog0000231.html), or just the html from the selected fragment (myHtmlFragmentViewModel.FragmentSourceParsed), etc.

Here's an example:

public string CreateQuote(string htmlFragmentSource) 
{
    string strIntroLink = string.Empty;
    HtmlFragmentViewModel vm = new HtmlFragmentViewModel(htmlFragmentSource);

    if (vm.SourceUrl.Length > 0 && vm.SourceUrlDomainSecondAndTopLevelsOnly.Length > 0)
    {
        strIntroLink = string.Format("From <a href=\"{0}\">{1}</a>:" + Environment.NewLine + Environment.NewLine,
            vm.SourceUrl, vm.SourceUrlDomainSecondAndTopLevelsOnly);
    }

    return strIntroLink + vm.FragmentSourceParsed;
}

Read how to use it here. Get it from GitHub, though the only file you really need is here.

And here's a quick article on how to get code back into the clipboard as html, which I'm not doing.

HTML Fragment Format

Want to paste clipboard HTML as HTML? Enter Microsoft's HTML clipboard format. Instead of using the standard code to read the clipboard's contents, like this...

string clipboardText = await Clipboard.GetContent().GetTextAsync();

... now you use this...

string htmlClipboardText = await Clipboard.GetContent().GetHtmlFormatAsync();

But when you look at the contents of htmlClipboardText, you quickly notice that strange things are afoot at the Circle K.

What comes back isn't a string with html source, but an "HTML Fragment" formatted string that looks like this:

HTML Fragment from Edge

Version:1.0
StartHTML:000000210
EndHTML:000003550
StartFragment:000002696
EndFragment:000003500
StartSelection:000002696
EndSelection:000003496
SourceURL:http://daringfireball.net/2016/10/mossberg_siri
<!DOCTYPE HTML>
<HTML lang="en"><HEAD>       <!-- Open Graph [jive] --> <!-- 
    <meta property="og:site_name"   content="Daring Fireball" />
    <meta property="og:title"       content="Walt Mossberg: ‘Why Does Siri Seem So Dumb?’" />
    <meta property="og:url"         content="http://daringfireball.net/2016/10/mossberg_siri" />
    <meta property="og:description" content="In addition to the engineering hurdles to actually make Siri much better, Apple also has to overcome a “boy who cried wolf” credibility problem." />
    <meta property="og:image"       content="https://daringfireball.net/graphics/df-square-192" />
    <meta property="og:type"        content="article" />
 -->         <!-- Twitter Card [jive] -->                    <TITLE>Daring Fireball: Walt Mossberg: ‘Why Does Siri Seem So Dumb?’</TITLE>        <LINK href="/graphics/apple-touch-icon.png" rel="apple-touch-icon-precomposed">     <LINK href="/graphics/favicon.ico?v=005" rel="shortcut icon">   <LINK href="/graphics/dfstar.svg" rel="mask-icon" color="#4a525a">  <LINK href="/css/fireball_screen.css?v1.7" rel="stylesheet" type="text/css" media="screen">     <LINK href="/css/ie_sucks.php" rel="stylesheet" type="text/css" media="screen">     <LINK href="/css/fireball_print.css?v01" rel="stylesheet" type="text/css" media="print">    <LINK href="/feeds/main" rel="alternate" type="application/atom+xml">   
<SCRIPT src="/mint/?js" type="text/javascript" async=""></SCRIPT>

<SCRIPT src="http://www.google-analytics.com/ga.js" type="text/javascript" async=""></SCRIPT>

<SCRIPT src="/js/js-global/FancyZoom.js" type="text/javascript"></SCRIPT>

<SCRIPT src="/js/js-global/FancyZoomHTML.js" type="text/javascript"></SCRIPT>
     <LINK title="Home" href="/" rel="home">     <LINK href="http://df4.us/pfz" rel="shorturl">  <LINK title="Apple Responds to Dash Controversy" href="http://daringfireball.net/2016/10/apple_dash_controversy" rel="prev">    
<SCRIPT src="http://daringfireball.net/mint/?record&amp;key=383950464d37374b39333637695970466458724e6779513431&amp;referer=&amp;resource=http%3A//daringfireball.net/2016/10/mossberg_siri&amp;resource_title=Daring%20Fireball%3A%20Walt%20Mossberg%3A%20%u2018Why%20Does%20Siri%20Seem%20So%20Dumb%3F%u2019&amp;resource_title_encoded=0&amp;window_width=1756&amp;window_height=921&amp;resolution=2438x1371&amp;flash_version=0&amp;1476397798179&amp;serve_js" type="text/javascript"></SCRIPT>
</HEAD><BODY onload="setupZoom()"><DIV id="Box"><DIV id="Main"><DIV class="article"><!--StartFragment--><P>Mossberg:</P><BLOCKQUOTE><P>For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.</P><P>Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.</P></BLOCKQUOTE><P>These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, <EM>they stop trying</EM>.</P><!--EndFragment--></DIV></DIV></DIV></BODY></HTML>

Wow. No, really, wow. That surprised me. I was using Edge this time instead of Chrome, and whoa. Edge includes the entire page's header. That said, that's really not a horrible idea. We have really good context for this fragment, and if we wanted the original CSS, for example, we know where to get it.

Aside: Look at the info DaringFireball's sucking in:

resolution=2560x1440&
flash_version=0&
1476452395012&
serve_js

Gruber doesn't use Flash. Kinda Panopticlicky, ain't it? Looks like it's supporting this.

Another Aside: Internet Explorer 11 creates the same fragment source, so if you think Edge was created from the ground up...

Interestingly, the old Internet Explorer copyright box used to reference Mosaic, but that's gone in IE11. Wonder if IE11 is really the child of IE6 and 7? But enough aside-ing...

HTML Fragment from Chrome

But let's take a look at where I initially started, with Chrome.

Version:0.9
StartHTML:0000000164
EndHTML:0000002719
StartFragment:0000000200
EndFragment:0000002683
SourceURL:http://daringfireball.net/2016/10/mossberg_siri
<html>
<body>
<!--StartFragment--><p style="margin: 0px 0px 1.6em; padding: 0px; color: rgb(238, 238, 238); font-family: Verdana, &quot;Bitstream Vera Sans&quot;, sans-serif; font-size: 11px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(74, 82, 90);">Mossberg:</p><blockquote style="font-size: 11px; margin: 2em 2em 2em 1em; padding: 0px 0.75em 0px 1.25em; border-left: 1px solid rgb(119, 119, 119); border-right: 0px solid rgb(119, 119, 119); outline: 0px; vertical-align: baseline; background: rgb(74, 82, 90); color: rgb(238, 238, 238); font-family: Verdana, &quot;Bitstream Vera Sans&quot;, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-t
ransform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><p style="margin: 0px 0px 1.6em; padding: 0px;">For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.</p><p style="margin: 0px 0px 1.6em; padding: 0px;">Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.</p></blockquote><p style="margin: 0px 0px 1.6em; padding: 0px; color: rgb(238, 238, 238); font-family: Verdana, &quot;Bitstream Vera Sans&quot;, sans-serif; font-size: 11px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacin
g: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(74, 82, 90);">These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems,<span class="Apple-converted-space"> </span><em>they stop trying</em>.</p><!--EndFragment-->
</body>
</html>

See the difference? Chrome's fragment initially looks much more focused, but also provides much less information about the code's original context. Worse, Chrome's puts all the CSS information inline, and repeating each CSS property inside of each pasted html element makes the source a long ways away from DRY. And that out-of-context, inline CSS is lossily translated, which will cause us some rendering problems that we'll see and discuss in a bit.

Original Source

Now Edge's fragment isn't perfect. Here's the original html of that snippet straight from the server...

<p>Mossberg:</p>

<blockquote>
  <p>For instance, when I asked Siri on my Mac how long it would take
me to get to work, it said it didn’t have my work address &#8212;
even though the “me” contact card contains a work address and
the same synced contact card on my iPhone allowed Siri to give
me an answer.</p>

<p>Similarly, on my iPad, when I asked what my next appointment was,
it said “Sorry, Walt, something’s wrong” &#8212; repeatedly, with
slightly different wording, in multiple places on multiple days.
But, using the same Apple calendar and data, Siri answered
correctly on the iPhone.</p>
</blockquote>

<p>These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, <em>they stop trying</em>.

NOTE: There's no closing </p> tag because I've cut the source at that spot.

Look how clean the original is! Why can't we just get that? Chrome's CSS injections and line flattening makes for a much less human readable snippet.

Edge is close, but still weird. Take another look. For some reason, Edge's fragment makes every tag uppercase like it's 1998. It also trashes Gruber's original (and thoughtful) whitespace. Also yuck. I mean, Gruber's using Markdown to write Daring Fireball. In a perfect world, I'd take that clipboard's html and turn it back into Markdown when it's pasted into MarkUpDown.

How do we get this original source?

Start and End Fragment

The important thing to catch is that the HTML Fragment format almost always includes a full html doc with html and body tags after the SourceURL and other metadata. Within that html code, there are markers, <!--StartFragment--> and <!--EndFragment-->, saying where the exact selection started and stopped.

Though Chrome immediately dives into your selected text (the "true" html fragment) immediately after the <body> tag, Edge gives you any open tags that come before your fragment too. In this case, you had three open div tags when you got to the highlighted selection: <DIV id="Box"><DIV id="Main"><DIV class="article">. Knowing this DOM could be really helpful if you were interested in retrieving the original CSS and formatting your fragment as close to the original markup as possible.

If you ignore context like this, though, it's an easy enough feat to take the string and parse out the "true fragment's" html by splitting the entire HTML Fragment string on <!--StartFragment--> and <!--EndFragment-->. Here's an early version of something I wrote to just take out the html, ignoring all the fragment metadata.

private string _parseHtmlClipboardFragment(string rawFragmentSource)
{
    string ret = rawFragmentSource;
    string delimiterStartAfter = "<!--StartFragment-->";
    string delimiterEndBefore = "<!--EndFragment-->";

    if (-1 < ret.IndexOf(delimiterStartAfter))
    {
        ret = ret.Substring(ret.IndexOf(delimiterStartAfter) + delimiterStartAfter.Length);
        if (-1 < ret.IndexOf(delimiterEndBefore))
        {
            ret = ret.Substring(0, ret.IndexOf(delimiterEndBefore));
        }
        else
        {
            ret = string.Empty;  // No luck, Ending not after Start; go back to nothing.
        }
    }

    return ret;
}

NOTE: I've got a nice library that parses all this up into a custom html fragment view model that I mentioned in TL;DR. I wouldn't and don't use the above code in most cases.

When we parse out that html and insert it into our Markdown from Chrome, however, we get some, um, interesting results.

NOTE: Remember that Markdown is a superset, so to speak, of html. You could have a "Markdown" file without any Markdown syntax that's pure html. So what I'm really saying is, "Let's see what those html snippets that we pulled from the fragment look like when they're injected into our regularly scheduled html."

I'm going to wrap these pastes in blockquote tags so that you can tell where they start and stop easily.

Edge Parsed Code

Mossberg:

For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.

Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.

These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, they stop trying.

Though you should remember that the original source has been smeared together, losing its newlines, and all the tags are strangely capitalized, the source looks pretty nice. It's plain, and adopts the local styling for the most part, but the most important markup is still there -- the blockquote, the italics, links if they'd been included, etc.

Chrome Parsed Code

Mossberg:

For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.

Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.

These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, they stop trying.

Um, ew. Wha' happen'd?

Unfortunately, even though Daring Fireball's entire page has the same styles applied, this inline kludge Chrome performs puts it only in each, in this case, <p> tag. So when we have a margins around paragraphs, there's nothing telling us to also put that color and style between the block-level tags. There's no surrounding tag to be a stand-in for, essentially, the body (or any other lost, open) tag.

Let's look at all the overhead we have for each p from that fragment:

<p style="
    margin: 0px 0px 1.6em;
    padding: 0px;
    color: rgb(238, 238, 238);
    font-family: Verdana, &quot;Bitstream Vera Sans&quot;, sans-serif;
    font-size: 11px;
    font-style: normal;
    font-variant-ligatures: normal;
    font-variant-caps: normal;
    font-weight: normal;
    letter-spacing: normal;
    orphans: 2;
    text-align: left;
    text-indent: 0px;
    text-transform: none;
    white-space: normal;
    widows: 2;
    word-spacing: 0px;
    -webkit-text-stroke-width: 0px;
    background-color: rgb(74, 82, 90);
">

And here's what we have from Chrome's Dev Tools' "Computed" style tab:

color: rgb(238, 238, 238);
display: block;
font-family: Verdana, "Bitstream Vera Sans", sans-serif;
font-size: 11px;
height: 19px;
line-height: 19.8px;
margin-bottom: 17.6px;
margin-left: 0px;
margin-right: 0px;
margin-top: 0px;
padding-bottom: 0px;
padding-left: 0px;
padding-right: 0px;
padding-top: 0px;
text-align: left;
text-size-adjust: 100%;
width: 425px;
-webkit-margin-after: 17.6px;
-webkit-margin-before: 0px;
-webkit-margin-end: 0px;
-webkit-margin-start: 0px;

But note that that's essentially all from one css file, as properties from user agent stylesheets are simply the browser's defaults:

If you'd like, we can compare the fragment's CSS vs. the stylesheet. Spoiler: It's a mess. Not a lot of matching. Click here to see. Ugly.

The bottom line is that background-color: rgb(74, 82, 90); is something that belongs to the body, but is inserted here by Chrome into its HTML Clipboard contents as something that's attached to each block element. That's wrong. It doesn't display correctly, and it needlessly clutters the resulting HTML.

With HtmlFragmentHelper, I have a couple of choices. One is to strip all inline styles out, or at least some subset of styling. If I want to maintain as much of the original as possible, I might just blast anything that injects colors into the html, like color and background-color. Or I could blast all the inline style info and get something more like Edge's code.

Or I could normalize the CSS and optionally (via a property on HtmlFragmentViewModel) wrap it all in a block-level tag that contains CSS that's everywhere. Or at least in all of the top-level tags. But now I'm starting to write an html parser, which is a little outside of my current scope.

I think I'm going to do the former, though it's not in there as of this writing. That Chrome paste looks horrible as is, though. If only there was an easier way...

HTML Fragment from Firefox

There are certainly other applications that create HTML Clipboard values. I ran into LibreOffice's when using Calc, their spreadsheet app, and its html clipboard doesn't include Start and EndFragment tags. Wonderful, folks, wonderful.

But isn't there one glaring omission I might want to add here? You know, I think there is at least one...

For fun, let's take a look at Firefox's fragment too. Firefox isn't as popular as it used to be, but it's probably worth taking a look. (Remember that this is all on Windows, and Safari is dead enough by now that I'm not going to bother with v5.)

Version:0.9
StartHTML:00000156
EndHTML:00001060
StartFragment:00000190
EndFragment:00001024
SourceURL:http://daringfireball.net/2016/10/mossberg_siri
<html><body>
<!--StartFragment--><p>Mossberg:</p>

<blockquote>
  <p>For instance, when I asked Siri on my Mac how long it would take
me to get to work, it said it didn’t have my work address —
even though the “me” contact card contains a work address and
the same synced contact card on my iPhone allowed Siri to give
me an answer.</p>

<p>Similarly, on my iPad, when I asked what my next appointment was,
it said “Sorry, Walt, something’s wrong” — repeatedly, with
slightly different wording, in multiple places on multiple days.
But, using the same Apple calendar and data, Siri answered
correctly on the iPhone.</p>
</blockquote>

<p>These sort of glaring inconsistencies are almost as bad as universal 
failures. The big problem Apple faces with Siri is that when people 
encounter these problems, <em>they stop trying</em>.</p><!--EndFragment-->
</body>
</html>

Oh, so beautiful. Same whitespace as the original, with the added bonus (?) that the final paragraph is also text wrapped. No crazy inline CSS attempted. No header overhead or 1998-style HTML TAGS.

You could argue that Chrome is better because it has that inline CSS (yuck!) or Edge because it has the full page headers, but honestly, we're only one step farther away from what Edge gave us. We have the URL here. If we have network access to get CSS, we have network access to read the header and figure out where the CSS lives ourselves. Don't overcomplicate things. Beautiful. Good looking code continues to look good when pasted inline with Markdown.

Firefox Parsed Code

And Firefox's snippet is just as nice when pasted.

Mossberg:

For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.

Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.

These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, they stop trying.

I miss Firefox.

Labels: c#, howto, html, long, markdown, markupdown, microsoft, tutorial

I think I was reading about CommonMark when I saw this mentioned.

Sad because true?

Honestly, I think CommonMark is probably a pretty good idea, all things considered. There are people that are going to get concerned with a "spec" that's, as CommonMark puts it, "ambiguous". You don't want strange edge cases that seem to conflict with reasonably hard and fast rules. I think the best complaint I've seen (and I can't dig the source back up in 10 mins of googling) is the "underline for headers" issue.

What do you do here?

This is a header. Or is it the last thing in a section before a horizontal line?
--------------------------------------------------------------------------------
I'm not sure. You tell me.

Apparently the original spec didn't address many edge cases like this, and you'd have to go to the code. I can't recall why that code wasn't good enough, other than folks occasionally disagreeing with it. And different users do have different needs, so I think that's a fair complaint.

And most importantly for coders, CommonMark gives you a test suite. We like to have explicit pass/fail conditions. You're only as good as your metrics and QA.

But yeah, any time somebody's providing something called Babelmark to help you see what's going on in different flavors of a markup format with unresolved edge cases, you've got a fragmentation problem.

Also worth mentioning: Is HTML a Humane Markup Language? from Atwood. Probably as good a visualization of why we might prefer Markdown to some other markup language that's got some traction right now as I've seen, though Atwood here (contra Atwood now, afaict) strangely decides that HTML is easier to eyeball than Markdown.

In other news, I had this choice definition of POJO open from Martin Fowler's site. I was pretty confident when I read "Anemic Domain Model" that he'd meant Plain Old Java Object, but the backstory is both hilarious and tragic.

POJO An acronym for: Plain Old Java Object.

The term was coined while Rebecca Parsons, Josh MacKenzie and I were preparing for a talk at a conference in September 2000. In the talk we were pointing out the many benefits of encoding business logic into regular java objects rather than using Entity Beans. We wondered why people were so against using regular objects in their systems and concluded that it was because simple objects lacked a fancy name. So we gave them one, and it's caught on very nicely.

(ー_ー﹡)

Labels: coding, fowler, markdown, standards

When you're writing a Markdown editor, you have to be worried about Markdown formatting in two distinct, and not always especially related, ways.

The first is that you have to format the Markdown into html markup correctly. That's a no-brainer requirement, and there are plenty of libraries to help you convert raw Markdown to html. That said, there are extensions to Markdown that sometimes require quite a bit more discussion than you'd expect. If your favorite Markdown parsing library doesn't include these extensions, do you write them yourself? Change parsers? Etc, etc. Typical library evaluation stuff.

The second consideration, however, is how you make your users' Markdown input more graceful.

Take the following example. Say I've got this text...

I was reading the following snippet on Mozillazine.org about the history of replying about and below quoted emails:|

... and I have the cursor exactly at the "pipe" character -- | -- what should happen if I pasted in my clipboard's contents as a quotation (Ctrl-Shift-V in my editor)?

There are a number of choices. The pasted quotation could read...

I was reading the following snippet on [Mozillazine.org](http://kb.mozillazine.org/Reply_above_quoted_message) 
about the history of replying about and below quoted emails:
> Traditional netiquette (especially on Usenet and mailing lists) is to reply contextually beneath quoted material, trimming down the quotes to the minimum needed to establish context. Also known as inline, interspersed and interleaved posting, this does not always put your entire reply beneath all of the quoted material, because if you are replying to multiple paragraphs, each section of the reply would be beneath the quote of the paragraph to which it is responding. Some people prefer to start their cursor at the top of the message in order to facilitate moving down through the message and trimming the quotes before typing replies contextually.|

Or they might prefer another newline before the quote so that you had the quoted material nicely set off...

I was reading the following snippet on [Mozillazine.org](http://kb.mozillazine.org/Reply_above_quoted_message) 
about the history of replying about and below quoted emails:

> Traditional netiquette (especially on Usenet and mailing lists) is to reply contextually beneath quoted material, trimming down the quotes to the minimum needed to establish context. Also known as inline, interspersed and interleaved posting, this does not always put your entire reply beneath all of the quoted material, because if you are replying to multiple paragraphs, each section of the reply would be beneath the quote of the paragraph to which it is responding. Some people prefer to start their cursor at the top of the message in order to facilitate moving down through the message and trimming the quotes before typing replies contextually.|

Or they might like to have that quote wrapped so that it's not one giant ugly line with a single greater than (>) in front of it ^[1]...

I was reading the following snippet on 
[Mozillazine.org](http://kb.mozillazine.org/Reply_above_quoted_message) 
about the history of replying about and below quoted emails:  

> Traditional netiquette (especially on Usenet and mailing lists) is to 
> reply contextually beneath quoted material, trimming down the quotes to 
> the minimum needed to establish context. Also known as inline, 
> interspersed and interleaved posting, this does not always put your 
> entire reply beneath all of the quoted material, because if you are 
> replying to multiple paragraphs, each section of the reply would be 
> beneath the quote of the paragraph to which it is responding. Some 
> people prefer to start their cursor at the top of the message in order 
> to facilitate moving down through the message and trimming the quotes 
> before typing replies contextually.|

(Or they might like wrapping but only one newline (so no full empty lines) between the quote setup and the quote itself...)

Curiously, those all render to the same html (which I'm setting off with an extra blockquote, below)...

I was reading the following snippet on Mozillazine.org about the history of replying about and below quoted emails:

Traditional netiquette (especially on Usenet and mailing lists) is to reply contextually beneath quoted material, trimming down the quotes to the minimum needed to establish context. Also known as inline, interspersed and interleaved posting, this does not always put your entire reply beneath all of the quoted material, because if you are replying to multiple paragraphs, each section of the reply would be beneath the quote of the paragraph to which it is responding. Some people prefer to start their cursor at the top of the message in order to facilitate moving down through the message and trimming the quotes before typing replies contextually.|

When you're trying to minimize application preferences, you really really want one of these options to be the a priori best.^[2] Because otherwise you're selecting one for all of your users and knowing everyone who prefers another option enough to notice is going to, if you're lucky, let you know about it.

^[1] To the point someone might've written an insanely ugly app to do just that for text-unfriendly Outlook.
^[2] And if you're paying attention, yes, once you embed your Markdown editor into an email client, you have to worry about where you put the quoted material too. Argh. Fun times. Remember -- Just Fn Ship, see how much purchase you get, and then head back to add preferences where users actually have a strong enough reaction to your choice that adding a preferences item is The Right Thing to do. Just too bad that I'm wired to spend so much time sweating which selection is correct for the mvp.

Labels: coding, features, indie, markdown, markupdown, style

Quick quote from High Performing Agile Teams: An Introduction on the Quality is Speed blog:

The parts of Agile I hate are the parts that ignore this essence. High performing teams are the goal. Not practices. Not user stories. Not estimation. Not Scrum. Not the manifesto. Not TDD. Too often the practices become the goal while we lose sight of the real goal: high performing teams.

Yes, thank you.

In other news...

Questions like this make me sad....please don't connect directly to your databases from your mobile apps, folks: https://t.co/FNKf8qDA18

— Greg Shackles (@gshackles) February 21, 2016

The question in question [sic] is bad news on a couple of fronts, but the one Shackles doesn't like is this bit:

SqlConnection con = new SqlConnection(@"server=MOON\SQLEXPRESS;"
    + "Initial Catalog=Moon;Integrated Security=True");

Ooops!

Had a similar thing happen to me recently creating the image upload service for my Markdown editor. It was surprisingly easy to set up the call to the API in C#, but it quickly occurred to me that it was also really easy for someone to Fiddler what I was sending, grab the API keys, and use my account to upload as much jive as they wanted.

I contacted Mashape to see if there wasn't some way to put a cap on my account, thinking I could just cross that bridge if I ever came to it. I mean, what's the point of limiting exploitation potential if nobody's going to buy the danged app to start with? (I'm an inveterate optimist, as you can see.) Then if I ever, say, hit $20 in a day (which would mean something like 21,250 image uploads in 24 hours), I'd quickly write an API, through it into the cloud, change the API keys, release a new build, and be done with a little egg on my face.

Well, Mashape never got back to me, so in an abundance of caution, I wrote the cloud service. Kinda a pain, atc. I eventually settled on a system of requesting an access key, creating a trivial hash that creates the key, giving the key a sunset time, then matching that key to upload requests. If someone hits more than X in period Y, they're shut down. If the whole system receives more than Z in Y, everything shuts down.

So I have my own rate limiter, and my API key, like the database connection in the bad SO question, is reasonably hidden. I mean, there's still a chance my cloud gets hacked and the code gets out, but that's a lot more difficult than grabbing Fiddler and sniffing API keys, even if they're SSL'd (which they weren't, initially).

Kinda painful how quickly a small feature addition because a pain, but at least I've got a nice cloud-based API presence now. On the down-side, I'm back to thinking I need to figure out how to move my entire website over to Linode eventually, as it's sooooo much cheaper to admin it all yourself.

PS -- How in the world did Redis get so popular for small-time hosted app services? I mean, if you have a distributed system, I get it, but if you want an in-memory key-value store for a single box, why would you bother? Just keep a dictionary in memory, right?

Labels: api, coding, images, markdown, style

Particularly useful for StackOverflow.

Labels: html, markdown, noteToSelf

And now, the reasoning behind the line breaks in Markdown, from the horse's mouth:

The implication of the "one or more consecutive lines of text" rule is that Markdown supports "hard-wrapped" text paragraphs. This differs significantly from most other text-to-HTML formatters (including Movable Type's "Convert Line Breaks" option) which translate every line break character in a paragraph into a <br /> tag.

When you do want to insert a <br /> break tag using Markdown, you end a line with two or more spaces, then type return.

Yes, this takes a tad more effort to create a <br />, but a simplistic "every line break is a
" rule wouldn't work for Markdown. Markdown's email-style blockquoting and multi-paragraph list items work best — and look better — when you format them with hard breaks.

I feel kinda like I'm arguing for JSHint over JSLint at this point (I WANT OPTIONS!), but this isn't code, it's composing content. (It's a distinction for me, anyhow. ;^D)

Wonder what the "hard-wrapped" use case is? Pastes from emails? Hard wraps in BBEdit? (That second one seems unlikely.) Editing content that already contains hard wraps always stinks. I can't imagine Gruber's doing that.

Labels: markdown

To force a line return, place two empty spaces at the end of a line.

I guess that's better than nothing, but it seems like br's for every return would be a common enough request to be an option in most editors.

Labels: markdown