From github.com:

Important update

On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.

This is an exceptionally important announcement, though it's one I've been predicting for a while.

StackOverflow is dead, and the bots sucked up all of its knowledge, and GitHub repos' knowledge, and MSDN's knowledge, and...

And we wondered how AI would continue progressing if there was no reason to keep posting answers to the net "for free".

But it was obvious. Now each AI engine is storing (will soon be storing?) the answer to each question in its own StackOverflow, so to speak, but one tuned for AI, not humans. Every time you give Copilot the thumbs up or positive feedback (or some other way it figures you likely used its code), it's going to file that away as "The Right Answer for You". You have your own Jon Skeet in your workstation in exchange for your answers being in their own paywalled database. (Not that there's anything inherently wrong with wanting to make a buck.)

If all the best programmers use Claude going forward, guess where the best answers are going to come from?

Or, as it looks like Copilot wants to do, what if all the best answers are stored in Copilot's corpus? Then regardless of which model you use, the best answers (and autogenerated code) to the questions (and prompts that implicitly ask those questions) that used to go to StackOverflow will come from Copilot.

That's what's next, folk. Actually, that's what's now. All the knowledge that you helped build if you leave that setting on its default will be behind a paywall.

There's a great podcast with Nilay Patel and the CEO of Grammarly where Patel essentially asks: "Okay, [very not] cool, you're using my name and ostensibly my style. How much are you going to pay me for that?"

(Though I suppose the opposite is, "You're happily paying $10/mth for Copilot now. You think you could even sniff that magic without contributing your labor? You'd still be posting to StackOverflow, hoping someone would be kind enough to post something useful back, which used to be amazingly common, but when's the last time that's happened for you?" It's an interesting counterpoint.

Perhaps we should unionize before they alter the deal any further, like making it $75/mth. Or $175. Or more. Because they could.)

Every so often go out to the middle of the woods, turn off WiFi and cellular, and make sure you can still code. And then realize that's not really your job any more.

Labels: ai, business, coding, labor, stackoverflow

Though so far I hate all the forced unwrapping .NET requires with nullables now -- which was stolen directly from Swift -- this comment on a Skeet answer explaining them a bit, authored by someone you should recognize if you can spell C#, includes one of the most important concepts I've been trying to PR into coworkers for quite a while. Not sure I've ever said it this eloquently, though, so...

Here's that key point from Eric Lippert (emphasis mine):

• The key thing that I learned when studying the Coverity checker is that code is evidence of the beliefs of its authors. When we see a null check that should inform us that the authors of the code believed the check was necessary. The checker is actually looking for evidence that the authors' beliefs were inconsistent because it is the places where we see inconsistent beliefs about, say, nullity, that bugs happen. 

  – Eric Lippert Dec 13, 2019 at 13:34

(This might be that Coverity stuff he's talking about. Not sure.)

One place I run into disconnects between code and intent routinely is with Jasmine tests. If tests essentially share the exact same code across Arrange sections, it should be in a shared describe block with a beforeEach DRYing up all that setup. If you only change the arrange on tests by something trivial, those tests too should share an beforeEach block with just the differences in the individual tests' Arrange sections. Otherwise I'm trolling through each arrange block trying to figure out what's different.

(Actually I expect a method even for the slightly different ones -- especially for the slightly different ones, because then you can have parameters communicating what's different in the calls out. Make sense? It's all about the DRYing, folks.)

Tell me what the tests do by grouping same and different, keeping things DRY so I can get familiar with concepts you're using over and over -- in ways I can't when you just cut and paste code all over the place.

Put more succinctly, I hate cut and pasted code because I expect code blocks that seem similar to be slightly different. Otherwise why not DRY them? Because unDRYed code means we've got some code blocks that are exactly the same mixed with some that aren't with no easy, scannable way to tell which is which.

Another domain that suffers is checking for strict equality against undefined or another falsy value in JavaScript. If I see if (myVar === undefined) that tells me null and 0 and any other falsy value is okay and different from undefined. Even if the value should be undefined at this point, if it's possible non-undefined value is, say, an object literal, then there's no reason not to also exclude anything falsy.

That is, unless it makes the code worse, follow conventions! When we're in JavaScript land, because it's a dynamic language, I always assume we prefer operating with a truthy/falsy mindset. If you're comparing to a specific value (or !!ing a Boolean to cast it to a strongly typed bool), I assume you have a good reason to use an anti-pattern, like with myVar || myVar === 0 (though consider parseInt(myVar, 10) (or float) instead?). If I'm wading through code to see a null for an object literal would also be bad news, I've wasted time. You didn't write what you meant.

If your anti-patterns aren't "evidence of your beliefs", we're doing it wrong! And if we don't share patterns, we should talk about it until we do -- or at least understand each other's dialects.

/rant

Labels: coding, style

You should not have a giant, monolithic application. Monolithic codebases are the mind-killers. They are the little-deaths that bring total obliteration.

Face your monolith. Allow it to pass over you and through you...

... into a well-modularized codebase.

But here's the corollary nobody links to this practice: 

The best modularization requires smart denormalization and repetition. 

You catch some of this when you hear about microservices, which hold down the other end of the spectrum from monoliths. I think it's largely because of their association with NoSQL, where being comfortable with denormalizing schemas is absolutely essential. 

I don't want to get too deep into NoSQL right now, but the concept is the same: To be most nimble in a modularized codebase requires that you -- and it's nearly against my programming north star to admit it -- Repeat Yourself.

And that's okay. If you have one service that calls another, and you need to massage your payload, you may have to change it two places, the caller and the server.

THIS IS FINE, NO MEME-MIC DOG & FIRE SHOW REQUIRED.

Because you know what's worse than changing a model two places? Changing it three places.

1. The shared library
2. Updating the reference for the caller
3. Updating the reference for the server

Those last two are too often nontrivial. Perhaps they should be trivial, but in practice they almost never are. There's always some argument that 1. requires a breaking change to the shared library, which requires we up its major version, and 2. & 3. are set to update automatically on minor changes, but not major changes, and now we have to redeploy in just the right order so that the other apps that depend on the library in 1. aren't brought down when we... I see you over there. You get it. This has happened to you too.

What were we hoping to save ourselves from again? Exactly this sort of tedium. Except now it's even worse than it was before in the monolith!

Look, you should be coding the caller and server defensively anyhow, as independent services. The payloads for both, once serialized, are probably coming across the wire as XML or JSON (please JSON) anyhow, so it's not like these model abstractions are necessary beasts. Why do we have to point them both to the same library to have a source of truth? We're comfortable consuming third party apps without sharing model definitions, and that's [at least partially] the setup you chose when you went to modular services.

Treat each app as an independent development and change the model twice -- or, when appropriate, add a new model to one with a new API endpoint to maintain backwards compatibility with other consumers. Denormalization between apps at the point of the interface is a-ok. 

I'm requesting foo. You're sending foo. Having two copies of foo, one for me and one for you, is fine. Even having a slightly different foo for each is often fine as long as the middle of the Venn is what the transaction in question is worried about.

No, for real. It's okay. Really. You'll thank me later.

Labels: advice, coding, nosql

Here's my reply, edited a little, to a recent code review where I got taken to the woodshed for using the underscore prefix for private variables in es5 JavaScript code used in an AngularJS project.

Stick around until the end. Spoiler: A, um, widely recognized AngularJS style guide author agrees with me.

The john papa style guide allows them.

"disallowDanglingUnderscores": null,

An underscore prefixed to a variable means the items are intended purely for internal use. They should not be directly exposed.

Here's a good sum from a StackOverflow answer on the underscore convention:

[An underscore prefix] means private fields or private methods. Methods that are only for internal use.

They should not be invoked outside of the class.

Private fields contain data for internal use.

They should not be read or written into (directly) from outside of the class.

Note: It is very important to note that just adding an underscore to a variable does not make it private, it is only a naming convention.

That's an important distinction for me. If I have side-effects for specific calls that I don't want to expose to a consumer, I want to ensure those things are "hidden" (not exposed outside of the file's scope). This helps me both declare and double-check that.

I don't have to do it, but it's not random and it's not a made up practice.

Your TypeScript reference is Google's (as in not Microsoft's). Nothing inherently wrong with that, but, um, why?

Though I disagree with some of the stuff already (there's nothing wrong with interfaces starting with I. Fight me ;^D), I can see why you wouldn't use _ in TypeScript -- TypeScript has a concept of and enforces that concept of private fields.

https://www.typescriptlang.org/docs/handbook/classes.html#ecmascript-private-fields

For example, if I want _myProp to be private, I just say so: private: myProp. That's a little different than my usage here, but you can twist it into the same thing.

Make sense? My point here isn't that I can't remove the _ if it's driving the team crazy. The point is that there is a clear reason to use it, and it is a popular, useful convention.

---

Fun update: John Papa from 2016 says exactly the same thing as me:

johnpapa commented on Mar 28, 2016 •

I don't like underscores personally ... but since there is no real private nor public in javascript, it is very helpful to have a way to differentiate them. So I use them

UPDATE: i was referring to ES5 ... with Typescript I don;t use underscore

I'm biased, but well said.

Though, again, this is why I hate working in dated codebases. The code doesn't rust, but it kinda feels like your career is stuck in an episode of Doctor Who.

Labels: coding, javascript, style

From here:

That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. It is hard to believe today, but under this embarrassing mess lies the ruins of the beautiful cathedral of Unix, deservedly famous for its simplicity of design, its economy of features, and its elegance of execution.

Labels: coding

From an introduction to Swift I find myself reading today:

But what if we forget the verification like below?

var stockCode:String? = findStockCode("Facebook")
let text = "Stock Code - "
let message = text + stockCode!  // runtime error

There will be no compile-time error. The compiler assumes that the optional contains a value as forced unwrapping is used. When you run the app, a runtime error is thrown with the following message:

fatal error: Can’t unwrap Optional.None

Huh?!! Then what’s the point?

We’re far enough along in 2018 that a language could have come with forced linting. The compile-time error should be, “Use of a force unwrap on an optional without a nil check is not allowed.”

I’ll admit I’m glad Swift exists. I thought it’d be fun to research SpriteKit and hack up a quick 2D game to relax, and Objective-C looks a little bit more of a head rethread than my usual [stolen] line that “all programming languages are just dialects of the same language”.

But it’s a pain that Swift’s let means the opposite of what it does, so to speak, in JavaScript, and this “Hey, we force one level of null checks in the language, but then reintroduce the ability to have run-time null errors just like you had before,” doesn’t seem really well thought out.

Linters ftw, imo.

Labels: apple, coding, swift

From Erich Reich at qmo.io, talking about our propensity to include too many libraries when starting up a JavaScript project:

Do you have any dates in your project? Why not throw in MomentJs? Do you deal with arrays? Throw in something like lodash or Ramda. What are you dumb? Use a linter for gawdsake! While you're at it, throw a few githooks in. I honestly don't know how you get out of bed in the morning if you aren't using Babel and Axios and a good CSS framework and all that other stuff you flat headed goof. Make sure to compile your transpiled isomorphic app. Even if it's just your Hello World app. Don't embarrass yourself. Perfection or nothing.

Ok, I've got that out of my system... I'm good now. But seriously, there are way too many devs who get caught up with doing everything all the time. Just because the herd is going one way, that shouldn't make it a foregone conclusion.

Flashbacks to the 15,683 files create-react-app creates...

This is why I'm writing that quick series on Templating without Transpilation with VueJS. I'm not saying it's The Right Way to create a client-side stack, but it's a thought experiment worth thinking through -- like Reich's, above, or even Dan Abramov's famous-ish "You Might Not Need Redux.

Labels: coding, javascript, style

From martinfowler.com on "Command Query Responsibility Segregation", or CQRS:

The mainstream approach people use for interacting with an information system is to treat it as a CRUD datastore. By this I mean that we have mental model of some record structure where we can create new records, read records, update existing records, and delete records when we're done with them. In the simplest case, our interactions are all about storing and retrieving these records.

As our needs become more sophisticated we steadily move away from that model. We may want to look at the information in a different way to the record store, perhaps collapsing multiple records into one, or forming virtual records by combining information for different places. On the update side we may find validation rules that only allow certain combinations of data to be stored, or may even infer data to be stored that's different from that we provide.

As this occurs we begin to see multiple representations of information. When users interact with the information they use various presentations of this information, each of which is a different representation...

This structure of multiple layers of representation can get quite complicated, but when people do this they still resolve it down to a single conceptual representation which acts as a conceptual integration point between all the presentations.

The change that CQRS introduces is to split that conceptual model into separate models for update and display, which it refers to as Command and Query respectively following the vocabulary of CommandQuerySeparation. The rationale is that for many problems, particularly in more complicated domains, having the same conceptual model for commands and queries leads to a more complex model that does neither well.

I saw this acronym for the first time today in a SO question about Azure Service Fabric, and my HOT TAKE!!11! was, "Who in the world doesn't do this?"

There are two main data transformations that happen in essentially all of my controller code. For GETs, I'm usually flattening a "database is truth" model into a data payload, stripped of stuff like, "LastModifiedBy" (or anything not needed by the client), but also of complex relationships with child objects. That is, when Entity Framework (or any POCO-returning library) gives me, say, an address' stateOrProvince as an entity, I usually flatten that to a StateName and StateId at the top level of a client-ready DTO for Addresses (or at the Address level on a collection of Addresses attached to a client).

The second is when I return changes to these DTOs. There, the PUT (or PATCH) command often isn't strictly RESTful either [using the database as The Source of Truth]. If I'm returning the ever-proverbial Contact with three Addresses, I don't typically call an Addresses endpoint with a PUT thrice, and then follow up with another PUT to Contacts. I'll send up a DTO that's essentially CompositeContactWithAddressesAndOtherJive (but with a better name, like ContactDTO). And though if I'm trying to keep our stack small in cases where team familiarity with SQL isn't a strong suit, I will, occasionally, still hit EF repository collections, I usually prefer to write the INSERT/UPDATE SQL by hand and perform what needs doing in as few optimized calls as possible. The move away from the ORM for PUTs hurts if you ever want to swap out your database engine, of course, but who swaps out their database engine? Talk about premature optimizations. Why should nearly every PUT suffer when you're likely hiring for SQL proficiency? For a clean object model? Stop it. Use your team's skills.

There is nothing that, in my experience, causes performance bottlenecks as quickly as folks blindly falling back on performing Commands (actions that "Change the state of a system but do not return a value") via fully hydrated ORM objects. Oh, the humanity.

Treating your PUTs and PATCHes as separate models from your GETs (or at least separate models from those in your database schema) allows you to spare yourself such madness.

Labels: coding, CRUD, fowler, style

If your architect hasn't written a tutorial -- not a cursory howto, but a tutorial -- you don't have an architect.

What does a tutorial look like? The canonical example for C# web apps, though now a bit long in the tooth, is NerdDinner:

NerdDinner Tutorial

The best way to learn a new framework is to build something with it. This tutorial walks through how to build a small, but complete, application using ASP.NET MVC, and introduces some of the core concepts behind it.

The application we are going to build is called "NerdDinner". NerdDinner provides an easy way for people to find and organize dinners online...

There's not only a nicely written walkthrough, but documented code and a live, running example.

If your architect hasn't taken the time to articulate their vision thoroughly, in plain, if expert-specific, [language of your locale], there's not much hope for your coders. Your codebase will necessarily be an unmaintainable cyborg (a topic I'm [not] surprised to find I've been ranting about since 2002).

The best job security is none. You've set your stage so well any competent programmer could come in and take over, getting up to speed incredibly quickly.

Those coders who work to ensure they're the most easily replaced are absolutely irreplaceable.

Labels: architecture, coding, cyborg, style

Jay Bazuzi was talking about how to make the work of distributed teams run more smoothly, and there was a term he used that I thought deserved a little googling...

There are a bunch of known good practices to help distributed teams not suck too much. Doing them won't get us to "awesome", but at least we can get up to "not sucky". So let's start by writing down these practices:
...

• Recognize the expression of Conway's Law: limited communications affect software architecture

As it turns out, Conway's Law is pretty interesting. Here's some more, from Demystifying Conway's Law on ThoughtWorks:

In "Exploring the Duality between Product and Organizational Architectures", a study by The Harvard Business School carried out an analysis of different codebases to see if they could prove Conway’s original hypothesis as applied to software systems. In it, they took multiple examples of software created to solve the same purpose (for example word processing, financial management and database software), and compared the code bases created by loosely-coupled open source teams, and those created by tightly-coupled teams. Their study found that the often co-located, focused product teams created software that tended more towards tightly-coupled, monolithic codebases. Whereas the open source projects resulted in more modular, decomposed code bases. [emph mine -mfn]

The real take-home comes just a little later, though, I think:

You may see tensions in your own organizations where your structure and software are not in alignment. You may have seen for example the challenges involved where distributed teams try and work on the same monolithic codebase. [emph me again -mfn]

That the work of distributed teams -- or, if you ask me, any team -- can be coordinated more easily when you take the time to clearly define the interface surrounding their work seems like a truism.

Though I'd never given the converse enough thought. If you have a widely distributed [read: "entirely remote"] team that doesn't tend to communicate, having a monolithic codebase, especially one without documentation, can absolutely hamstring development. Or, at best, it means that your team will tend to ignore what exists and replace with whatever seems best.

If you start eating away at the monolith in good, interfaced, discrete chunks, however, that's probably about as good as means of erasing it as you can get.

Regardless, I think the lesson here is that the downfalls of poorly documented, monolithicly designed codebases are compounded in easy to identify and productivity unfriendly ways in distributed/remote work environments.

Say that three times quickly. Even better: Remote environments require well factored code.

Labels: coding, monolithic, productivity, remote, style

From silvrback.com (via Michael Tsai):

For a while, I’ve been using a different definition of Technical Debt. It helps teams frame their work in a way that highlights their choices and it can lead to better ones.

 

Technical Debt is the refactoring effort needed to add a feature non-invasively

Yes.

But with their example -- granting it's a simplest case, "staged house" example -- no. No. Heavens no.

While they admit it a bit...

It’s easy to challenge this example. It borders on over-engineering.

Yes. It doesn't just border, it's over the line.

Labels: coding, style

From The Single Page Interface Manifesto:

The Manifesto in other languages

Spanish

Note: these translations may be slightly out of date because this manifesto is "alive".

Ukranian thanks to Mario Pozner

Russian thanks to Andrey Geonya

Serbo-Croatian thanks to Jovana Milutinovich

Slovakian translation thanks to Knowledge Team

German thanks to Valeria Aleksandrova.

Romanian translation provided by Science Team.

Macedonian thanks to Katerina Nestiv

Hungarian thanks to Elana Pavlet

That's an interesting group of languages. I wonder if it isn't fairly representative of at least where operative internet architectures were being developed when it was written. There seems to be an exceptionally skilled contingent of programmers in Eastern Europe. I wonder what they're doing right, and how hard it'd be to do elsewhere.

Labels: coding, geography, style

If there's one often-overlooked feature in Windows that I really enjoy using, it's the "Cascade all windows" feature for the taskbar.

I've been told -- unprovoked, so you know it's bad -- that I tend to have a lot of windows open when I'm working. It's true. I do. And when it's time to clean them all up, it's nice not to have to declare "windows management bankruptcy" and close everything I have open in an app before closing it out.

To dodge this, you can hold down shift as you right-click a taskbar icon, and select the option from this menu:

Well, often you can do this and quickly go through whatever the app has open. What Chrome does is less than particularly helpful.

It's hard to see there, but you can't read any window titles other than the very first one. You can click the image to make it full-sized. What's displayed sort of gives you an idea of how much you're going to have to sift though, but it's otherwise not insanely helpful.

Here it is, up close.

I can tell my order has shipped in the front window, and how many tabs I have open in the other five, but that's it. Done.

Look at what Edge does.

It's a little hard to see the details in the picture, but you can get a feeling of how absolutely beautiful (and by "beautiful", like any good engineer, I mean "practical" and "useful") that is. I can see every window, every tab, every title. Now I can quickly hit the X at the top right to close any window I'm done using & that needs to go away.

Here's what Edge shows when "cascaded", zoomed in.

Being a good resident

Edge gets Windows. You might say you're not surprised, but what is surprising is how badly Chrome flubs it. I mean, I understand the, "They even use Material Design on iOS, man!" argument, but I'm not buying that doing so requires that you be a bad Windows resident.

Edge looks distinctly different from Internet Explorer, for instance. You've got plenty of leeway before you lose your design language, so to speak. And even though Apple, for instance, completely ignored all the good Windows resident requests with iTunes and WinSafari, two wrongs don't make a right.

In short, it's not that hard to push the titles up into all of that dead white space in Chrome.

Why not use that for displaying real information, even if there wasn't the option to cascade windows? And since there is, well, let's just say that it's a horribly efficient, beautiful thing when the title bar/browser tabs are done right.

Labels: chrome, coding, design, edge, fail, style, windows

And now, for a little from the Flux intro docs:

We found that two-way data bindings led to cascading updates, where changing one object led to another object changing, which could also trigger more updates. As applications grew, these cascading updates made it very difficult to predict what would change as the result of one user interaction. When updates can only change data within a single round, the system as a whole becomes more predictable.

Chicken dinner.

Labels: client, coding, javascript, style

Today, I ran across a clickbaitily titled post called Just Say No to More End-to-End Tests posted at the Google Testing Blog, written back on April 22, 2015, detailing a fictional end-to-end (e2e) testing run. The scenario was very clearly fictionalized, but you can forgive most of it, as they're simply trying to illustrate a few points.

Here's their "analysis" of the test, which apparently took over a week to complete and had some serious errors with the testing apparatus as well.

What Went Well 

• Customer-impacting bugs were identified and fixed before they reached the customer.

What Went Wrong 

• The team completed their coding milestone a week late (and worked a lot of overtime). 
• Finding the root cause for a failing end-to-end test is painful and can take a long time. 
• Partner failures and lab failures ruined the test results on multiple days. 
• Many smaller bugs were hidden behind bigger bugs. 
• End-to-end tests were flaky at times. 
• Developers had to wait until the following day to know if a fix worked or not. 

...
Although end-to-end tests do a better job of simulating real user scenarios, this advantage quickly becomes outweighed by all the disadvantages of the end-to-end feedback loop:

[Criteria]	Unit	End-to-end
Fast	✓	X
Reliable	✓	X
Isolates Failures	✓	X
Simulates a Real User	X	✓

The author uses this cluster of an e2e test to argue for a preponderance of unit and integration tests...

Integration Tests

Unit tests do have one major disadvantage: even if the units work well in isolation, you do not know if they work well together. But even then, you do not necessarily need end-to-end tests. For that, you can use an integration test. An integration test takes a small group of units, often two units, and tests their behavior as a whole, verifying that they coherently work together.
...

Testing Pyramid

Even with both unit tests and integration tests, you probably still will want a small number of end-to-end tests to verify the system as a whole. To find the right balance between all three test types, the best visual aid to use is the testing pyramid. Here is a simplified version of the testing pyramid from the opening keynote of the 2014 Google Test Automation Conference:

...
As a good first guess, Google often suggests a 70/20/10 split: 70% unit tests, 20% integration tests, and 10% end-to-end tests. The exact mix will be different for each team, but in general, it should retain that pyramid shape.

That's a lot of info to digest.

Here's the problem: "Simulates a Real User" isn't a single point. Their chart really isn't a scorecard, though that's how it's presented.

My quick list of critiques:

1. How many jobs have given you permission to create good unit tests?
2. Of those jobs, how many programmers actually created worthwhile, real-world, success & fail case tests?
3. Of the two unit tests you have left from the filters of 1 & 2, how many jobs then also factored creating integration tests into your schedule?
4. Did you use TDD? Because otherwise the tests you made in 1-3 are crap. No, honestly. No Scottishness at all.

Look, more typical situations in my experience are that you either have lip-service unit tests [only] or that you have no testing at all.

Guess what's most valuable if you have to pull teeth to convince management that testing is important? In my experience, it's an "end to end" test. You'll get the most return on your testing resource dollar with smoke tests.

I do like to call it a smoke test, and I've had pretty good results using Selenium to automate a browser using C#. Which browser? Well, you get your biggest bang for the buck by just using one. It'd be great to test in IE, Firefox, Edge, Chrome, and even macOS & iOS Safari, but each time is a diminishing return. The first test in whatever browser is going to catch 75% of what's wrong.

I recommend using either 1.) The lowest common denominator for browser functionality, usually IE, or 2.) Whatever's the easiest to get to behave for your tests. Timing can be a pain in Selenium.

Recently, I've used Selenium's Chrome web driver. That's dangerous. Chrome seems the most fault tolerant of the browsers, and on a developer's phat box, you can add slow downs to browser incompatibilities as problems that are often hard to notice.

But one browser, going through an automated master set of user stories, will quickly ensure that the 80% of your website that's your real bread and butter works and wasn't screwed up by whatever whizbang gizmo your latest release just pushed out the door.

If there is an error, sure, it can be hard to immediately identify the exact cause, which the Testing Blog seems to think is the end of the world. And perhaps a unit test to cover each error you do discover and track down will be useful. Do that.

But, again, unless you're doing TDD, to ask developers to write useful unit tests ahead of time almost never covers what your users are actually going to do and doesn't prevent bugs your own developers and QA are going to catch anyway. To get useful unit tests, you're going to have to spend even more time getting developers to code review others' unit tests, and you barely had time to green light unit tests to begin with. (How do I know that? If you had plenty of time, you'd be using TDD.)

And if you've gotten so far down the line that you can't tell if an error is coming from the front or back end, as in the Testing Blog's worst-case scenario, well, you have worse problems.

My suggestion if you're losing days looking for bugs your e2e tests turn up, and more days with your test lab going down?

Forget the users; your development process is broken. It's time to start arguing for TDD.

Update: Interesting to look at the slide linked in the quote, above:

And the notes address what I'm saying exactly (and argue against it):

Automated API tests are the best bang for the buck wrt tests that are easy to write and stable and a reliable signal. Integration Tests are always valuable; but we need to ration the number of integration tests one writes as they can be more difficult to debug and have a high noise factor due to all the moving pieces Automated component testing allows one to test a particular component (server) in isolation. you mock everything else out and ensure this component behaves as intended

As evident; the cone needs to be inverted in a lot of teams and companies. There is too much focus on Automated UI testing. At Google; we realized that half a decade back and have been putting in the required effort in the Integration testing and unit testing layer.

Interesting. I'm going to guess Google's coders do a much better job writing tests. If you want to convince me, talk more about how tests are written, and less about how they're better than automated UI testing if you have no testing now.

But wow, look at some later slides:

Push on Amber

• Daily pushes to prod

• Stable top of tree

• Smarter regression testing

• Critical tests cannot be bypassed

Comments to that slide:

In an ideal world; we should push on green; but we live in a practical world. I want baby carriers with beer holders; but they don’t make them. so I improvise and use my denim back pocket as my beer holder.

Not great. I've heard what's important when coding is to know which bugs to ship, but which bugs in tests to ignore? What are we doing? Later on...

How tests have become a nightmare. due to flakiness. stability.
I remember when I joined a team about 7 years back; I was excited; to be in server world @ Google. had been on client stuff before. When I asked around folks told me they had 1000s of end2end automated tests. I was like w00t! Turns out only 100s of them were providing real enough signal. some were flaky; some were broken forever.

EXACTLY. I think the Testing Blog grossing oversimplified the take-home from this fellow's presentation. I'll job bombarding pixels now.

Labels: coding, Google, selenium, tdd, testing

Have you tried using Facebook's create-react-app?

Fifteen-thousand, six-hundred, eighty three files for over sixty megs of space, and I haven't added a single line of code. Wow, just the readme of how to use what you just installed is 57 kilobytes of light reading.

Shawn Wildermuth talked to a similar issue, here Angular, but with an eye on any large framework, on Yet Another Podcast with Jesse Liberty in August.

8:18:

The actual programming [Angular] day to day, I really enjoyed. The ceremony of setting it all up, I found painful.

11:12:

I'm open to being wrong... [but with CLI for PhoneGap, and now with Angular]... it's hard to understand what all the generated code is doing. And my bread and butter, the thing I care about, is I want to know what's going on. I don't want to just trust that all the scaffolded stuff is just going to work, because ultimately I'm going to have to debug that at some point.

I'm pretty much in the same camp, as you likely know. I think you'd have to talk me out of using a "logic-less" templating system like moustache or handlebars first.

I like the concept of React's virtual DOM, but when diffing DOM and updating selectively is a cheaper operation than your standard update paradigm, that's a pretty hefty code smell. When you have something like shouldComponentUpdate on top of virtual DOMs, you're in trouble. We're inching back to getting 80% of our performance up front, but then the last 20% costs more than owning it all in house at the start. Over 15k files, y'all.

Remember folks: If you're performing cascading calculations on the client, that's an architecture smell. Get that logic where it belongs, and don't use frameworks that encourage you to place it somewhere unwise.

Quick update 20161231: Check out how long it takes to expand and copy over a zipped copy of the unmodified create-react-app folder contents... (Okay, granted, this is my wildly underpowered Lenovo 100S, but still...)

Labels: coding, javascript, libs, react, style

From Tips for Writing Portable Node.js Code:

Bonus: Something that Breaks on Linux and Solaris!

Both Windows and, by default, OS X, use case-insensitive file systems. That means if you install a package named foo, any of require("foo") or require("FOO") or require("fOo") will work—on Windows and OS X. But then when you go to deploy your code, out of your development environment and into your Linux or Solaris production system, the latter two will not work! So it's a little thing, but make sure you always get your module and package name casing right. [emphasis mine -- mfn]

This is a problem when serving files by name with express' static middleware too. Argh. And check out this schmoe's problem, with a huge legacy php site migrating to Linux, from case insensitivity to case sensitive. Ouch. Can you imagine the tech debt there?

So what if you want to use Express to serve files from the filesystem on, I don't know, Ubuntu on linode [which uses ext4]? And what if you want folks to be able to access your privacy policy at both http://rufwork.com/Privacy/ and http://rufwork.com/privacy/?

Even worse, what if you told your app reviewer to try http://rufwork.com/privacy/ and that meant they had to reject your app submission now that you've changed servers? /facepalm

From MarkUpDown - Review Results:

Locations: mumduwp1.4.0.0x86x64_arm.appxbundle

Tested OSes: Windows.Universal

Tested Devices: Acer Iconia W700

Notes To Developer

The privacy policy link did not resolve to a functional webpage.

(>ლ)

I've never liked it when I've typed in a URL by hand and got a 404 because I had one or two letters in the wrong case. I mean, it makes sense that case sensitive file systems are easier to maintain. It's just an end-use-case pain when you're serving files.

But URL case in Edge is broken too... 🙄😱

And insult to injury? Microsoft Edge (Internet Explorer's replacement in Win10) "remembers" case when you retype a URL! What, wait? So I can't navigate to the "correctly cased" URL with Edge even if I want to -- and I wonder if I get the same reviewer(s) if this'll happen to them too.

(>ლ)

How to fix it

How do you fix case sensitivity for express so that it's never a problem with ext4? Well, you could make everything in your filesystem lowercase, and then just always run request.url through .toLowerCase(). That seems pretty nuclear. Slightly better is to try the URL as requested first, and then do this...

From expressjs.com:

When a file is not found, instead of sending a 404 response, this module will instead call next() to move on to the next middleware, allowing for stacking and fall-backs.

Still have to set the filesystem to a single case throughout, and then if you want to use mixed case in your URLs to make them easier to remember or what-have-you, you're creating a good deal of extra work for your server every time mixed case is used.

How bad is it to make your web server take a URL, not find it, then search the filesystem again for a lower cased version? It might be smarter/less work just to lowercase the string every time and ensure your filesystem is in sync. It has to be cheaper to lowercase the url every time than to hit express.static twice in 30+% of your requests, right? Kind of a pain when you're editing files and want to glance at human readable names, but I think that's the only other downside.

So I guess it looks like I get to go through and change the case of my website's files and folders to a single case. Right? Isn't that The Right Thing to do to approximate case insensitivity? Is it ever better for the end user to have a case sensitive http server? I don't think so. Seems unnecessarily processor churny and hacky to lower case the earth, but man, I hate case sensitivity for URLs.

That fishing metaphor that tells you I'm about to comprise my best intentions...

In any event, for now, I've decided to embarrassingly give the server a fish, I'm afraid.

if (req.originalUrl === "/privacy" || req.originalUrl === "/privacy/") {
    req.url = "/Privacy/";
}

That's probably worth a Unicode facepalm too.

I really enjoy using linode, though my needs are so small I might drop down to Digital Ocean's $5 a month plan if it's just as easy to administer. It's great having your own "box" on the net. So much more powerful than shared hosting. Feel like I was about a decade behind with hosts, but, boy, I'm caught up now. Wonder how long before I start using .NET Core on it...

Labels: case, coding, encoding, node, noteToSelf, rufwork

There's a repeated joke in the Rockford Files that everyone in or that has been in jail was innocent; just ask them. (Of course, we're supposed to assume Jim really is innocent, but I've leave the meta alone for now.)

Ever heard something similar with programmers? Have you ever met a programmer who won't tell you they're a superstar? Coders' self-proclaimed brilliance is like ex-cons' proclamations of innocence. Seriously, who doesn't think they're above average? The sea of programmers thinks itself an extended version of Lake Wobegon.

A more interesting discussion would be to ask yourself why every coder thinks that and who is encouraging it, but I'm leaving that alone for now too. ;^)

Labels: coder, coding

From daringfireball.net:

Mossberg:

For instance, when I asked Siri on my Mac how long it would take me to get to work, it said it didn’t have my work address — even though the “me” contact card contains a work address and the same synced contact card on my iPhone allowed Siri to give me an answer.

Similarly, on my iPad, when I asked what my next appointment was, it said “Sorry, Walt, something’s wrong” — repeatedly, with slightly different wording, in multiple places on multiple days. But, using the same Apple calendar and data, Siri answered correctly on the iPhone.

These sort of glaring inconsistencies are almost as bad as universal failures. The big problem Apple faces with Siri is that when people encounter these problems, they stop trying.

Exactly right. I've had it do the same thing when I ask Siri to give me directions home, usually because I want to know if there's enough traffic to go an alternate route, and I get the same, "Who are you?!" complaint. But if I open Maps, "Home" is usually the first location listed.

QA sucks at Apple. I'm no longer politely questioning it.

Spend ten million (he said figuratively) and get the best QA staff in the business, and make sure there's no silo making QAing app interactions an issue. If Maps borks like this, the QA team "for Maps" has to be able to hold Siri and Contacts (or whatever else) accountable. No software ships until this blocking bug is fixed.

Good QA will think of and try these sorts of pretty obvious, yet creative, use (not edge, simply "use") cases before you ship.

Pitiful. Seriously pitiful.

Labels: apple, applefail, coding, qa, quality, siri

Oh, Pivots in UWP, you so crazy.

You may have heard that I'm writing a Markdown editor, MarkUpDown. It can have several files open at once, of course, and I'm using a UWP Pivot control to handle navigation.

As I got closer to releasing, it hit me that Ctrl-W, though common, isn't the most discoverable way to close those PivotItems. And now that I'm maintaining state between starts, so that whatever tabs you have open when you close the app are back on restart, closing each file "on purpose" is a lot more important than when they all closed on each restart.

That is, it looks like I need a UI for closeable tabs.

Not as easily said as done. You can customize the Header of the PivotItem pretty easily. In fact, a Header can be any object. You create a custom UserControl, throw in your favorite UI widgets, and you're golden.

In that UserControl, I put a TextArea and an icon of some sort, here a box with an X in it donated from Material Design Icons... Took a little experimenting to figure out exactly how to put text and an icon into the header presentably, though. I initially used a ViewBox, which seemed to scale the Path style image I'm using fairly well, no resizing necessary, but the ViewBox hated responding to clicks (or, in this case, "Taps" and "Presses"). Apparently you can manually set something called HandledEventsToo to make it catch events that are already caught by something with higher priority, but that strange Boolean reminds me of CSS' !important a little too much.

I replaced the ViewBox with a Button to be conventional, and then placed the Path style icon in there. I've got decent click event handling now, though it was difficult (well, at least "non-trivial") to get the icon to display centered. I went through the old process of using WPF Geometry Transformer to "Tansform" [sic] the Path into something smaller, and poof... -ish.

<PivotItem
    x:Class="CloseableTabs.CloseableTabHeader"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:CloseableTabs"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="300"
    d:DesignWidth="400">

    <StackPanel Orientation="Horizontal">
        <TextBlock Margin="0,0,0,0" Name="txtHeader" FontSize="18">Test</TextBlock>
        <Button Background="White" Click="Button_Click">
            <Button.Content>
                <PathIcon Data="M13.3000001907349,2.09999990463257L3.5,2.09999990463257 2.51005053520203,2.51005053520203 2.09999990463257,3.5 2.09999990463257,13.3000001907349 2.51005053520203,14.2899494171143 3.5,14.6999998092651 13.3000001907349,14.6999998092651 14.2899494171143,14.2899494171143 14.6999998092651,13.3000001907349 14.6999998092651,3.5 14.2899494171143,2.51005053520203 13.3000001907349,2.09999990463257 M13.3000001907349,13.3000001907349L3.5,13.3000001907349 3.5,3.5 13.3000001907349,3.5 13.3000001907349,13.3000001907349 M11.8999996185303,5.87999963760376L9.3799991607666,8.39999961853027 11.8999996185303,10.9200000762939 10.9200000762939,11.8999996185303 8.39999961853027,9.3799991607666 5.87999963760376,11.8999996185303 4.90000009536743,10.9200000762939 7.42000007629395,8.39999961853027 4.90000009536743,5.87999963760376 5.87999963760376,4.90000009536743 8.39999961853027,7.42000007629395 10.9200000762939,4.90000009536743 11.8999996185303,5.87999963760376z" />
            </Button.Content>
        </Button>
    </StackPanel>
</PivotItem>

That's fun, isn't it?

That looks okay, but there are still two problems.

1. How do I set the Label (now placed in txtHeader) of my UserControl in XAML?
2. How I do I remove the PivotItem once its button is clicked.

Okay, three problems -- I really should change the icon to a greyed-out version when the PivotItem's not active. That looks awful. Mark that down for version two.

Accessing a property in custom XAML from C

I found the answer to the first pretty easily: You have to use a Dependency Property, of course!

Dependency property values are not stored as fields on the class, they are stored by the xaml framework, and are referenced using a key, which is retrieved when the property is registered with the Windows Runtime property system by calling the DependencyProperty.Register method.

But wait! There's more!

Defining a dependency property can be thought of as a set of concepts. These concepts are not necessarily procedural steps, because several concepts can be addressed in a single line of code in the implementation. This list gives just a quick overview. We'll explain each concept in more detail later in this topic, and we'll show you example code in several languages.

• Register the property name with the property system (call Register), specifying an owner type and the type of the property value.
• There's a required parameter for Register that expects property metadata. Specify null for this, or if you want property-changed behavior, or a metadata-based default value that can be restored by calling ClearValue, specify an instance of PropertyMetadata.
• Define a DependencyProperty identifier as a public static readonly property member on the owner type.
• Define a wrapper property, following the property accessor model that's used in the language you are implementing. The wrapper property name should match the name string that you used in Register. Implement the get and set accessors to connect the wrapper with the dependency property that it wraps, by calling GetValue and SetValue and passing your own property's identifier as a parameter.
• (Optional) Place attributes such as ContentPropertyAttribute on the wrapper.

Note If you are defining a custom attached property, you generally omit the wrapper. Instead, you write a different style of accessor that a XAML processor can use. See Custom attached properties.

... [Oh, but there's more... -mfn] ...

In some scenarios, you are defining dependency properties for objects that are used on more than one UI thread. This might be the case if you are defining a data object that is used by multiple apps, or a control that you use in more than one app. You can enable the exchange of the object between different UI threads by providing a CreateDefaultValueCallback implementation rather than a default value instance, which is tied to the thread that registered the property.

I feel like the Knights of the Round Table after they've considered whether to go to Camelot. Okay, nevermind. 'Tis a silly place. I won't make these new headers XAML-friendly for "version mvp". We're going to set the Label somewhere outside of XAML. Moving on to #2...

Removing a PivotItem on custom UI click

My first bright idea, which I've used before, is to take the sender, cast to the appropriate widget, find its Parent, and just slowly climb my way back up the UI hierarchy until I find what I need. That is, I'll drill up, like a jQuery closest, until I have the PivotItem, then the Pivot, that I need. Then I can remove the PivotItem from the Pivot's Items collection. Easy, right? The best part is that this normally doesn't change, and you can have the PivotItem remove itself without any serious hooking in with what's around it. (I could probably do this more impressively with reflection, and essentially hook an extension method up for UIElement that parallels the jQuery closest method I references earlier.)

But crimminy. Here's the drill-up code I ended up with, originally in the UserControl's Button's Click event. Remember that this is just exploratory code at this point.

System.Diagnostics.Debug.WriteLine("Click");
Button btn = (Button)sender;
System.Diagnostics.Debug.WriteLine(btn.Parent.GetType());

var spam = (StackPanel)(btn.Parent);

var header = (CloseableTabHeader)(spam.Parent);
var headerItem = (PivotHeaderItem)(header.Parent);

PivotHeaderPanel headerPanel = (PivotHeaderPanel)(headerItem.Parent);
Grid weirdPiGrid = (Grid)headerPanel.Parent;

ContentControl contentControl = (ContentControl)weirdPiGrid.Parent;
Grid anotherGrid = (Grid)contentControl.Parent;
PivotPanel pivotPanel = (PivotPanel)anotherGrid.Parent;
ScrollViewer scrollViewer = (ScrollViewer)pivotPanel.Parent;
Grid yaGrid = (Grid)scrollViewer.Parent;
Grid yaGrid2 = (Grid)yaGrid.Parent;

That's interesting, right? Not only is it much more convoluted a design than I would've expected, we also never get a PivotItem or a Pivot -- yaGrid2 is the "root element" of the page, I think. And the stuff we do get, like PivotPanel, come with warnings like this:

PivotPanel isn't typically used for general XAML UI composition.

/sigh This was easy, wasn't it? My UI traversal has me lost in the funhouse rather than in a cleanly executed UI widget hierarchy that follows its own mental model. Feels like I went out of the back door and I'm seeing the outside of the house isn't finished quite yet. Unfortunately, I get the impression that UWP is held together by chicken wire and duct tape a lot more than I'm used to seeing in a Microsoft UI toolkit.

There's probably some easy way to save what I'm trying to do from the fire, but forget it. For version mvp, I'm changing the CloseableHeader constructor from this...

public CloseableTabHeader(string headerText) : base()
{
    this.InitializeComponent();
    txtHeader.Text = headerText;
}

... to this...

public PivotItem itemParent = null;
public Pivot pivotParent = null;

public CloseableTabHeader(string headerText, PivotItem itemParent, Pivot pivotParent) : base()
{
    this.InitializeComponent();
    txtHeader.Text = headerText;
    this.itemParent = itemParent;
    this.pivotParent = pivotParent;
}

I think you see where I'm going. Yes, I feel a little dirty too. Now I can remove it easily in the Button's Click event:

private void Button_Click(object sender, RoutedEventArgs e)
{
    if (null != this.itemParent && null != this.pivotParent)
    {
        this.pivotParent.Items.Remove(this.itemParent);
    }
}

Hrm. Perhaps "parentPivot" is better than "pivotParent", and same with itemParent. In any event, you want to bubble up to the closest PivotItem and Pivot, and this makes that stupidly straightforward, though I do wonder how much Dispose-ing I should be doing.

I don't love that set-up, and I really dislike the way the buttons' bounds "highlight" simply when you mouse over the icon, like this:

... but overall, that's a workable solution that will make discovery for tab closure much more straightforward for non-keyboard addicts.

Labels: c#, coding, markupdown, uwp