Put the knife down and take a green herb, dude.
One feller's views on the state of everyday computer science & its application (and now, OTHER STUFF) who isn't rich enough to shell out for www.myfreakinfirst-andlast-name.com
Using 89% of the same design the blog had in 2001.
FOR ENTERTAINMENT PURPOSES ONLY!!!
Back-up your data and, when you bike, always wear white.
MarkUpDown is the best Markdown editor for professionals on Windows 10.
It includes two-pane live preview, in-app uploads to imgur for image hosting, and MultiMarkdown table support.
Features you won't find anywhere else include...
You've wasted more than $15 of your time looking for a great Markdown editor.
Stop looking. MarkUpDown is the app you're looking for.
|Wednesday, January 04, 2017|
Today, I ran across a clickbaitily titled post called Just Say No to More End-to-End Tests posted at the Google Testing Blog, written back on April 22, 2015, detailing a fictional end-to-end (e2e) testing run. The scenario was very clearly fictionalized, but you can forgive most of it, as they're simply trying to illustrate a few points.
Here's their "analysis" of the test, which apparently took over a week to complete and had some serious errors with the testing apparatus as well.
The author uses this cluster of an e2e test to argue for a preponderance of unit and integration tests...
That's a lot of info to digest.
Here's the problem: "Simulates a Real User" isn't a single point. Their chart really isn't a scorecard, though that's how it's presented.
My quick list of critiques:
Look, more typical situations in my experience are that you either have lip-service unit tests [only] or that you have no testing at all.
Guess what's most valuable if you have to pull teeth to convince management that testing is important? In my experience, it's an "end to end" test. You'll get the most return on your testing resource dollar with smoke tests.
I do like to call it a smoke test, and I've had pretty good results using Selenium to automate a browser using C#. Which browser? Well, you get your biggest bang for the buck by just using one. It'd be great to test in IE, Firefox, Edge, Chrome, and even macOS & iOS Safari, but each time is a diminishing return. The first test in whatever browser is going to catch 75% of what's wrong.
I recommend using either 1.) The lowest common denominator for browser functionality, usually IE, or 2.) Whatever's the easiest to get to behave for your tests. Timing can be a pain in Selenium.
Recently, I've used Selenium's Chrome web driver. That's dangerous. Chrome seems the most fault tolerant of the browsers, and on a developer's phat box, you can add slow downs to browser incompatibilities as problems that are often hard to notice.
But one browser, going through an automated master set of user stories, will quickly ensure that the 80% of your website that's your real bread and butter works and wasn't screwed up by whatever whizbang gizmo your latest release just pushed out the door.
If there is an error, sure, it can be hard to immediately identify the exact cause, which the Testing Blog seems to think is the end of the world. And perhaps a unit test to cover each error you do discover and track down will be useful. Do that.
But, again, unless you're doing TDD, to ask developers to write useful unit tests ahead of time almost never covers what your users are actually going to do and doesn't prevent bugs your own developers and QA are going to catch anyway. To get useful unit tests, you're going to have to spend even more time getting developers to code review others' unit tests, and you barely had time to green light unit tests to begin with. (How do I know that? If you had plenty of time, you'd be using TDD.)
And if you've gotten so far down the line that you can't tell if an error is coming from the front or back end, as in the Testing Blog's worst-case scenario, well, you have worse problems.
My suggestion if you're losing days looking for bugs your e2e tests turn up, and more days with your test lab going down?
Forget the users; your development process is broken. It's time to start arguing for TDD.
Update: Interesting to look at the slide linked in the quote, above:
And the notes address what I'm saying exactly (and argue against it):
Interesting. I'm going to guess Google's coders do a much better job writing tests. If you want to convince me, talk more about how tests are written, and less about how they're better than automated UI testing if you have no testing now.
But wow, look at some later slides:
Comments to that slide:
Not great. I've heard what's important when coding is to know which bugs to ship, but which bugs in tests to ignore? What are we doing? Later on...
EXACTLY. I think the Testing Blog grossing oversimplified the take-home from this fellow's presentation. I'll job bombarding pixels now.
posted by ruffin at 1/04/2017 01:23:00 PM
All posts can be accessed here:
Just the last year o' posts: