Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.

Best of 2018 to you. But this isn’t a real blog post, it’s a placeholder to debug a subtle Unicode bug in my comment system that doesn’t appear on my staging environment. *sigh*

Aaaaand, that does it. The bug was introduced when I switched over to Ruby 2.x, approximately five years after I should have. Ruby 2 is known to have differences in UTF-8 handling. That turned out to be OK, the bug was in the fifteen-year-old Perl substrate. I have to say I’m a different kind of person than I was when I bashed out that Perl in 2002. And while I don’t want to diss Perl, I would never do anything like that again.

Oh well, pardon the interruption.

A large proportion of the information I consume on my mobile comes from two apps: The Economist (iOS, Android), and a whole lot of feeds via Feedly (iOS, Android). They both present text in narrow columns and share a lovely, simple navigation metaphor which unfortunately has recently broken (on Android at least).

In both cases, each story is presented in a single column that fills the screen from left to right and extends arbitrarily far down, as long as need be for the piece’s body. The stories are side by side. There are only two ways to navigate: Up or down the columns, and right or left between stories. Here’s a picture.

I’m sure other newspaper and information-dense apps must have adopted this metaphor because it’s just so intuitive. It works great on mobile because no matter how fat-fingered you are, the app only has to distinguish between vaguely-North/South and vaguely-East/West and the right thing almost always happens.

Now, there’s a subtlety: If you swipe left to go to the next story, you want to bounce up to its top. But if you swipe back, you almost certainly don’t want to bounce to the top of the previous story, because you already read that, and you probably are going back because you accidentally swiped while reaching for a drink, or realized a microsecond after you swiped that that paragraph was more interesting than it’d seemed when you launched your finger.

And if you’ve read a ways down story B and swipe back to A, then forward again to B, once again you don’t want to pop to the top of B. Which is to say, it should remember your scroll position, at least for the immediate predecessor and successor of whatever it is you’re reading.

And, both apps used to. But sometime in 2017 they stopped, snapping remorselessly to the top of whatever column you swipe to, forward or back. It’s annoying as hell; maybe because I often change my mind about leaving an article and decide to go back. And the fact that it happened in both the Economist and Feedly makes me wonder if something in the Android app framework broke.

So I’m not sure who I’m mad at; the developers of two of my favorite apps separately, or just someone in the Android group.

If you’re reading this and you think it’s you I might be mad at, well hey, no biggie, but please fix?

The IETF just published RFC 8259 (also known as “STD 90”). Editor, yr humble servant. The legacy-ASCII full text is here and there’s a nice-looking HTML verion here. I think this is the last specification of JSON that anyone will ever publish.

The story of how we got to RFC 7159, this RFC’s predecessor, is told in JSON Redux AKA RFC7159 and I won’t re-tell it. The reason 8259 exists is that the ECMAScript gang went and wrote their own extremely minimal spec, Standard ECMA-404: The JSON Data Interchange Syntax, and there was reason for concern over dueling standards. But, after a certain amount of standards-org elephant-gavotte, each of ECMA 404 and RFC 8259 normatively references the other and contains a commitment to keep them consistent in case any errors turn up. Which is a good thing, but this text has been re-examined and re-polished so many times that I doubt either side will ever revisit the territory, thank goodness.

8259 contains one new sentence: “JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].” Given that, by 2017, an attempt to exchange JSON encoded in anything but UTF-8 would be irrational, this hardly needs saying; but its absence felt like an omission.

Which spec should you use?

If you want to understand JSON syntax, you still can’t beat Doug Crockford’s original formulation at JSON.org. If you want to use an RFC as foundation for a REST API or some other Internet protocol, I actually don’t recommend 8259, I recommend I-JSON, RFC 7493, which describes exactly the same syntax as all the other specs (by referencing 7159), but explicitly rules out some legal-but-dumbass things you could do that might break your protocol, for example using anything but UTF-8 or having duplicate member names in your objects.

What next?

I wouldn’t be surprised if the IETF JSON Working Group shut down forever. On the other hand, we still lack a decent high-quality JSON Schema language. Also, JSONPath, which is damn useful, doesn’t have a proper definition anywhere. So there’s work to be done if anyone wants to do it.

Thanks and apologies

To the working group, its chairs and Area Directors and members. Also, at one point in the process I got all grumpy and sulky and they didn’t let that get in the way, just went ahead and got the job done. The Internet owes them thanks.

I spent four nights in Las Vegas while at re:Invent. The city’s daytime aspect is kind of flat and low-contrast because who cares what it looks like then? Photographers there come out like vampires after dark. I took along the ludicrous “Achromat” lens for the sparklies, so got ’em if ya want ’em. Also hotel, music, and venue recommendations you won’t want to miss.

Let’s start with the sparklies.

That lens sure is fun. Not to mention the effect when you tell your table-mates you brought along your Las Vegas Lens and pull a whole lot of tubular gleaming brass out of your bag.

The Bellagio & its fountain

Veterans of the strip may wonder how this picture came to be. Reflections in a window were involved.

The first time I’ve stayed there, and I think I’ll go back. The rooms are nowhere near as glam as in recent-Strip joints like the Wynn or Aria, but things pretty well just work, the casino is less sprawly, and it’s kind of in the middle of things, and then there’s that view if you pay up for a Fountains room.

That picture is an outtake from a two-minute video shot from my 20^th-floor room, with faint tinny sound. Taken with the Pixel 2 and really not bad; note how the vibration control squashes the effect of my chronically shaky hands. Warning: It’s 124 MB! The full-size 4K version, which looks glorious on my 15" retina screen, is 710 MB; yow.

That picture doesn’t really capture the scale of the thing; here we are down on the street watching the watchers.

Sitting at the desk in my room working on my speech, I was amused by the fountain-maintenance process. They could do better, because every time I’ve watched the fountain, there’ve been annoying imperfections: jets that don’t jet, lights that stay dark.

I’d like to see someone get ambitious with the fountain music, maybe something by Phil Glass or *gasp* a rock & roll piece. That wouldn’t be very Vegas I guess.

Doubling Down with the Bastards

That would be Thee Swank Bastards, a surf-guitar band, playing at the Double Down Saloon, located not that close to the Strip. They offered a couple hours of pure pleasure.

The woman playing bass in the first picture spent most of the night in showgirl mode; her costume, especially the gloves, was fabulous, and she threw in some tasteful striptease. She and the band were having nearly-infinite fun and the standard of surf-guitar playing was impeccable. You can loathe America’s politics and many of its cultural tropes, but no other nation comes close at the pure joyful ecstatic madness of low-rent rock&roll; a tip of my hat across the border.

The Bastards leavened their set with appalling, filthy, dumb jokes whenever they stopped playing. The only one I remember is “What’s the difference between Jesus and a picture of Jesus?” “You only need one nail for the picture.”

As for the Double Down, it’s uh, sincere. A whole lot of tattoos and beards and ripped tights. The big sign on the wall says “Shut Up And Drink”, the one on the bar with embedded poker machines says “Shut Up And Gamble”. We thought about ordering the bacon martini, but nobody did. The Bastards play the Double Down the last Wednesday of every month, and if you’re in Vegas I recommend dropping by.

People, watching

(The people are more interesting after dark, too.)

I actually still don’t like Vegas very much. My camera does though, but it only comes out at night.

The Atlantic published The Coming Software Apocalypse by James Somers, which is full of dire warnings and strong claims. Here’s one: Since the 1980s, the way programmers work and the tools they use have changed remarkably little. My first programming job was in 1979, I still construct software, and I can testify that that assertion is deeply wrong, as is much else in the piece.

I would very much like to place an alternative view of my profession before the people who have consumed Mr Somers’, but I wouldn’t know how, so I’ll just post it here; maybe an Atlantic reader or two will stumble across it.

Oops

When I read this piece I tweeted “Reading non-coders’ words about coding is… weird.” That was wrong because there’s plentiful evidence that he’s a well-accomplished developer. So, apologies. But he’s still wrong.

Wrong, you say?

First of all, the people Somers describes, who write the matter-of-life-and-death logic at the center of the systems that dispatch 911 calls and drive cars and fly planes, are a tiny minority — it’s like a dog-care piece focused on wilderness search-and-rescue dogs. There’s nothing wrong with that kind of dog, nor with the people who program safety-critical systems, but I’ve never met one, and I’ve been doing this for almost four decades.

There’s another problem with Somers’ piece: its claim that writing code is passé, that we’ll be moving away from that into a world of models and diagrams and better specifications and direct visual feedback. This is not exactly a novel idea; the first time I encountered it was in a computing magazine sometime around 1980.

Yes, the notion that you build complex interactions between computing devices and the real world by editing lines of code feels unnatural and twisted, and in fact is slow and expensive in practice. We’ve been looking for a better way since I got into this game; but mostly, we still edit lines of code.

And as for the sensible-sounding proposal that we just write down our requirements, not in code, but in something much higher level, in such a way that a computer can understand them as written and execute them? That’s another old and mostly-failed idea.

So, Somers is wrong twice. First, in asserting that software is moving away from being built on lines of code (it isn’t), and second, that the craft of constructing software isn’t changing and getting better (it is).

So, what do you actually do, then?

Glad you asked. All sorts of things! We developers are now some millions strong worldwide — almost certainly more than ten million and I suspect less than fifty; but it’s hard to measure.

As in most professions, most of the work is strikingly pedestrian; discovering what our co-workers need their computers to do, and also what their managers want, and trying to arrange to keep these tribes happy and at peace with their computers and each other.

To a huge extent, that involves acquiring, deploying, and configuring software that was created by others. Thus, a lot of time in meetings, and then even more figuring out how to make the travel or scheduling or amortization app do what people need done.

On the other hand, some of us write software for rockets, for music synthesizers, for Pixar movies; all these things have an obvious cool factor. And others (surprisingly, among the most-admired) write “low-level” software, useful only to programmers, which underlies all the software that is useful to actual humans. There are many kinds of this stuff: for example “Operating Systems”, “Database kernels”, “Filesystems”, “Web frameworks”, and “Message brokers”.

Software is getting better

Let me be more specific: Compared to back when I was getting started, we build it faster and when we’re done, it’s more reliable.

The reasons are unsubtle: We build it faster because we have better tools, and it’s more reliable because we’re more careful, and because we test it better.

Reviewing

The big software builders (for example Amazon Web Services, where I work) have learned to follow simple practices with big payoffs. First, those lines of code: They never get put to work until they’ve been reviewed by a colleague; in the vast majority of cases, the colleague finds problems and requests changes, arguments break out, and the new code goes through several revisions before being given the green light. For major pieces of infrastructure code, required approval from two more reviewers, and ten or more revision cycles, aren’t terribly uncommon.

Unit Testing!

Software is constructed of huge numbers of (mostly) very small components; we use names like “functions”, “routines”, and “methods”. They are the units that Unit Testing tests. The unit tests are other pieces of software that feed in many different pieces of data in and check that what comes out is as expected. There are commonly more lines of code in the unit tests than the software under test.

We have loads and loads of tools specifically set up to support Unit Testing; among other things, when you look at those lines of code, there’ll be a vertical bar in the margin that’s green beside lines of code that have been exercised by the unit tests, red beside the others.

These days, we don’t always demand 100% coverage (some code is just too routine and mundane) but we expect anything nontrivial to be covered well by the tests. I think the rise of unit testing, starting sometime not too long after 2000, has yielded the single biggest boost to software quality in my lifetime.

There are other kinds of testing (“Integration”, “Smoke”, “Fuzz”) and we use them all, along with tools that read your code and find potential problems, just like Microsoft highlights your spelling mistakes.

Night and day

It doesn’t sound like much. But seriously, it’s like night and day. Does it sound a little tedious? In truth, it is. But also, our tools have been getting better year over year; programming in 2017 is really a lot more pleasant than it was 2007, 1997, or 1987.

It’s like this: You sit down to improve a piece of software, make a couple of changes, and suddenly a lot of unit tests are failing, leaving ugly red trails on your screen. (In fact, if you made changes and didn’t break unit tests, you worry that something’s wrong.) But then you dig into them one by one, and after not too long, it’s all back to green; which is really a good feeling.

I’m not going to argue that the advanced methods Somers enumerates (being model-driven, state machines, things like TLA+) are useless, or that they’re not being used; I personally have made regular use of state-machine technology. But by and large they’re side-shows. We build software better than we ever have, and it’s just a matter of reviewing and testing, and testing and testing, and then testing some more.

We’re not perfect. But we’re really a lot more grown-up than we used to be. And, relax: There’s no apocalypse on the horizon.

Sometimes when we’re trying to hire a senior employee, I get asked to do a “sell call”, tell them what it’s like to work here. Since I’m coming up on three years and haven’t quit, I guess the supposition is that I’ll be positive. Since these candidates are outsiders and some of them don’t come to work for us, nothing I can say can be a secret. So why don’t I tell everyone?

Before I dive in any further, everything here relates to AWS, not Amazon as a whole. It might be true of the retail side too, but I don’t know because I don’t work there.

So, what is it like to work here?

Customers

That’s the big deal. AWS has over a million and you get to meet them all the time, and while there are lots of geek introverts who might not like that, I do. Anyhow, lots of the customers are geek introverts too.

When you ship low-level horizontal general-purpose technology, there’s no way to predict what people are going to do with it. Which in practice means that you’re going to be shocked by the way it gets used. For an old guy like me it’s hilarious to hear the kid engineers looking at a bug report or feature request and saying “WTF, why would anybody want to do that?”

Me, I like computers, and I like hearing about interesting things people do with them, and it makes me happy when the things they do work out well, especially when I helped build what they’re using. I could tell you stories that’d curl your hair (or straighten your wavy locks) but they’re not mine to tell.

Anyhow, if you like talking to customers you’ll like working here, and if not, probably not.

Uptime

It matters more than anything. Some of our services are cooler than others, but what I think customers care about most is confidence that the services, cool or boring, will be there 24/7/365. What that means is that everything has to be automated, and much of the most brilliant engineering at AWS, done by some of the smartest people, does its work behind the scenes where nobody will ever see it.

So if you’re the kind of person who, for example, thinks figuring out a better way to automate detecting hot-spots in back-end clusters and re-routing traffic to cool things down is interesting work, then you’ll like working here. If not, maybe not.

Only maybe not, because we have lots of people who wrangle JavaScript frameworks and mobile SDKs and build tools and database kernels all day. But operational automation, that’s the heart of the matter.

Being document-driven

Go ahead and Google “Amazon six-pager” if you haven’t already.

If you’re the kind of person who’s OK with spending a lot of time constructing carefully-written narratives, and being in meetings that start with 20+ quiet minutes while everyone reads the narrative, you’ll like working here, and if not, definitely not. Disclosure: I’ve written over a million words on this blog so I’m seriously predisposed to like this part.

There is one downside: Suppose you’re an author of the document being read. It can be simultaneously crushingly boring and twitchingly nervous while AWS’s senior leadership plows their way through your words in stony silence.

The actual software

I don’t think anyone would be surprised. The services are usually conventional RESTful services, usually composed of multiple microservices, usually in mainstream programming languages using mainstream programming frameworks, We use conventional compilers and debuggers and IDEs and frameworks for HTTP and unit testing and integration testing and dependency injection and so on.

Yes, there’s some secret sauce voodoo down in the infrastructure that’s pretty magical. But most of us don’t work on that most of the time.

What’s unusual is the proportion of the code focused on availability and data durability. But you already knew that. We also care a lot about doing better than O(N), because N is typically so freaking huge at Amazon. I personally seem to end up working mostly on message processing of one kind or another which is sort of stuck on O(N) in the number of messages, so we’re often left with micro-optimizations.

You know what makes me happy? One little library I wrote is currently being used by several different teams to efficiently do a useful thing on a million-ish messages per second in aggregate. There aren’t that many places where your code gets that kind of opportunity.

Organization and culture

The teams at AWS have a huge amount of independence. If you’re on a team that’s trying to ship something or operate something or improve something, that’s generally great; nobody will get in your way. On the other hand, if you’re trying to build something that works across multiple service teams, it can give you grey hairs. But I already had those; and it’s mostly a feature not a bug.

Corporate “culture” is a thing I find it hard to be articulate about. I find the asshole density nonzero but lower than average. We are trying to do very difficult things and we fail a lot; Mature technology companies try make those teachable moments, and we do a pretty good job of that. There is [*gasp*] politics. I’m going to argue that the proportion of really bad managers is unusually low; but that’s just an anecdote, wouldn’t know how to measure it.

My time rarely feels wasted.

Money?

If you work in a tech job at any large high-tech company, you’ll be well-paid by any reasonable standard. Want more than just “well-paid”? The brutal truth is that the way high-tech compensation is structured, you’re making a bet on your employer’s share price. If it goes up really a lot while you’re working there, you’re gonna make out like a bandit, and if not, not. All these things are as true at Amazon as anywhere else. Welcome to twenty-first century capitalism.

What sucks?

After I’ve talked through all this stuff, the person who might come to work for us wants to hear about the downside. Fair enough.

I guess the biggest one is that we’re not perfect at operational automation, so everything we fail to automate, every combination of compute and network and storage failure we weren’t smart enough to auto-remediate in advance, has to be dealt with manually. Which means people have to be on-call and yeah, sometime the phone goes off after midnight.

The good news is, we’re getting better, learning how to build automation in at the core. Which means that the fresh new AWS services are better at self-maintaining and self-healing, so on-call sucks less. On the other hand, the older services tend to have bigger teams and thus less on-call time per person, so it evens out.

But there are some people who sufficiently dislike the prospect of that occasional wee-hours page that they just don’t want to stick around with us. Fair enough.

I dislike our office-automation setup, but then I’m the kind of person who really likes Google Docs and Calendar and Gmail, so maybe I’m an outlier.

One more thing: Most jobs at AWS, you’ll never be able to explain to civilians what it is you do for a living. I generally say “I help keep the Internet running” and you know what? That’s not a total lie.

On balance

I could go on about diversity and work environments and benefits and process and promotion and so on, but at the end of the day it’s just another high-tech company, nothing surprising.

I don’t really need the money, but I haven’t quit.

In The bubble without any fizz, The Economist addresses the fact that financial-asset prices (stocks, bonds, and the like) keep drifting up and up in a world where inflation doesn’t; are we in a mega-bubble? What’s really going on? Looks obvious to me, but then I’m a left-winger. I think it’s all a straightforward consequence of economic efficiency and class warfare.

Inflation?

It stays low because our globalized economy is hyperefficient at making the things we want, extracting the fuels we burn, and growing the food we eat. And also flexible enough that it can scale up and down to meet demand without apparently kicking off waves of inflation or bankruptcy.

All of which is true, but I think the effect of class war is even bigger. Let’s start with a picture.

Unions are in retreat and conservatives rule in the great economies of the developed world. Generation Z is being forced into the gig economy, into low-agency, low-paying, no-commitment jobs where being a shift lead at Starbucks is an aspirational goal.

When you don’t have to pay people much, well yeah, you get low inflation.

Financial Boom?

When it’s the prices of household goods, they call it “inflation”. When it’s financial assets, they call it “a bull market”. But the cause is the same: Too much money chasing too few targets.

It’s like this: There’s a surplus of accumulated wealth (see Piketty), and the people who hold it can’t possibly spend it on goods and services — how many yachts can you waterski behind? So they throw it at financial products, which then inflate.

I think this is bleedingly obvious, but that’s possibly because I’m a citizen of the technology scene, and of Vancouver. In the venture-capital business, there’s just way too much money chasing way too few unicorns. Go to a startup showcase sometime and hear the pitches, see who’s getting funded. If you’re like most people, you’ll leave shaking your head.

I also sit in Vancouver, which is experiencing severe inflation, in our housing sector. Since Vancouver is a relatively low-paying city, not just on a world scale but compared to its Canadian peers, it’s obvious that global capital is part of the problem. Our real-estate frenzy has become an international news story (it appears in the Economist piece linked above), but I un-humbly think my own 2015 Game of Homes captures the essentials pretty well.

So, what do Vancouver real estate and financial instruments have in common? Limited supply, that’s what. Check out A Dearth of I.P.O.s, but It’s Not the Fault of Red Tape ; not only has the IPO rate plunged (from 706 in 1996 to 105 in 2016), so has the number of public companies (from 7,322 to 3,671). Declining supply, increasing demand, d’oh.

Other financial assets, e.g. bonds, may not be as supply-starved (I couldn’t turn up good numbers) but it stands to reason that when companies all over the world are reporting robust profits (class war, remember?) they probably don’t need to borrow as much. And (as The Economist notes) when investors are driven to buying 100-year bonds from Argentina, a nation that has defaulted on its bonds six times in the last 100 years, you know they’re getting desperate.

More evidence, were any needed, of the global capital surplus? How about the $21 trillion being hidden from the tax-man here and there around the world?

What next?

Beats me. The sad thing is, I don’t see any reason why things can’t go on the way they are for the foreseeable future. Objectively, it would probably be good for the world if a lot of the accumulated wealth was just vaporized. Unfortunately, the only way that seems to happen historically is in major wars, as Piketty’s graphs illustrate. Which nobody wants.

Then there’s the leftist world-view, for example On the Left from last year. Tl;dr: Tax wealth, force radical transparency on asset ownership, jail business criminals, offer universal basic income. Works for me; but I’m still looking for the right political party.

There’s just too much money out there not doing anything particularly useful.

There are two kinds of client applications: The first kind has a “refresh” or “reload” button to make sure your app’s in sync with its server’s view of the world. The second kind is broken.

Of late, I have to deal regularly with several apps, notably including an emailer and a car-sharing service, that lack such a button. I can imagine why — a customer-focused product manager said “Steve Jobs taught us that fewer controls are better and we should just take care of making sure we’re in sync with the cloud. So lose the button.”

Except for, it doesn’t work. Apparently nobody in the world is smart enough to arrange for flawlessly reliable hands-off client/cloud synchronization. There are times when you just know that what you’re seeing on the screen is wrong and if the stupid app would just take your word for it that its world-view is stale and ask for a brain transplant from its server, things would be OK.

The car-share app is particularly aggravating, showing me a map dotted with nearby vehicles when I know for sure that this time of day I’ll be lucky if there’s even one within walkable distance. Since it totally refuses to sync, I have to switch to the Android “recents” screen and kill it. When I re-open the app, it’s OK.

Why might this happen? Well, maybe switching from WiFi to cell data (or vice versa) left some layer confused about network truth. Possibly, because your app was written in a programming language with multiple threads and shared mutable state (silly, silly programmer), the cache is malfunctioning. Maybe that same thing happened on the server. Or in the CDN. Or maybe there’s an obscure hiccup in a distant node_modules dependency that you’re tickling.

Dear product managers: Show some humility. When a customer really thinks your app is wrong and they know how to fix it, don’t get in their way.

I mean No Man’s Sky the game, which I’ve been playing again lately. It’s been accused of being mostly a platform for generating cheesy sci-fi book covers, but that’s not true, and also I love cheesy sci-fi book covers, so this is mostly to show you some. With a few words on the game.

Updates

No Man’s Sky launched just over a year ago, preceded by a massive hype wave and followed by howls of disappointment when it fell short of expectations. There’ve been three major updates, each with massive additions. and it’s becoming increasingly like a “normal” game, with missions and NPC’s and economies and so on.

Its biggest charm probably remains that it’s an easy game, occasional flashes of excitement, but mostly just cruising along from star to star, enjoying the views.

The game is starting to add inklings of multi-player, and multiple players are self-organizing, most notably to create The Galactic Hub, which I hope to reach some day.

Aliens!

In NMS they’re, more than anything, fun. The planets have their own procedurally-generated menageries, but there are three main species you can actually have relationships with, and learn the languages of. There’s lots of back-story that you soak up as you move through the game. Here’s a Gek.

The beasts in NMS are chatty and come in all sizes, shapes, and colors. They’re more playful than scary. And like I said, it’s an easy game, any creature that attacks you is pretty well toast. But, here’s Jabba the Flowerpott.

My experience

In the picture just above, that’s my current starship, parked on the left. Nice ride, eh? For the cognoscenti: A decent little Samamoga 27-slot B-class fighter, modest stats but a reliable pirate-killer. Which sort of sums up my playing style. I’m busy, with a job and family; thus not a serious gamer. One of the really nice things about NMS is you can do it for an hour here and an hour there and keep progressing and never really get stuck.

I restarted after the 1.3 update (like a lot of others) and I’m going to keep playing till I get a freighter.

Gender

The three primary alien races you deal with do not participate in the human notion of gender. Thus, when individuals are mentioned in text (and there’s a lot of text in NMS) it’s always in a scrupulous third-person: They, them, their. It’s amazing how quickly you get used to it.

On the other hand, the beasts that populate the planets are gendered, and while male and female do appear, one also finds Asymptotic, Asymmetric, Orthogonal, Prime, Radical, Indeterminate,and Vectorised.

So the authors of NMS are having some real postmodern twenty-first century fun; which on balance is a fair description of the whole game. I’m still on board.

Our media, pro and social, echo with blasts of self-righteous anger over proposed legislation which would eliminate a few popular tax dodges. Weirdly, I see no-one arguing the other side; that the tax proposals are reasonable. I think I’m qualified to make that argument, so I will. [If you’re not Canadian, you can probably stop reading here.]

The proposed tax changes

They’re summarized pretty well here. Basically, if you have a business and it’s incorporated — say you’re a doctor, lawyer, contractor, accountant, that kind of thing — you can use your corporation for tax tricks, the effect being that you pay less tax on the same income.

The tricks have been well-known for years; every competent financial advisor and wealth manager in the country tells every client about them.

Disclosures

(And I wish a few more people publishing op-eds on this subject would offer theirs.)

1. I have a good income and pay a lot of tax.

2. I’ve had capital gains over the years, from a successful startup and a couple of lucky investments.

3. My wife and I have a corporation, which is useful in supporting the consulting businesses both of us have run from time to time. But we’ve hardly ever been able to use any tricks; one time by accident and acqui-hire the corporation ended up with pre-IPO Twitter shares, and we did save some tax bucks when we sold them.

4. I was a co-founder of a company that currently has just over 14,000 employees.

5. I didn’t vote for the government that’s making the proposals.

My feelings on tax generally? My bills are shocking, but on the other hand I’m a heavy direct user of government services: roads, bridges, public transit, bikeways, libraries, athletic facilities, public broadcasting, health care, emergency first responders. Plus, having grown up in a third-world country, I have a hearty appreciation for the rule of law and the social safety net. So yeah, I’d like a lower tax bill and I might vote for a party that had specific proposals on paying for one with cutbacks on the parts of government I don’t like.

But I generally do think that people who make about as much money as me should pay about as much tax as me.

The complaints

It amounts to a bunch of well-off people who are going to start paying the same tax rate I do explaining why they shouldn’t have to, because they work hard, create jobs, and are just all-around nice people.

Here’s an anesthesiologist who’s so upset that, he says, he and the other doctors are going to go mini-John Galt and start working less.

Here’s a guy who pays $225K in tax and says he’s “already contributing FAR more than his fair share”.

The complainers complain that they don’t get a pension. Neither do I; in fact nobody does any more, except civil servants.

They also bitch about not getting paid vacation or “any other benefits”. Oddly enough, when we’re on vacation in sunny destinations, we tend to encounter lots of doctors and lawyers and contractors and so on, so somehow they manage to get away. And now that we’re older and know a few retirees, I gotta say that the small-biz-owner contingent seems well-represented among those in their “golden years” where by “golden” I mean “rolling in dough”.

Also, do the math: Anyone who’s paying $225K in tax is a very well-off individual, with lots of room to save for retirement, and enough cash-flow to visit Maui or Cabo every winter.

But they do have a point; It wouldn’t seem completely insane to me if there were tax deductions for those whose employment situation is low on benefits. But that should be done explicitly, rather than nudge-nudge-wink-winking at small-biz corporate fiddles.

That aside, I’m sorry but I just don’t see any reason why someone who makes my kind of money shouldn’t have to pay my kind of taxes, just because they’re a small business. And in the testimonies I read, they come across as hypocrisy-drenched greedheads.

If you want lower taxes, I think there are two good courses of action: Move to a jurisdiction that has them, or do the political work to elect a party that will cut them (and correspondingly, services). Spare me, please, the “supply-side” fantasy in which tax cuts generate increased revenue. It’s been tried.

So, tax all income equally?

Maybe. In Canada, the tax on capital gains is half that on salary. The idea is to encourage people to start businesses and invest in other businesses. I’ve created businesses and invested in them too, so it’d be easy for me to say “Look, it works” and maybe I’d be right. But maybe not; I probably would have gone ahead and done those things anyhow. It’s amazing how many tax policies officially aimed at one good end or another seem mostly to result in rich people paying less.

Also I’ve started to hear progressive economists arguing against this kind of thing. And a hard-line policy of “income is income” has the advantage that people don’t pay tax consultants to fool around and try to make one kind of income look like another. So it’s not a slam-dunk.

What’s going to happen?

The politics is interesting. The people lined up against the tax proposals have loud, well-funded voices, and donate lots to political parties, so their concerns are going to get careful attention from legislators. And it’s easy to convince the public that any government tax tactic is a grubby revenue grab.

But at the end of the day the noise is coming from a bunch of fat cats trying to pay less tax than other fat cats, based on what feel like really flimsy arguments. They exude entitlement. I think the politics on this one could go either way.

I recently read Prussian Blue by Philip Kerr. It’s good — a Fifties-Iron-Curtain spy thriller gracefully mashed up with a pre-war murder mystery set in Hitler’s Bavarian country getaway, Berghof. It’s a repeat appearance for Kerr’s Bernie Gunther, an appealingly hard-boiled veteran socialist cop who finds himself working for National Socialist management.

And yep, there are Nazis in this story; the real not tiki-torch flavor. A couple of mega-Nazis, Bormann and Heydrich, and some relatively minor odd fish like Karl Brandt and Gerdy Troost.

Nazis are convenient for a novelist, because they are reliably evil and twisted, so he can economize on characterization and leave room for plot and atmospherics. Having said that, he cuts Troost a little slack; you’d have to be a better historian than me to know whether that’s a travesty or not.

Did I mention atmospherics? You’ve come to the right place. While substantial parts of the novel aren’t at der Führer’s country digs, the ones that are bask in deep you-are-there weirdness, and (I thought) a pretty deep take on what it’d be like for the yokels in a pretty backwater when the Fascist architects come to put up palaces. Some, predictably, react a lot better than others. But Kerr doesn’t pretend the presence of any serious resistance.

Hey, and here’s a fascinating little sidelight: The lead contractor for the Berghof construction was a company called Polensky & Zöllner; some part of which apparently still exists, albeit in Abu Dhabi. Their motto: “All knowledge comes from experience”. And yeah, they’ll still build you a villa.

Oops, I got distracted. This is a fine piece of writing, intense and atmospheric and instructive and just really good fun. Recommended.