What Is Missing From Big Data

This is an excellent TEDTalk on what is missing from bigdata (hint: it is the human element).

Why do so many companies make bad decisions, even with access to unprecedented amounts of data? With stories from Nokia to Netflix to the oracles of ancient Greece, Tricia Wang demystifies big data and identifies its pitfalls, suggesting that we focus instead on “thick data” — precious, unquantifiable insights from actual people — to make the right business decisions and thrive in the unknown.

An interesting (but not too surprising) stat from the intro is that 73% of all bigdata projects deliver no value.

Turn Off All Your Push Notifications

Really? Someone had to write a 500 word ‘article‘ about what should be common sense?

There’s a solution, though: Kill your notifications. Yes, really. Turn them all off. (You can leave on phone calls and text messages, if you must, but nothing else.) You’ll discover that you don’t miss the stream of cards filling your lockscreen, because they never existed for your benefit. They’re for brands and developers, methods by which thirsty growth hackers can grab your attention anytime they want. Allowing an app to send you push notifications is like allowing a store clerk to grab you by the ear and drag you into their store. You’re letting someone insert a commercial into your life anytime they want. Time to turn it off.

The New Razor and Razor Blade?

It seems like ‘free’ access to large datasets is the new giveaway/razor in the hopes that revenue will be generated by usage of the AI/analytics tools/razor blades that are co-hosted with the datasets.

Welcome to Our Startup Where Everyone is 23 Years Old Because We Believe Old People Are Visually Displeasing and Out of Ideas 


This is hilarious because it is true. I’ve seen so many ‘startups’ spend a huge amount of money and effort trying to imitate the trappings of a startup rather than having original ideas and actually producing something. Here is a sample (more at the link above):

Hello, and welcome to our startup. We hope you’re enjoying your complimentary snifter of vaporized coconut water. Once you’re done, please place the glass into one of the blue receptacles around the office, which will send the glass to be washed and dried. Do not place it into one of the red receptacles. The red receptacles take whatever you put inside of them and launch it into space.

If you look to your left, you’ll see one of our employees using a state-of-the-art ergonomic sleeping desk. Most startups have standing desks, but we have sleeping desks, dancing desks, and even skateboarding desks. The skateboarding desks are just large skateboards you can use to skate around the office. Be careful while skating, though, because we don’t offer any sort of medical insurance, since our benefits budget all goes toward cool desks.

Volvo’s Electric Car Announcement

I think that Volvo’s announcement regarding electric vehicles (EVs) has been largely misunderstood or mis-reported as them stating that they will only have EVs by 2020. Actually, they stated that they will only design and release NEW EVs after that date. The existing stable of gasoline powered vehicles will continue to live on past 2020.

Reactions To Failure

Great article on system failures in IT and how groups/people react to them. Here is a summary:

tl;dr: Catastrophic system failures are remarkably common in IT-dependent environments. The reactions to such failures varies but is often some version of blame-and-train. There are a number of problems with blame-and-train but perhaps the most important is it is a form of organizational blindness that forestalls improvement.

Three things:

  1. These failures are markers of systemic brittleness, the inverse of resilience.
  2. The blame-and-train reaction is a diversion, a red herring, and counterproductive; it increases brittleness.
  3. There are productive reactions to failure but they are difficult to accomplish, especially when the failure has big consequences.

The Disease of Superfluous Intermediation 

There is a lot of talk these days of the evil of ‘fake news’. As far as I can tell, fake news is just a symptom. The real problem is the ‘easy money’ mentality of the online ad machine. This largely anonymous (and most definitely unregulated) mechanism rewards any and all bad behavior by handing out cash for page views. These days, that means page views creaking from the overload of irrelevant advertising that delivers no value to the viewer but does enrich the bottom-dwellers that plaster ads all over the page. Fake news, clickbait, porn, gossip, real news – at this point they are rewarded equally by advertisers.

Fake news is only the most headline grabbing; there is so much more of this dubious activity festering everywhere and in more subtle ways. Most recently, I have noticed online retailers starting to use unnecessary parcel tracking services ‘to better server their customers’. In this case, to better serve customers bandwidth wasting, unnecessary advertising. I have strongly suggested to some retailers that I do business with to just stick the tracking number for my order in the shipment confirmation email. I don’t want to (or need to) click on a series of links that are awash with advertising just to get to the tracking number that could and should be provided me in plain text in my email.

I can’t tell if companies are just ill-informed or just don’t care that much about customer satisfaction and privacy when it comes to things like this. We recently stayed at a hotel in San Diego that offered as one of it’s ‘customer services’ the ability to text the hotel if something was needed. What was not disclosed was that this service is not operated by the hotel, but by a third party. So, by texting the ‘hotel’ you are (probably unwittingly) providing this third party with your cell phone number, name, info about your stay and who knows what else. That information gets sold immediately and you get nothing for it. Just like the dubious ‘free’ safe in your room that requires you to swipe a credit card to ‘activate it’. As soon as you swipe, some unknown, undisclosed third party now has your credit card number, name and whatever else is encoded on your credit card’s magstripe. That’s right, I don’t need or want your data harvesting in the middle.

Additionally, I have little sympathy for all of the web sites that block visitors if they are using ad blocking software (which has been shown to prevent the distribution of ad-based malware (aka, forbes, businessinsider, wsj, wired, etc). They whine about not getting their vig from online ads but are silent about the 10, 15, 20 trackers and beacons (in the form of cookies and local storage) that they DO profit from that are placed on your system without your knowledge or approval (again ‘to better provide you service’). But, don’t believe me, run a browser extension like Ghostery to see all the garbage that gets placed on your system when you visit one of these cookie cesspools. Alternatively, you can at least click on the site information icon in Chrome and see all the ‘3rd party’ cookies that are placed on your system.

Increasingly, the web is moving away from its roots as a means to easily share information (and actual data) into the realm of the quick buck, ‘publish anything that will generate a click’ crapfest we have now.

Install ad-blocking in your browser and think before (and after) you click.

The State of the Internet – 2016

Self proclaimed futurist tweets an obfuscated link to an ad-encrusted pull quote that links to an article… behind a paywall.

For some the internet has moved from a means to share information and ideas to one that exists solely to generate clicks that have zero information value (well, except to them – ‘ad impressions’ and all that). And, no, I don’t want to sign up for you email-harvesting ‘newsletter’ that you never publish but benefit by selling on my contact information.

IoT and Auto-Replenishment – A Good Thing?

I guess I am taking a little more cautious/skeptical stance when it comes to the auto-replenishment feature touted by many IoT pundits and vendors. If you aren’t familiar, this would allow a device to determine that you were out of or running low on a given consumable (be it a food item, dish soap or toilet paper) and then order more of it on your behalf.

Here is the problem: the vendor and the device don’t have your best interests at heart and might tend to exaggerate the current state of consumable and (maybe) tend to order more of it more frequently that you might actually need (or want). For example, if you have ever owned an inkjet or laser printer you have probably experienced this already – persistent warnings/notifications to replace a toner or ink cartridge when, in reality, the useful life of the item is much, much longer that you are being led to believe. Heck, I have a laser printer that has been telling be for 13 months that I need to replace the toner. In that time my family and I have printed hundreds of additional pages with this ’empty’ toner cartridge.

Consider also the existing confusion over the meaning of ‘sell by’ and ‘best by’ designations on other consumables (most notably food). What if vendors add a ‘replenish by’ or ‘order by’ date into the mix? Not a great situation for consumers, especially if they have delegated this to an networked device in the name of ‘convenience’.

Creating a shopping list: Evernote vs Google Keep

Evernote experience:

1) create list in desktop app
2) attempt to share it with my wife; sorry, have to upgrade to paid version for this
3) finally share with wife, she attempts to edit shared list; sorry, she has to upgrade to paid version (screw that)
4) remember a few more items on the go, add to list via mobile app
5) attempt to sync from mobile; get loads of errors – sync fails
6) only way to fix sync error is to copy note contents, delete note an paste contents into a new note
7) repeat from step 2 or just give up

Google Keep experience:
1) create list on tablet using Keep app
2) share with wife; no problem – she has access to it within seconds
3) she needs to add items to the list – no problem; she adds them and they automatically sync with me
4) edit list on mobile – no problem; list automatically syncs
5) both of us run Keep app in grocery store, ticking off items from the list; no problem – list syncs automatically
6) marvel at the superior user experience from Google Keep
7) BONUS: I can set a reminder on the list that is a location; Google Now notifies me when I am near the store.

Evernote just keeps getting worse and worse. About the only thing that keeps me using it is the web clip functionality in the browser. Come on Keep, add that and I can leave Evernote behind.

Learn How To Learn And Stop Blaming Your Education

I saw this whiny article in the Washington Post that was just begging for a response. The gist of the article is that the author got a Computer Science degree and wasn’t given his dream job out of the gate. This then becomes an indictment of the education system rather than the typical sniveling millennial i-wasn’t-given-the-world-without-having-to-work-for-it screech that it is. Let’s take a look at some quotes from the posting:

My college education left me totally unprepared to enter the real workforce. My degree was supposed to make me qualified as a programmer, but by the time I left school, all of the software and programming languages I’d learned had been obsolete for years.

The think this belies a misunderstanding of how higher education works – it is not what you are given, it is what you do with it (the whole learning how to learn thing). It is as if he expects to read a book on swimming but never gets into the pool; and ‘surprise’ he really can’t swim because he put no effort into applying the learnings. Also, if ‘all of the software and programming languages’ were obsolete, what were they teaching? FORTRAN?, RPG?, Visual Basic?

To find real work, I had to teach myself new technologies and skills outside of class, and it wasn’t easy.

Poor you. You should have been doing this all along. The Computer Science curriculum should be teaching you fundamental concepts in how computers work, programming concepts and techniques that can be applied across specific programing languages, databases and platforms. Actually, it is a bit shocking how many recent CS grads don’t have a grasp of fundamentals.

Businesses aren’t looking for college grads, they’re looking for employees who can actually do things – like build iPhone apps, manage ad campaigns and write convincing marketing copy. I wish I’d been taught how to do those things in school, but my college had something different in mind.

Businesses are indeed looking for those things, but they are looking for people who can learn and grow and apply what they have learned in the past. If you have a CS degree and can’t figure out how to write an iPhone app you either had a horrible curriculum or slept through most of your class time. The fact that you weren’t specifically trained for that is not a problem with your education. Rather it is a failure to apply what you should have learned.

At least 90 percent of my college education (and that of so many others) boiled down to pure terminology, or analysis of terminology. My success in any given class was almost wholly based on how well I could remember the definitions of countless terms – like the precise meaning of “computer science” or how to explain “project management” in paragraph form, or the all-too-subtle differences between marketing and advertising.

Wow. Ok. So, if that percentage is accurate, I can see why you can’t get a job. When I got my CS degree (many moons ago) that was maybe 1% of what we were taught.

To me, this is the root of our college problem: The average college student is paying $30,000 a year for the chance to learn valuable skills from professors who haven’t had the opportunity to learn those skills themselves. Maybe it’s a crazy idea, but if you’re going to spend all that money for a college education, shouldn’t you expect to learn real-world skills from people who know what they’re doing?

This seems excessively harsh and a bit misguided. If you want to be learning what is new and trendy, go to a conference, a user group, or actually talk with people who are doing interesting things. By the time those things get packaged up into an approved curriculum, the technology might be on the stale side. But, again, if you don’t understand the fundamentals, you are not going to be able to effectively apply new technology and concepts. No one can give that to you at any price.

Solving the issue of inexperienced teachers may be even simpler: have schools relax academic requirements for professors and focus far more on hiring effective businesspeople. With a little more leeway, academically-minded candidates will have more freedom to gain job experience, and schools may even attract more talent directly from the business world. Success in business and success in the classroom are certainly different things, but I’d wager that it’s a lot easier to show an accomplished businessperson how to teach than it is to show a teacher how to be an accomplished businessperson.

So it sounds like what you want is for all universities to be trade schools, focused on cranking out very specific skills and techniques rather than more broadly educating students and preparing them to apply a wide set of competencies to a range of problem domains. This sounds a bit like the certification trap from the 90s – go get a very narrow, often vendor specific certification but still have no practical experience in applying that knowledge. When that vendor falls out of favor, you are a bit stuck unless you can teach yourself the reasoning and abstraction skills you would have learned in college.

To steal the trite closing from the original article: But what do I know, I have been happily applying my Computer Science degree for nearly 30 years with technologies, programming languages and platforms that never even existed when I graduated.

Algorithm Maps The Most Beautiful Route To Where You’re Going

I would love to see (and use) this in more locations. Sadly, it will likely be quickly perverted to route visitors to/near shops and other unattractive locales.

If you want to find the most scenic route to get somewhere, there may soon be an app for that. Daniele Quercia and colleagues at Yahoo Labs in Barcelona have come up with a way to create a crowd-sourced measure of a city’s beauty, and made an algorithm to find the prettiest way to get from one point to another. “The goal of this work is to automatically suggest routes that are not only short but also emotionally pleasant,” the scientists told Technology Review:

Quercia and co begin by creating a database of images of various parts of the center of London taken from Google Street View and Geograph, both of which have reasonably consistent standards of images. They then crowd-sourced opinions about the beauty of each location using a website called UrbanGems.org.
Each visitor to UrbanGems sees two photographs and chooses the one which shows the more beautiful location. That gives the team a crowd-sourced opinion about the beauty of each location. They then plot each of these locations and their beauty score on a map which they use to provide directions.

“Useful” Russian Words With No English Equivalent

I am not so sure how ‘useful’ these examples are. Is the fact that most of them have negative connotations a reflection of the person who curated them or of the Russian language?

This was one of my favorites:

переподвыподверт (‘per-e-pod-‘voy-pod-‘vert)
Reddit user deffun on /r/doesnottranslate defined this noun as “to do something in a complex, incomprehensible way.”

The word kind of embodies itself, as it has four prefixes including one that repeats itself twice.

Thoughts on APIStrat Panel on The Future of the Internet

About a month ago, I attended the API Strategy and Practice conference in San Francisco. Overall, a pretty good conference and, as always, much of the value was in connecting with people between and after sessions.

One panel discussion was concluded with the question ‘what is the future of the internet?’. The responses seemed to fall into two categories 1) code/APIs everywhere and/or 2) intelligent consumption and composition of the available APIs.

I wanted to point out that the first category of thought reflected the view of a (now defunct) little technology outfit just down the highway who voiced the credo of ‘the network is the computer’. They had a great spec/implementation for a technology called JINI that very much reflected the philosophy of ‘be a node and not a hub’ – inherently scalable and cluster-able running practically anywhere.

The second group reminded me of the great AI gold rush in the mid-90s, when ‘intelligent agents‘ were going to manage all our personal data and book travel and plan meetings based on all of the metadata that we surround ourselves with. Companies were funded and failed trying to deliver on this vision (General Magic anyone?). Perhaps it was an idea before it’s time and enough has changed and opened up that it might work this time. We shall see.

Women And Tech Events

Not even sure where to start with this psycho-babble rant I just read.

To me, the whole thing reads like an insecure individual trying to justify their warped world view by blaming it on everyone else. Sure, I know that there are bad things that happen to women at tech conferences (and elsewhere) and that is stupid and inexcusable. But, to use that as justification for the view that *all* men are rapist/gropers/whatever is also stupid and inexcusable. The simple fact is that if you are looking to be offended, you will find offense in everything and everybody around you. Especially, if you play both sides of a situation: if someone engages with you, it is for strictly sexual purposes and if they don’t then they are *obviously* denigrating you because you are a female. There is no good way out of that spiral other than recognizing that the premise is setup for self fulfilling distrust.

Being guilty of whatever darkness is in another person’s head is just raw prejudice with a unhealthy dose of over-generalization/labeling. Sure, lets play the game (fill in the blanks): All ______ are lazy. All _____ are cheap. All _____ are bad drivers. And all men at tech events are misogynistic predators. Right.

I have some recent evidence on this front. I was at a tech conference in San Francisco last month and had some fantastic conversations with some of the female attendees there (and the male ones as well). Topics ranged from privacy, genetic testing, pregnancy(!), hypermedia, security concerns for services, agile practices, gardening, coding standards and whisky. No one was groped or molested or talked down to. Oddly enough, at the drinks reception on the last night, I did have a woman approach me several times and try to invite herself back to my room (which I gently but firmly declined). Do I think all women at tech events are horn-dogs because of that? No, not for one minute.

For an excellent exploration of learning to be offended, see this article at NPR. The referenced article is coming at this topic from race rather than gender, but I think it resonates with many of the points I made.

Internet of Things

I am a little wary of the vague but hype-intensive discussions around the Internet of Things (IoT). I am particularly leery when I ask a pundit in the area ‘what *specifically* is IoT going to have the biggest impact on?’ The answer tends to meander along the path of ‘it will change everything’ and ‘there isn’t anything you can’t do with it’. Right. Sort of reminds me of the 90s-era hyperbolic proclamations about Object Oriented databases and how they were going to change everything. As one wag rightly summarized that hype: ‘Object oriented databases are a billion dollar market with no customers’.

SCADA, RFID, SNMP, RPC, etc – didn’t all of these come with the same set of snares and delusions that seems to surround the IoT piper? I fear the only thing that is different this time around is that IoT is paired with the equally rabid running mate ‘Big Data’ that is desperately trying to find a problem to solve and in so doing might encourage the accumulation of whatever data from IoT that it can take on.