While not a basic introduction, this article is a valuable chronicle of some hands-on learnings from using kubernetes. The hand-drawn illustrations are a great addition.
Here is an excellent, in-depth discussion on creating scalable microservices.
In this article, we will look at microservices, not as a tool to scale the organization, development and release process (even though it’s one of the main reasons for adopting microservices), but from an architecture and design perspective, and put it in its true context: distributed systems. In particular, we will discuss how to leverage Events-first Domain Driven Design and Reactive principles to build scalable microservices, working our way through the evolution of a scalable microservices-based system.
Great article on system failures in IT and how groups/people react to them. Here is a summary:
tl;dr: Catastrophic system failures are remarkably common in IT-dependent environments. The reactions to such failures varies but is often some version of blame-and-train. There are a number of problems with blame-and-train but perhaps the most important is it is a form of organizational blindness that forestalls improvement.
- These failures are markers of systemic brittleness, the inverse of resilience.
- The blame-and-train reaction is a diversion, a red herring, and counterproductive; it increases brittleness.
- There are productive reactions to failure but they are difficult to accomplish, especially when the failure has big consequences.
Serverless architecture uses a lot of services — hence why some prefer to call the architecture “service-full” instead of serverless. Those services are essentially elements of an application that are independent of your testing regime.
An external element.
A good external service will be tested for you. And that’s really important. Because you shouldn’t have to test the service itself. You only really need to test the effect of your interaction with it.
Here’s an example …
Let’s say you have a Function as a Service (e.g. Lambda function) and you utilise a database service (e.g. DynamoDB). You’ll want to test the interaction with the database service from the function to ensure your data is saved/read correctly, and that your function can deal with the responses from the service.
Now, the above scenario is relatively easy because you can utilise DynamoDB from your local machine, and run unit tests to check the values stored in the database. But have you spotted something with this scenario? It’s not the live service — it’s a copy of it. But the API is the same. So, as long as the API doesn’t change we’re ok, right?
To be honest, I’ve reached a point where I’m realising that if we use an AWS service, the likelihood is that AWS have done a much better job of testing it than I have. So we mock the majority of our interactions with AWS (and other) services in unit tests. This makes it relatively simple to develop a function of logic and unit test it — with mocks for services required.
Here is a quick summary of what is new in the latest Android Things Developer Preview.
Earlier this month Google announced a partnership with AIY by releasing a co-produced build-your-own Google Home kit. Built on Google’s Raspberry Pi 3 developer board, the kit showcased Android Things’ ever-expanding features, particularly the integration of Voice Kit. Enabling developers to build a proper Voice User Interface (VUI) Voice Kit is an open-source platform which can integrate cloud services such as Google Assistant SDK or Cloud Speech API or simply run similar services directly on the device with Tensorflow – Google’s on-board neural-network.
Google also added some important drivers to the mix – most notably those necessary for implementing Google Assistant SDK on any certified development board. Also in tow is support for Inter-IC Sound Bus (I2S) which has been Added to the Peripheral I/O API. A Voice Kit sample for which is included, aimed at demonstrating the use of I2S for audio.
Developer Preview 4 will also bring new hardware, adding a Board Support Package for the NXP i.MX7D. Also, in a display of Android Things’ scalability, Google has released Edison Candle – a sample of custom hardware which fits modularly with SoM’s (system-on-modules) running the lightweight OS. Code for this sample is hosted on GitHub while hardware design files can be found on CircuitHub.
Things seem to be coming together quite well for Google’s IoT solution. With the 1.0 release of Tensorflow in February and I/O kicking off, we hope to see even bigger strides today.
Microservices Need Architects – An excellent article on the complexity of something with ‘micro’ in it’s name. And, yes, I know and I am here to help with over a decade of experience in service design and enterprise integration skills.
For the past two years, microservices have been taking the software development world by storm. Their use has been popularized by organizations adopting Agile Software Development, continuous delivery and DevOps, as a logical next step in the progression to remove bottlenecks that slow down software delivery. As a result, much of the public discussion on microservices is coming from software developers who feel liberated by the chance to code without the constraints and dependencies of a monolithic application structure. While this “inside the microservice” perspective is important and compelling for the developer community, there are a number of other important areas of microservice architecture that aren’t getting enough attention.
Specifically, as the number of microservices in an organization grows linearly, this new collection of services forms an unbounded system whose complexity threatens to increase exponentially. This complexity introduces problems with security, visibility, testability, and service discoverability. However, many developers currently treat these as “operational issues” and leave them for someone else to fix downstream. If addressed up front—when the software system is being designed—these aspects can be handled more effectively. Likewise, although there is discussion on techniques to define service boundaries and on the linkage between organizational structure and software composition, these areas can also benefit from an architectural approach. So, where are the architects?
One of the last things that was keeping me using Evernote was the web clipper. The recent, nearly month-long outage with the web clipper pushed me to find an alternative. Oddly enough, the alternative was there all the time within Chrome in the form of the Share->Print->Save to (Google) Drive option which renders the current page as a PDF and saves the PDF to GDrive.
I had been a Evernote user since 2009, having moved from del.icio.us and diigo. Evernote added some good features (like the web clipper, PDF upload, OCR, multi-device sync) and things were great. Then they lost the plot. I mean, this was supposed to be a note taking app, but someone decided it needed to have chat built into it (bad idea). Next they messed up the sharing options by polluting the URLs to point at evernote.com instead of the original source. Then the convoluted sharing model that made trying to share anything extremely cumbersome and time consuming. The last straw was the announced plan to add ‘machine learning’ to Evernote which would require user to consent to having people ‘inspect’ their content if you wanted to keep using the service. My question was ‘who needs this?’ It just sounded like they are trying to data mine my content while adding this me-to AI-ish sounding feature.
Along with these missteps, it seemed the Mac OS X client just got slower and slower as more and more irrelevant features where added. For reference, I just started the Mac Evernote client – 4 minutes and 48 seconds later I could finally click on a menu and have it drop down. In comparison, I can open Google Drive and begin using it within seconds.
Dealing with Legacy Evernote Notes
Last August, I started using CloudHQ to migrate my tens of thousands of Evernote notes to Google Drive. After a few hiccups, the (one way) sync process was working flawlessly (I think at the time, I was the largest migration that CloudHQ had ever done, at least with Evernote). It took several months to move all of the notes to Google Drive because Evernote would ‘throttle’ access to CloudHQ, forcing me to re-authorize CloudHQ’s access to my Evernote content.
In September, I downgraded my Evernote account from Premium to Plus. In December, I downgraded to Free. As of January 2017, I am going to upgrade my Evernote subscription to ‘Delete my Account’.
There is a lot of talk these days of the evil of ‘fake news’. As far as I can tell, fake news is just a symptom. The real problem is the ‘easy money’ mentality of the online ad machine. This largely anonymous (and most definitely unregulated) mechanism rewards any and all bad behavior by handing out cash for page views. These days, that means page views creaking from the overload of irrelevant advertising that delivers no value to the viewer but does enrich the bottom-dwellers that plaster ads all over the page. Fake news, clickbait, porn, gossip, real news – at this point they are rewarded equally by advertisers.
Fake news is only the most headline grabbing; there is so much more of this dubious activity festering everywhere and in more subtle ways. Most recently, I have noticed online retailers starting to use unnecessary parcel tracking services ‘to better server their customers’. In this case, to better serve customers bandwidth wasting, unnecessary advertising. I have strongly suggested to some retailers that I do business with to just stick the tracking number for my order in the shipment confirmation email. I don’t want to (or need to) click on a series of links that are awash with advertising just to get to the tracking number that could and should be provided me in plain text in my email.
I can’t tell if companies are just ill-informed or just don’t care that much about customer satisfaction and privacy when it comes to things like this. We recently stayed at a hotel in San Diego that offered as one of it’s ‘customer services’ the ability to text the hotel if something was needed. What was not disclosed was that this service is not operated by the hotel, but by a third party. So, by texting the ‘hotel’ you are (probably unwittingly) providing this third party with your cell phone number, name, info about your stay and who knows what else. That information gets sold immediately and you get nothing for it. Just like the dubious ‘free’ safe in your room that requires you to swipe a credit card to ‘activate it’. As soon as you swipe, some unknown, undisclosed third party now has your credit card number, name and whatever else is encoded on your credit card’s magstripe. That’s right, I don’t need or want your data harvesting in the middle.
Additionally, I have little sympathy for all of the web sites that block visitors if they are using ad blocking software (which has been shown to prevent the distribution of ad-based malware (aka, forbes, businessinsider, wsj, wired, etc). They whine about not getting their vig from online ads but are silent about the 10, 15, 20 trackers and beacons (in the form of cookies and local storage) that they DO profit from that are placed on your system without your knowledge or approval (again ‘to better provide you service’). But, don’t believe me, run a browser extension like Ghostery to see all the garbage that gets placed on your system when you visit one of these cookie cesspools. Alternatively, you can at least click on the site information icon in Chrome and see all the ‘3rd party’ cookies that are placed on your system.
Increasingly, the web is moving away from its roots as a means to easily share information (and actual data) into the realm of the quick buck, ‘publish anything that will generate a click’ crapfest we have now.
Install ad-blocking in your browser and think before (and after) you click.
Happy New Year!
Looking forward to a healthy, safe and prosperous 2017.
I have been saying this for decades. This was one of the things the movie The Siege got absolutely right – in 1998.
This is especially true since they won’t service anything older than 3 years…
- Type docs.google.com into browser
- Select template (optional)
- Start creating doc
Elapsed time: 5 seconds to productivity
Microsoft office for Mac
- Click on Word app icon
- Watch icon bounce in doc for 60-90 seconds (4core system with 16GB of RAM)
- Click to allow access to Microsoft Identity on keychain (x8)
- Wait for Auto Update to run
- Install 2MB update
- Wait for Auto Update to re-run
- Office now wants to download 2.6 GB of updates before continuing.
- Wait for downloads and updates to finish
- Open Google Docs and start typing so you are productive for the next 30 minutes
- Wait for downloads to finally finish
- Grant Admin access so install can continue
- Wait for install(s) to complete
- Close Word so install can complete
- Wait for Auto Update to re-run
- Dismiss Auto Update
- Click on Word App icon
- Watch icon bounce in the doc for 90-120 seconds
- repeat step #3
- repeat step #4
- Create new doc
- Paste info from Google Docs into Word
- Continue editing document created in Google Docs
Elapsed time: 35 minutes to productivity
Tell me again why/how using MS Office docs are so much more productive (or even preferable)…
Self proclaimed futurist tweets an obfuscated link to an ad-encrusted pull quote that links to an article… behind a paywall.
For some the internet has moved from a means to share information and ideas to one that exists solely to generate clicks that have zero information value (well, except to them – ‘ad impressions’ and all that). And, no, I don’t want to sign up for you email-harvesting ‘newsletter’ that you never publish but benefit by selling on my contact information.
Much has been written in the last decade about the ‘Consumerization of Corporate’ IT with the primary example being corporate users wanting to use their smartphones and tablets from home in a corporate ecosystem.
I would argue that the inverse of that trend has started in the last few years. That is concerns that were once firmly in the corporate space are starting to bleed into the consumer space. These include:
- a focus on security for personal devices with more emphasis on firewalls, encryption, SSL, password strength and even two-factor authentication.
- a growing interest and need for analytic and visualization tools for the growing amount of data from wearables and other in-home devices. Currently this is served by one off tools from each vendor with more platforms emerging that are corporate-style integration platforms that take in data from disparate systems and provide a more unified ‘dashboard’ view to consumers.
- additional emphasis on in-home automation and monitoring control systems for everything from thermostats, lighting, locks, motion sensors, flow sensors. Previously, this was the realm of building security groups and manufacturing plants. Automation and monitoring is also driving the previously mention areas of security and analysis.
I guess I am taking a little more cautious/skeptical stance when it comes to the auto-replenishment feature touted by many IoT pundits and vendors. If you aren’t familiar, this would allow a device to determine that you were out of or running low on a given consumable (be it a food item, dish soap or toilet paper) and then order more of it on your behalf.
Here is the problem: the vendor and the device don’t have your best interests at heart and might tend to exaggerate the current state of consumable and (maybe) tend to order more of it more frequently that you might actually need (or want). For example, if you have ever owned an inkjet or laser printer you have probably experienced this already – persistent warnings/notifications to replace a toner or ink cartridge when, in reality, the useful life of the item is much, much longer that you are being led to believe. Heck, I have a laser printer that has been telling be for 13 months that I need to replace the toner. In that time my family and I have printed hundreds of additional pages with this ’empty’ toner cartridge.
Consider also the existing confusion over the meaning of ‘sell by’ and ‘best by’ designations on other consumables (most notably food). What if vendors add a ‘replenish by’ or ‘order by’ date into the mix? Not a great situation for consumers, especially if they have delegated this to an networked device in the name of ‘convenience’.
Another new year! All the best to our blog readers.
Canary.is promote their product as an alternative to a real home security system. Nothing could be further from the reality. Here is the simple truth:
NONE of these things is true with a real home security system.
You must understand that this is basically just a dumb camera unit that requires an internet connection to do anything. There is no local storage or functionality in the unit itself which means if your internet connection is out or slow, the Canary is absolutely useless. As a consequence, it is constantly trying to upload video for analysis (motion detection) — it can do noting on its own. Make sure you set it to ‘privacy’ mode when you are home to cut down on it hammering your wifi.
I had one unit and it sort of worked, I added a second one and BOTH of them stopped working. The more units you add, the more of your upload bandwidth they suck up (and suck they do).
Sadly, tech support is basically useless. At some point they will have you run a test on speedtest.net and if you EVER tell them you experienced an upload speed of less than 1Mbps, then, game over, that is THE problem and apparently the end of their sorry support script. It seems their ‘engineers’ are unfamiliar with data compression, efficient data streaming and error handling algorithms, etc — if you are .01 under 1Mbps (my case), then you are screwed, they won’t support their product (or allow you to return it because it says 1Mbps on the web site). It doesn’t matter if you can facetime or run google hangouts without any glitching, ‘the problem’ is your bandwidth, not their dubious implementation.
Seriously, save your money and/or look for alternatives. This canary is dead in the coal mine.
1) create list in desktop app
2) attempt to share it with my wife; sorry, have to upgrade to paid version for this
3) finally share with wife, she attempts to edit shared list; sorry, she has to upgrade to paid version (screw that)
4) remember a few more items on the go, add to list via mobile app
5) attempt to sync from mobile; get loads of errors – sync fails
6) only way to fix sync error is to copy note contents, delete note an paste contents into a new note
7) repeat from step 2 or just give up
Google Keep experience:
1) create list on tablet using Keep app
2) share with wife; no problem – she has access to it within seconds
3) she needs to add items to the list – no problem; she adds them and they automatically sync with me
4) edit list on mobile – no problem; list automatically syncs
5) both of us run Keep app in grocery store, ticking off items from the list; no problem – list syncs automatically
6) marvel at the superior user experience from Google Keep
7) BONUS: I can set a reminder on the list that is a location; Google Now notifies me when I am near the store.
Evernote just keeps getting worse and worse. About the only thing that keeps me using it is the web clip functionality in the browser. Come on Keep, add that and I can leave Evernote behind.