Network World: Five cloud costing tools reviewed

Certainly, using a cloud provider can be cheaper than purchasing your own hardware, or instrumental in moving a capital expense into an operating one. And there are impressive multi-core hyperscale servers that are now available to anyone for a reasonable monthly fee. But while it is great that cloud providers base their fees on what resources you actually consume, the various elements of your bill are daunting and complex, to say the least.

Separating pricing fact from fiction isn’t easy. For this article, we looked at five shopping comparison services, including Cloudorado, CloudHarmony’s CloudSquare, CloudSpectator, Datapipe and RightScale’s PlanForCloud.com. Some of them cover a lot of providers, some only focus on a few.

You can read the full review in Network World today here.

Why you need to review your stats regularly

I admit it; I have fallen out of the habit of reviewing my various stats on my websites and other content-oriented places. For many years I dutifully kept track of how my posts were doing, who was commenting, where backlinks were coming from, and so forth.

For some reason, I stopped doing this in the past year. Maybe it was just being lazy, maybe because I had gotten very busy with a lot of very interesting assignments. Maybe it was just old age: I have been writing stuff for more than 25 years, after all.

Well, all of those (and others) aren’t valid excuses. You need to check your stats, and check them regularly. There are lots of interesting things hidden in them that you might not realize, and some of these things can help you delivery better content, or target new audiences, or figure out what you are doing right (and do it more often) or wrong (and avoid or improve).

WordPress’ Jetpack delivers an annual email summary of your blog and its posts: this is a very useful reminder that you need to dive in deeper and see what is going on with your blog (or blogs, in my case). And Slideshare.net also has some great analytics. This service is a wonderful place to post PowerPoints of my presentations. Looking at these analytics, I would have found out:

  • Influence can be found in odd places. A post that I wrote for SoftwareAdvice.com about real-time retail store tracking was picked up by a blogger for the point-of-sale system Vend.com, that brought a bunch of visitors to my site back in the spring when I was quoted by their blogger. Could have been an opportunity to talk more about the subject.
  • Don’t knock the long tail. I am still the leading expert on a very obscure Windows error message: if you were to Google “Windows Media Player error c00d11b1” you will see my post in the first ten or so results. The post has received more than 380,000 views in the more than eight years since I wrote it, and it is still getting comments on my blog and links in the Microsoft forums too. Why is this important? All this traffic on a very specific subject can help raise your Google ranking, and also provide an entry point into your content ecosystem if you manage it properly.
  • My influence beyond North American borders is somewhat quirky. The second most-visited place of origin for my Slideshare.net account is Ukraine, with about half the views from the US over the past year. Again attesting to the very long tail, a good chunk of these views came from a presentation that I posted five years ago on how to set up your first blog and business email. (That kind of makes sense.) For my blog, other popular countries of origin for my visitors were India and Brazil. Don’t forget the rest of the world when you are posting your content and widen your perspective to engage more of these readers.
  • Twitter and Facebook were both important traffic drivers for my blog over the past year. This emphasizes how critical your own social media accounts are and how you need to cross-link posts among them. Combined the two were equal to the traffic brought in from Google organic searches, which is another important element in referring traffic too. Don’t just post blog entries on your blog: I have begun cross-posting my content on LinkedIn Pulse and Medium and they are getting a fair share of views there too. The analytics for those sites could be better though: for example, Medium only allows you to look at month-long intervals at a time and Pulse will only send you static results in regular email summaries.

There are lots of Twitter analytic tools, and some that are quite pricey. One that I like that has a free version is TwitterCounter, including who unfollowed and followed you over time. For example, I got excited this week to see that the actor Taye Diggs followed me (he has been following my wife’s Tweets for some time) but our local mayor dropped me (oh well). You can see the kind of graphs it produces such as this, which to me indicate a fairly steady stream of new followers replacing the drop-offs with a slow overall growth:

twittt

Happy new year and may your stats encourage you to deliver better content in 2015!

Everyone is in the software business

Everyone is in the software business You may not know it, but you are in the software business, no matter what your actual business may appear to be. It doesn’t matter what you produce, whether you are a “bricks and mortar” retailer or a “guys in trucks” distributor, software is where you are going to end up.

virgin-america-logo-1Why is everyone in the software business? Simply because software is become the lifeblood of so many decisions on what a business makes, how it is sold, and how customers are kept happy. All the interesting business operations are happening inside your company’s software. Software is where you can find out if your customers are going elsewhere, if your profits are coming from some new markets, and if your employees are helping or hurting your overall reputation.

As an example, the airline Virgin America has billions of dollars invested in planes and the people that fly them, but their brand lives and thrives based on their mobile and web experience. That experience is all because of the airline’s booking software, and understanding what is happening with that software will make the difference between success and failure for the airline.

Take as another example a national food service distributor. Their business is getting food into trucks, and then getting those trucks to restaurants and other institutional caterers and retail kitchens. The company has had an ecommerce business for the past decade, and a pretty significant one at that. But lately their customers have been shifting their emphasis from calling their sales representatives with the weekly orders and wanting to do more online. The distributor needed to scale up their online business, and also be more data-driven. Rather than letting their truck drivers or regional offices make decisions about distribution, they wanted a single view of their business, and use the changes in their orders and other data to fine-tune their deliveries. This food distributor is now firmly into the software business.

What is driving everyone to software? Several things.

  • The cloud. The days where you had to build your own servers and data centers are over. Post Holdings is probably the largest cloud-only company, and their revenues are in the billions.
  • Everyone wants an online storefront. Just like the food distributor, even the most basic industries are finding out there is value is selling their stuff online.
  • Big Data is getting more familiar. You can now find Hadoop clusters in many traditional Fortune 500 companies. The IT staffers at the giant retailer Sears eventually spun off a side business in helping others get started with Big Data.
  •  Customer experience is king. One way that businesses can differentiate themselves is by paying attention to their customers. This isn’t anything new: Nordstrom’s department stores have been doing this for decades. What is new is a range of software tools to help figure this stuff out.

I will have more to say about this topic, right now I am working on a white paper for a client that will dive into this deeper. Check back here in the fall when I can post a link to it.

New Relic blog: JQuery Foundation’s Dave Methvin Shares his Rules of the Road for Speeding Up Your Website

It’s practically a universal truth: just about every website is too slow. There are many, many reasons for this unfortunate fact, of course, but Dave Methvin, president of the jQuery Foundation, has some practical advice on how to get the biggest speed improvements with the least amount of effort. This is based on an in-person meeting as well as some of his talk that he gave at a programming conference earlier this year.

You can find my article on the New Relic blog here.

Why New Jersey needs to slow down its traffic

It is ironic that the same thing that got Chris Christy in trouble – delaying traffic into Manhattan — is being used by others to build a multi-million business. I am talking about network traffic for stock traders. Perhaps you have seen the stories about them based on Michael Lewis’ latest book Flash Boys.

In the celebrated George Washington “bridgegate” fracas, it was delays introduced by closing various on-ramps to the bridge. In the case of the stock market, it was delays of 350 microseconds that made high-frequency trades more equitable across exchanges.

Wait a minute, come again? Adding in latency is a good thing? Yes, that is another irony of the situation. A company called IEX developed a technique where they slow down the trades so that the exchanges can get the trading requests at the same time. This means that exchanges closer (in terms of connection time) to the Big Apple trading desks can’t trade a few microseconds ahead of the others. The technique IEX developed is basically a big spool of fiber optic cable that is 35 miles long (pictured above), the length it takes a beam of light to traverse 350 microseconds. The problem is that the biggest trading exchanges are located at different peering points in New Jersey, with some being up to 35 miles apart from others, at least in terms of how their packets are routed. Another irony: these locations were chosen so that the exchanges would have as little latency as possible to gain access to the trading data streams (see the map from the New York Times below).

It is probably the first time that anyone was deliberately introducing more network latency to improve their business that I have heard of. Many of us have spent a good chunk of our careers trying to cut down on latency issues: I can remember when I rolled out the first PC local area network application at MegaLith Insurance back in the middle 1980s. The file transfer app that normally took a few seconds to get from one point to another across our mainframe network now took tens of minutes. That wasn’t a good thing.

Programming professor Donald Knuth wrote in one of his seminal books how “programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs.” (Thanks to Dave Methvin of jQuery Foundation for uncovering this quote.) And in the world of high-frequency trading, a company called Spread Networks invested hundreds of millions of dollars building a fiber connection from Chicago to NYC a few years ago, only to find that a microwave network could shave even more time off their latency figures.

So we have the Internet to thank for introducing all kinds of unpredictable latencies into our apps and drive us nuts trying to track down the culprit. Now lower latency has been productized, thanks to the smart guys at IEX. Maybe the next step will be for the traders to actually colocate their offices on top of the major peering points themselves: that could actually work for cutting down on the times of some New Jersey commuters, too.

St. Louis Trains Hundreds of Coders

jimck“Have you ever had the opportunity to work with someone who is the best in the world?’ That question got at the heart of a presentation from Jim McKelvey last night at a rather unusual event that I attended at our newly renovated central library downtown. I’ll get to Jim in a moment, but first I want to tell you the context of the event.

Here in St. Louis, like many areas of the world, we have a coding shortage. There are dozens of companies, some big and some just getting started, that can’t hire good programmers. It isn’t from lack of trying, or resources: they have the money, the open positions, and the need. The problem in the past has been explained that either they can’t find them or don’t know where to look. But there is a third possibility: the coders exist, they just need some training to get started. That is where an effort called LaunchCode comes into play.

For the past several weeks, hundreds of folks have been taking the beginning computer science programming class, CS50 that Harvard offers over the EdX online platform. The class started with more than a thousand participants and is now down to about 300 or so hardy souls who spend anywhere from 20 hours per week or more learning how to code. Each week they gather in our library to listen to the lectures and work together on the various programming problem sets.

David Malan, who went to Harvard himself and is a rockstar teacher, teaches the course. I watched a couple of his lectures and found them interesting and engaging, even when he covers some basic concepts that I have long known. If I had him teaching me programming back in the day, I might have stuck with it and become a coder myself.

The CS50.tv collection online is pretty amazingly complete: there are scans of the handouts, quizzes, problem tests, additional readings, supplemental lectures and so forth. The courseware is very solidly organized and designed and very impressive, from my short time spend looking around.

But here is the problem: while the online class is fantastic, only one percent of the people who take the class complete it satisfactorily. That is almost a mirror image of the completion rate for those attending in-person at the Harvard campus, where 99% of the students finish. I was surprised at those numbers, because Malan goes quickly through his lectures. You have to stop and rewind them frequently to catch what he is doing.

This is where LaunchCode comes into play. The operation, which is an all-volunteer effort, is trying to short-circuit the coder hiring process by pairing the students who complete the course with experienced programmers in one of more than a 100 target tech companies who are looking for talent. They think of what they are doing as going around the traditional HR process and building a solid local talent pool. It is a great idea. I spoke to a few students, many of who come from technical backgrounds but who don’t have current coding experience. They are finding the class challenging but doable.

LaunchCode is also supplementing the CS50 lectures and online courseware with meatspace assistance. They have space reserved downtown for the students to get together and help each other. Some students have actually moved to St. Louis so they could take the class here: that was pretty amazing! LaunchCode has created mailing lists and Reddit forums where students can share ideas. But that isn’t enough, and last night we learned that Malan is coming to town in a few weeks, bring a dozen of his teaching assistants with him for a special evening hackathon for the class participants. Wow. Will that help get more students to finish the class? I hope so, because I want Malan & Co. to make a regular trip here to see the next class, and the next.

The problem with teaching programming is that you have to just do it to become good at it. No amount of academic study is going to help you understand how to parse algorithms, debug your code, figure out what pieces of the puzzle you need and how to organize them in such a way to make more efficient code. You just have to go do “build something” as McKelvey told us all last night.

Back to his question posed at the top of my post. Obviously, he thinks Malan is the best programming teacher in the world. He challenged everyone in the auditorium to think about what questions they would ask Malan when he comes into town, and how they can leverage their time with the master. He used the analogy of when he built his glassblowing studio he was able to spend time with Lino Talgiapietra, a master Venetian glassblower. Last night he once again told the story of how humbling an experience that was and how he was allowed to only ask a single question of the “maestro.” Wow.

McKelvey was very gracious with his time, and answered lots of questions from the LaunchCode students. Many of the questions last night were how the students were going to position themselves to get a coding job once the class was over in a few weeks. McKelvey kept emphasizing that they need to just “rock the class” and not worry about whether they were going to be programming in php or Ruby. “That isn’t important,” he kept saying: just demonstrate to Malan that they could write the best possible code when he comes here in a few weeks.

I have heard McKelvey speak before and last night he was in fine form. Will LaunchCode succeed at seeding lots of beginning coders? Only time will tell. But my hat is off to them for trying an very unconventional approach, and I hope it works.

Comments always welcome here:

Stop Web Scraping With ScrapeDefender

Copying content from the Web can be both a good and bad thing. There are companies that make it easy to scrape public data archives such as ScraperWiki.org that are used by data sciences and journalists to track trends and uncover government abuses. And Google and other search engines use various kinds of scraping algorithms to index and categorize your site, and to ensure that your content is ranked appropriately.

But for the most part scraping is bad news. Chances are good that someone has copied your Web content and is hosting it as their own elsewhere online. This happened with LinkedIn not too long agoscrape dashboard2, where someone picked up thousands of personal profiles to use for their own recruiting purposes. That is a scary thought, indeed.

And lest you think this is difficult to do, there are numerous automated scraping tools that make it easy for anyone to collect content from anywhere, including Mozenda and Scapebox. I won’t get into whether it is ethical to use these on a site that you don’t own the content. Some of these attack sites are very clever in how they go about their scraping, with massive numbers of ever-changing IP addresses blocks to obtain their content.

So what can you do to prevent the bad kind of scraping? There are several companies that try to protect your site from being scraped by a bad actor, including Distill Networks and CloudFlare’s ScrapeShield.

But today’s post is to tell you about another one that goes even further than these two tools called ScrapeDefender. You can watch a screencast video that I just produced here that shows its features.

Scrape Defender is easy to get started with: you just plug in your site’s URL and it will take about a day to look at your site and see where you are vulnerable. When I tried it with my own domain strom.com I was surprised to see it listed 150 different exploits. Some of them have pretty oddball names, such as dripping water or shotgun that show where anyone can come in and grab your content. The service provides a piece of Javascript tracking code that you add to each of your site’s page headers. Once this is in place you can monitor what is going in in near-real time and protect your site against these abusers.

For example, you can view how many pages a potential abusive IP address has visited, any geolocation information, which risk metrics were tripped, what alarms were generated because of this activity and other IP addresses that are owned by the same organization. All that information can help you figure out if your site was suddenly very popular or was being targeted by one of your competitors or someone that wants to steal your content. Their service is Web-based; you bring up your browser and can view these metrics and reports, along with suggestions on best security practice to defend your content too.

The hard part about defending and hardening your site against potential scrapers is that it is difficult to distinguish between a legitimate visitor and an automated bot that is collecting your content. That is the secret sauce of ScrapeDefender: they have looked at thousands of websites to figure out when a bad actor is present, and have code these various behaviors into their system.

You can try Scrape Defender for free, the paid service starts at $79 per month to keep track of a single domain, with more expensive and extensive plans available. It is well worth a look.