Holding ISPs accountable: Let's build an app to monitor net bias

In the continuing fight for holding our broadband providers accountable for their actions, our front line weapons will be the cloud and the Internet of Things.
Written by Jason Perlow, Senior Contributing Writer

Video: Net neutrality is gone -- welcome to biased net

A New Year and a new relationship with our broadband and wireless services providers begin for Americans as we wake up to an internet post-net neutrality.

This is the internet in the age of "net bias."

Read also: Welcome back, sneakernet: Why net neutrality repeal will drive us to the edge

Since the repeal of the net neutrality rules enacted by the Obama administration on Dec. 15, the Tier 1 service providers have all issued statements along with the major trade groups representing them that they don't and will not block or throttle connections to content and sites.

They have said, emphatically, that they won't engage in most forms of paid prioritization -- the practice of charging sites (and end-users) more for faster delivery of streams and downloads.

Will they stick to these promises? For the next year, while the FCC is under legal challenge, they are likely to keep a low profile and not make significant changes. But, past that, it's possible subtle changes may occur, which will be slowly rolled out in the hopes that consumers don't notice them.

When that happens we must prepare to fight back. We must empower ourselves as end-users and stop the content providers that serve us from being discriminated against. We will hold these huge telecoms and regional broadband providers accountable for their actions when they do eventually cross an unacceptable line.

The cloud and the Internet of Things

How do we hold them accountable? We use the best weapons in our arsenal: The cloud and the Internet of Things.

We must have actual visibility into the performance of our service providers -- in essence, we need the network equivalent of a weather map to tell us if we are actually getting what we pay for.

We need to know, using a consumer-friendly visualization and user interface, if the ISP is neglecting its infrastructure or that an event of some kind is occurring -- such as whether it is DDoS or malware-based.

We need to know if port blocking, throttling, or QoS is being applied to and from where. We need to be able to point out that it isn't uniquely something wrong with our on-premises equipment or that other broadband consumers are experiencing similar issues, so that, effectively, we can construct a report card and a record of the behavior.

We need a real-time leaderboard that ranks how the providers perform against their peers when it comes to service responsiveness and openness and indicates when they are clearly dropping the ball.

Today, there is no easy way for a consumer to correlate anything when events do occur. Nor do they necessarily know it is happening or what is causing it.

Yes, there are open-source tools out there like Nagios that you can install on a server to monitor publicly available infrastructure with pings and such, but there's no way to know if other customers are experiencing the same problem.

It's also not a consumer-friendly piece of software, and you need to have network and systems expertise to set it up.

We can't depend on the government

What we need is an easy to configure app or a service that runs on all of the major OS platforms (Windows, Mac, Android, iOS, and Linux) and on our IoT devices/premises equipment that provides the needed telemetry for correlation and reporting in the cloud.

Obviously, we can't depend on the government to build this and hold broadband and wireless services providers accountable. The industry needs to build this.

I see this as a major priority for the Apples, Amazons, Googles, and Microsofts of the world -- for a number of reasons.

First, they build the platforms we use, the PC, and mobile operating systems. They also run the public clouds these platforms use to consume content. They also host many of the content providers on their public clouds.

They have their own reputations they need to keep pristine, and they need to keep their enterprise broadband provider QoS agreements and SLAs with these very same ISPs in check, too.

He-said, she-said situation

Today, if the broadband consumer experiences any kind of throttling or slowdown, it's very much a he-said, she-said situation. A lot of finger pointing with no substance. The broadband provider is going to be inclined to blame it on the on-premises equipment at the residence -- or they'll blame it on the content provider.

If we had the ability to see what was actually going on in real time, across thousands or millions of representative devices, we can catch the broadband providers red-handed when the inevitable transgression occurs.

I also see this as a tool for increasing competition between the broadband providers. The ones with cleaner records and proven performance will be able to point this out in their advertising and marketing material. And the ones who screw up will be called on the carpet by their competition and consumer protection/advocacy groups (especially if they block the telemetry).

How do we build this? I envision this as a combination of a cloud service that runs on each of the public clouds -- Amazon AWS, Microsoft Azure, and Google Cloud Platform -- and a telemetry service that runs on your devices.

The more data points we collect on ISPs, the better

The telemetry service, which I see as being a core function built into each operating system platform, would measure performance/responsiveness against a baseline list of popular public websites and services, such as Netflix, Hulu, Youtube, Facebook, Twitter, Skype, and any number of others that a major cloud provider runs.

CNET: In 2017, the FCC made life easier for your internet provider

It could run on your PC, your smartphone, your smart speaker, and other smart devices, or even your router -- or all of them. The more data points we collect on ISPs, the better.

The telemetry service would have to be opt-in. You'd have to agree to turn it on so that you can start collecting a history of your household's and device performance with these services relative to your provider.

You would be able to see individual reports of how your provider is performing against your equipment and also that of your neighbors and national subscribers on a historical basis.

I see this as essentially a version of OOKLA's or FAST.NET's speed tests on steroids -- but this would measure performance on a service and application level versus a snapshot of broadband link speed. And it would be running all the time, sending data to the cloud provider over a secure web protocol.

Each of these cloud providers would then take this telemetry data and display a leaderboard in real time of how all the services providers are ranking against their public cloud services.

Google would run its own for Android and Chrome OS on Google Cloud. Microsoft would run its own for Windows, Xbox, and Office 365 on Azure. Apple would run its own for iOS and Mac in iCloud. Amazon would run its own on its own devices and services in AWS.

Netflix, Twitter, and Facebook can build this into their respective apps for the platforms they run on, or they can tie into the OS-specific collectors/telemetry services via a standardized API.

"Internet weather map"

I believe that the major cloud service providers should then share this metadata/rankings with each other so that the overall composite "internet weather map" can be created as a standardized app/service for each device platform.

Ideally, the collector/telemetry service and the public mirrors of the "net bias leaderboard" should be an open-source effort that is modular and the cloud and platform providers would agree to maintain it as a group effort.

In my opinion, this is the killer app for big data. It is the analytics app everyone wants and it will allow the invisible hand of competition to work the way it is supposed to by providing the visibility consumers need to make informed decisions about who they do business with.

It would allow competitors to determine whether they can challenge poor-performing incumbents in established markets where few choices exist.

TechRepublic: Net neutrality: The smart person's guide

It becomes a much more powerful tool when Google, Amazon, Facebook, Netflix and Microsoft -- as well as any other cloud services provider -- can say as a group representing their customers that a broadband or wireless provider is behaving badly.

If something wacky starts happening, alarms could go off. Notifications could be sent to mobile devices and on your PCs. Your smart speakers, such as Alexa and Google Home, could also be configured to audibly tell you when a "storm" was occurring.

You would know immediately if service is being degraded or when it resumes to normal, or a network-wide security event was occurring.

Today, we do not have these tools at our disposal. But the infrastructure in the cloud and the platforms exist to build this now -- and if net bias is the new normal, we need to have the ability to shame the ISPs as a collective when they cross the line into unacceptable behavior.

Do we need better tools to hold our broadband and wireless providers accountable? Talk Back and Let Me Know.

Previous and related coverage

The real battle for net neutrality begins: The people v. FCC

The FCC will not be getting the last word on the fate of net neutrality.

FCC votes to repeal Obama-era net neutrality rules

The three Republican members of the commission voted to repeal the net neutrality rules.

Editorial standards