Internap was back at GDC last week, and this time we had both a session (with our own Adam Weissmuller and Todd Harris from Hi-Rez Studios) and a booth at the show floor. It was very exciting to interact and learn from the gaming community and listen to their needs and wants, so we can make it easier for their games to launch and flourish. It was also a great opportunity to try out the latest tech, like the VR Headsets from PS4 and Oculus Rift, recently purchased by Facebook.

Here are some of the highlights from our GDC experience:

The morality of F2P
Free-to-play games continue to grow in number, and the way players are monetized has come under scrutiny due to what many players feel are unfair practices. During the conference, much of the F2P talk was focused on shifting people’s perception of the model by assessing whether it hinders gameplay, and the idea of using positive rewards (a.k.a. purchases as a reward) rather than negative ones, where players have to pay when they fail or to try again. But how much is too much? And how can developers still benefit from a F2P system while still keeping their players happy and engaged? The answer seems to lie in creating games that allow players to spend more time in the game without highlighting the monetization aspect of it.

How Is game publishing changing?
Publishing has undergone constant change from the days of the old big publishing houses, and that continues to be true now that developers have a wider array of options that allow them to distribute their games. Some publishers have become developers and vice-versa while most small developers have turned to app stores to find the right audience. Although none of this is breaking news, what is noteworthy in the publishing scene is that regardless of popular perception, app stores remain a small yet growing publishing segment, and there is increased consolidation in the market with companies like Tencent leading the charge (they currently own Riot and Epic games among others).

Games in the cloud
Using the cloud to develop and deploy games has been an ongoing trend for several years, and as expected, VMs have continued to become faster and more efficient in order to meet increasing demands. Currently, some cloud and hosting providers are offering specialized cloud instances whose specs may cater more strongly to game developers, while other developers are turning towards bare-metal cloud to find the performance they seek. Additionally, platforms like Unity are entering the cloud space to help developers monetize and market their games while others like Microsoft, and their newly unveiled DirectX 12, are introducing APIs to improve CPU and GPU usage.

With all the developments happening in the world of gaming, GDC was a great opportunity to interact with the gaming community and keep up to date on the latest trends.

Check out our presentation from GDC with Hi-Rez Studios, Building a Scalable Infrastructure Platform for SMITE, Global Agenda and Tribes: Ascend.

The annual Game Developers Conference is always an incredible experience, and 2014 is no exception. The Internap team is proud to show our support for game developers and publishers, and also to be part of the buzz around new games and the latest innovations in gaming technology.

As the launch of the new multiplayer online battle arena game SMITE dawns, we teamed up with Hi-Rez Studios to present a session on gaming infrastructure at the conference. Todd Harris, Co-founder & COO of Hi-Rez Studios, discussed the positive impact of bare-metal cloud on the online gaming user experience while providing the company with a more flexible and cost-effective IT infrastructure. Having a scalable platform has been instrumental for Hi-Rez Studios to support their rapidly growing user base. Internap is proud to provide the high-performance cloud solution that helps Hi-Rez Studios achieve success. Learn more about testing, developing and deploying online games.

Be part of the SMITE launch
If you’re going to be in Atlanta the weekend of March 28, check out the SMITE Launch Tournament. Worldwide SMITE teams will converge at Center Stage in midtown to battle it out in true March Madness-style competition, complete with a “Release the Bracken!” bracket challenge.

To learn more about Hi-Rez Studios, download the case study.

Watch Mark Zagorski, CEO, and Brent Keator, Director of Infrastructure for eXelate, discuss how Internap helps accelerate global data delivery and real-time application deployment of the eXelate platform.

eXelate is a data technology company that strives to make digital media more relevant for consumers and more effective for advertisers. The company requires a strong infrastructure backbone to ingest and manage trillions of data points on consumer behavior each month.

Internap’s top-notch technology, including bare-metal servers, provides a high-performance cloud solution that can grow and scale to accommodate eXelate’s global expansion. As a result, the eXelate platform enables marketers and data providers to make smarter digital marketing decisions.

Download the complete eXelate case study here.

While many transit agencies still rely on collecting cash from riders, that model is fast disappearing as a result of advancements in the area of fare collection for mass transit. Vix Technology, a global provider of integrated transit and mobility systems, continues to deliver an exceptional experience for public transit commuters through its innovations in contactless payment technologies.

With worldwide headquarters in Australia and North American headquarters in Seattle, Vix Technology delivers hardware and software solutions for electronic fare collection (EFC) systems. Vix recently received the Transport Ticketing Technology of the Year award for their latest product innovation, designed for Salt Lake City’s UTA (Utah Transit Authority). This system gives transit riders a variety of ways to pay their fares, including student and employee ID cards, bank-issued credit cards, ISIS mobile wallet and Google Wallets, in addition to the new FAREPAY pre-paid contactless card issued by UTA. The ability to accept multiple payment formats differentiates Vix Technology’s customers from other transit systems.

With features such as mobile account management and lost card protection, the UTA FAREPAY system creates value for riders and transit agencies alike. In order for Vix to provide a seamless, integrated commuter experience, they require an IT infrastructure that can handle large numbers of transactions in multiple payment formats. In Seattle alone, the ORCA system processes more than 100 million transactions per year.

Prior to transitioning to Internap, Vix leveraged dedicated colo cabinets in local data centers. In addition to purchasing and deploying the hardware, Vix also managed the environment. Not only was it expensive to purchase hardware up-front, but having dedicated operations and support staff available 24/7 to monitor the environment was also costly.

In its desire to move away from this model and evaluate options for outsourced hosting, Vix sought a more affordable solution that provided superior support, physical security and offered a highly scalable and flexible service. After evaluating three local companies, Vix selected Internap as the best fit.

Today, Vix Technology uses Internap managed hosting service via its Seattle and Santa Clara data centers. The Internap solution met Vix requirements in three main areas:

Security
With strict regulations governing the privacy and storage of personal information and processing financial transactions, Vix requires complete visibility and control over how passenger data is managed. Internap’s infrastructure provides a secure, reliable environment to process payments and maintain data integrity. Vix Technology can leverage the benefits of cloud without the security and performance concerns of being on a shared platform with other users.

Scalability
Internap’s flexible infrastructure provides the scalability of cloud computing in a dedicated, secure environment. Scalable storage solutions ensure that Vix Technology can accommodate increased amounts of data and traffic as business demands change. This is especially important for transit agencies planning to scale their services and expand ridership levels. With Internap’s global data center locations, Vix Technology can support service expansions across multiple countries.

Support
With support from Internap’s Network Operations Center (NOC), Vix Technology can focus on meeting the needs of their customers instead of worrying about system maintenance. Certified Internap engineers are available 24/7 to provide support across worldwide locations and help ensure high availability.

Vix Technology’s innovations also give transit agencies the ability to analyze rider data, enabling them to predict and adjust routes and schedules accordingly. Analytics such as these are critical to the planning of mass transit systems in the future.

Whether you’re travelling by bus, train or ferry, Vix Technology is transforming the way people connect and commute. Through an innovative technology journey that includes reliable, scalable infrastructure from Internap, Vix continues to make mass transit more convenient for riders and less expensive for transit agencies.

Over the past week or two, the Internet has been abuzz with coverage of the deal between Netflix and Comcast. The various themes and technical concepts discussed were hardly news to me; I first covered this subject a little over three years ago as a member of the founding technical team at Voxel, which was purchased by Internap in 2011.

Though the names and congestion points have changed somewhat, a lot remains the same, including the media’s inability to comprehend or explain how traffic is exchanged at scale on the global Internet: “peering,” in the vernacular. This is not to call any journalists lazy or irresponsible; rather, peering is a highly specialized pocket of network engineering knowledge, and if you’re not working in its trenches, it’s easy to fall behind. That most engineers “in the know” are bound by strict NDAs with their employers and/or connectivity partners doesn’t help the situation; fortunately, my employer is transparent enough to allow me to share a few words of wisdom.

Settlement-free peering is dead! Long live settlement-free peering!
Before continuing further, it is important to define terminology. As explained in our technical data sheet on peering:

Peering is a connectivity method where two networks establish a direct IP connection between their networks, bypassing any third-party networks or middlemen. The most popular form of peering is known as “settlement-free peering”, where two networks agree to exchange traffic with one another directly without any form of compensation. Another form is “settlement-based peering” or “paid peering”, where two networks agree to exchange traffic directly, and compensation is involved.

Many news articles and blogs incorrectly state that settlement-free interconnection has gone the way of the dinosaur, and now content providers must pay access providers to reach customers directly. Though there are certainly well-documented examples of money changing hands, it is not fair to describe any recent announcements as a change in precedent, or even the norm.

Pretty much since inception, Verizon, AT&T, Time Warner Cable and Comcast have always required money for exchanging traffic with Internet content producers and their bandwidth suppliers (there were regionalized and M&A-related exceptions: for example, SBC peering more pervasively before the M&A which made them the “new” AT&T; likewise Adelphia prior to the fall of the Rigas family empire). This trend has continued given the expanding monopoly or oligopoly positions in the access market.

At the same time, it is important to consider a tier of networks below the “big four”. Many thousands of networks, representing both big access and big content alike, agree to exchange traffic in a settlement-free manner. Peering policies exist not to restrict data flow, but to ensure that a base set of technical standards are maintained; for example, to make sure that networks maintain a professionally staffed Network Operations Center (NOC) responding to security and operational issues, or that sufficient traffic is exchanged to justify the administrative burden of configuration or scaling router ports. I submit that these exchanges comprise the norm, not the exception. They have garnered little fanfare in the media because they continue to scale in a reliable and customer-friendly manner.

While financial terms of settlement-free peering agreements are usually confidential, there are exceptions. The Netflix Open Connect Program is a settlement-free arrangement of sorts: access providers agree to house Netflix servers in their datacenters, or connect to Netflix at carrier-neutral exchange points; in turn, their customers are granted direct access to Netflix’s content library, bypassing third-party intermediary networks. Notable participants include SuddenLink, Cablevision, RCN, and Sonic.net. Why would these access providers allow Netflix, who reportedly represents over 30% of the Internet’s traffic during peak hours, to abuse their pipes for free, when others have charged tolls? It’s really quite simple: their customers benefit.

I live in a pocket of northern New Jersey where Cablevision and Verizon are options for broadband Internet. Though Verizon has the upper hand in the “speeds and feeds” arms race, their congested peering makes for a sub-par user experience: with congestion on a large portion of their peering connections with other ISPs, Netflix and YouTube are hardly usable between 6:00 PM and midnight. In contrast, though limping along with its slower DOCSIS 3.0 access technologies, Cablevision runs its edge connections free of congestion, allowing me to stream cat videos or “House of Cards” (Season 2, Episode 1? Just wow!) in beautiful 1080p. Cablevision has successfully proven the value of a more open peering policy as a competitive differentiator.

Ratios make the world go round!
All the same, telco die-hards and their sympathizers will continue to argue that settlement-free peering doesn’t add up. To them, large content producers are abusing access networks by lumping traffic on their pipes, whether or not this is the same content requested of their revenue-generating customers. Care and feeding of large-scale networks does not come cheap, and everyone needs to pay their fair share in network upgrades. It is also believed that traffic ratios should factor into economic settlement: if a network is generating traffic predominantly towards a network, rather than receiving traffic from it, then it should bear the majority of the cost burden.

Some additional food for thought:

• Counter to the above wisdom, it is believed that scaling a subscriber’s upstream bandwidth on a DOCSIS-based cable network costs approximately five times as much as scaling a user’s downstream bandwidth. Shouldn’t backup operators, and other services with “pull heavy” traffic profiles, be the ones truly viewed as abusers?
• Paid peering is said to level the playing field by allowing a content provider to offset an access provider’s costs of servicing heavy traffic flows. Do transit/peering costs at carrier-neutral datacenters, where ISPs and content providers connect, have a bearing on costs of scaling the last mile? Has there been any open disclosure of what these costs even are?
• Relating Internet traffic to conventional telecom terms (as has been common over the past week or two), why should IP traffic be viewed as “sender pays” when “Bill and Keep” is a more apt analogy?
• Customer support is a major expense for broadband ISPs, and it is said that a single truck roll (service call) can move a subscriber into the red for a full year. I don’t believe any ISPs maintain statistics on truck rolls incorrectly ordered for congested peering; if they did, they might find that their yearly OpEx in misdiagnosed support incidents dwarfs any possible revenues in paid peering.

A rose by any other name
A key tenet of the FCC’s Open Internet Order (December 2010) is “no blocking” summarized as:

ii. No blocking. Fixed broadband providers may not block lawful content, applications, services, or non-harmful devices; mobile broadband providers may not block lawful websites, or block applications that compete with their voice or video telephony services;

Leading up to last week’s announcement, Comcast and Verizon both issued public statements confirming that they did not block or rate-limit Netflix traffic. These statements were technically accurate; it was highly doubtful either party logged into its routers or deployed any Deep Packet Inspection to specifically alter traffic to or from Netflix’s infrastructure. Nonetheless, what was the intent behind delaying certain capacity upgrades at the edges of their networks? The data presents an interesting picture:

The above graph illustrates the quality of connectivity between Comcast and Tata (a transit provider Comcast uses for reachability to large swaths of the Internet, including the backbones operated by mega-telcos NTT, TeliaSonera, and XO, who provide Netflix with bandwidth) in New York. The evening packet loss displayed in blue is sufficient to degrade video quality to where rabbit ears look like a viable alternative, even with the most advanced of adaptive codecs/players.

Similarly, this graph illustrates connectivity between Comcast and Cogent Communications, a large supplier to Netflix, in New York.

In contrast, this graph illustrates one of few uncongested paths to Comcast in New York, identified and used by our Managed Internet Route Optimizer^TM (MIRO) technology.

Though this post is biased towards my hometown, a look into Comcast’s capacity in other markets, to the overwhelming majority of ISPs, presents a similar picture. While Netflix was not blocked by configuration, might it have been blocked by intent?

Conclusion and customer impact
While we have not witnessed a change in peering dynamics as a result of the Netflix/Comcast transaction, a trend we have seen over the past few years is the degrading quality of bandwidth from conventional “tier 1” ISPs, where peering edges have become congested due to the games described above. Network operators commonly discuss on mailing lists how the big four access shops all maintain edges which are boiling hot unless you pay them, or buy from an intermediary paying them. Where it was once possible for an enterprise or content shop to enjoy “good enough” connectivity purchasing from these providers directly, one now must enter the complex game of multi-homing to a half dozen or more providers, or purchase from a route-optimized “tier 2” like an Internap, in order to enjoy a positive and congestion-free user experience.

Contrary to popular opinion, Netflix did not “sell out”, nor were they traitors to the cause of net neutrality; rather, they did what they had to do in order to scale their growing online streaming business. One should not blame Netflix, but rather our marketplace for residential broadband, which allowed such a transaction to occur. I yearn for a day when we see true competition in the last mile — when providers compete on their ability to both innovate and deliver a good service for a fair price. Meanwhile, content and service providers looking to innovate in this space should invest in peer-to-peer delivery as well as on-premise caching. Ordinary users should support a regular framework towards enabling municipal broadband and competitive fiber-to-the-home options.

Internap customers can rest easy knowing that we’ll continue connecting them to Verizon, Comcast, Time Warner Cable, AT&T and a myriad of other access networks with reputations for congestion, without any undue latency or packet loss. Routing customers over constrained connections to prove a point is still not in the cards for us, nor is it something our customers would ever accept, as we sell IP services in a highly competitive marketplace. The Internap team will continue working diligently to circumnavigate the tangled-up tubes; we’ll develop next-generation technologies to keep the bits flowing as efficiently as possible, whatever the regulatory climate.

Internap’s webinar, Obstacles and Opportunities for the Cloud-Wary and Cloud-Wise, explored the details of our Cloud Landscape Report. Based on survey responses from 250 global Internet infrastructure professionals, the results highlight the problems faced by cloud-wise organizations that currently use cloud services, and those with no immediate plans to do so (the cloud-wary). Led by Adam Weissmuller, Internap’s Director of Product Management, Cloud Services, the webinar also discussed the growing demand for high-performance cloud solutions such as bare-metal cloud.

Performance challenges
Both the cloud-wary and the cloud-wise cite performance as a top pain point, with a majority of cloud-wise respondents specifically noting internal network performance problems. This is not surprising given the popularity of web-scale and Software-as-a-Service (SaaS) apps that are designed to work best in the cloud. Along with fast, big data apps, new workload demands are putting pressure on public virtual clouds and creating challenges around performance and cost-at-scale, making it clear that the cloud is built for elasticity, not performance.

Cloud doesn’t have to be virtual
The changing application requirements are driving interest in bare-metal cloud, which offers the performance and security of a physical server with the flexibility and scalability of the cloud. 73% of survey respondents who cited cloud performance as a problem recognized that a single tenant or dedicated server would outperform a comparable virtualized offering. The survey also revealed that many organizations mistakenly equate virtualization with public cloud.

Hybridization
With flexible, on-demand provisioning capabilities, bare-metal servers do not require separate management tools from the rest of your infrastructure. At Internap, we believe that true hybridization includes having single-pane-of-glass visibility across your systems, along with a network interconnection that allows different platforms to talk to one another. The Internap customer portal provides visibility into your colocation, cloud and hosting assets, all in one place, without the need for multiple management consoles.

Watch the webinar on demand to learn more about the different challenges faced by the cloud-wary and cloud-wise.