Article Preview
Top1. Introduction
ShadowStream (Chen&Richard, 2012, pp. 347-358) is a novel Internet live streaming system that integrates performance evaluation as intrinsic capability. It introduces a novel production-CDN-experiment (PCE) streaming machine layout to protect real viewers' quality-of-experience (QoE) in experiment, at the same time gets accurate results.
An essential component in ShadowStream system is lifecycle control. In general, it lets a production viewer participate in the experiment by assigning arrival time and lifetime to emulate a virtual client's arrive and departure events. To distinguish it with clients' real behavior time in physical world, we call it virtual arrival/departure time.
To achieve this, we have introduced distributed mechanism in the process of control. When appointed a specific behavior scenario, orchestrator will send relevant parameters to testing clients to let them locally compute their arrival/life time for testing. In case of a client quits the experiment ahead of its virtual departure time unexpected, which we call early-quitted client, orchestrator will choose another viewer as a replacement and duplicate its status to the client.
However, early-quitted clients do have a negative influence on accuracy, since the process of replacement cannot be totally seamless. Thus we merely sort out stable viewers in physical world to perform distributed control for a specific behavior scenario to minimize the impact of replacement, which, on the other hand, has imposed restrictions on the scale of experiment in ShadowStream. Since in general cases stable clients are usually too small a group to be effectively utilized in production channel, which has been proved in Figure 1 (Wang&Liu, 2008). Figure 1(a) indicates that the majority of viewers just stay in channel for a quite short period in physical world, and if consider a client as stable when its lifetime exceed 40% of the observed session, it takes up only 5% to 18% of the whole viewers in different traces. Furthermore, Figure 1(b) explains the percentage of stable clients in a per-snapshot view in channel, and it is clear that there exist only 54% to 90% stable clients in a snapshot under the circumstances above.
Figure 1. The general situation about lifetime distribution and ratio of stable nodes per snapshot in physical world. (a) CCDF of life time distribution in different traces. (b) Ratio of stable nodes to All Nodes per snapshot.
In this paper, a novel distributed client lifecycle control is developed to get rid of restrictions caused by limited number of stable viewers in live-testing streaming networks. And dedicates to increasing real viewers' utilization level in physical world and decreasing replacement times in the process of experiment.
The major challenge in the course of experiment control in live testing platform is about the real viewers. As a live testing system, ShadowStream is designed to orchestrate desired experimental scenarios from production viewers, without disturbing their quality of experience. Other than clients in a traditional testing platform, production viewers in live testing cannot be controlled. Furthermore, we are not allowed to interfere in their own behaviors (e.g., arrive, depart), or change their behaviors casually. That is to say, before sending a command to a client, guarantees should be made to ensure that the command which determines client's act in the experiment will not be in conflict with its real behavior at some point in the future.
The core idea is to match the desired experimental scenario with viewers' real behavior in physical world to scale up the experiment. Moreover, we make a further survey about taking advantage of a certain group of stable viewers in physical world to achieve the desired scenario by letting them rejoin the testing under the specific situation.
The remainder of this paper is organized as follows: Section 2 introduces the background of our research about ShadowStream and experiment orchestration in detail. And Section 3 describes the basic idea about predicting each viewer's residual lifetime in physical world and matching it with the specific scenario in experiment. Then Section 4 explicitly explains the implementation of orchestration and relevant algorithms to control testing clients' behavior in experiment before we carefully evaluate its performance and testify the validity in Section 5.