Learning Methodologies to Support E-Business in the Automated Negotiation Process

Introduction

The development of Information and Communication Technology (ICT) allows to develop several electronic applications in different fields. The main field is the Business to Business (B2B) e-commerce that allows to provide real value to manufacturing industries (Aberdeen Group, 2006). Their use by firms, as a recent research of the Aberdeen Group (2006) testifies, leads to the following improvements in performance: an increase in spending under management by 36%; a reduction in their requisition-to-order cycles by 75%; a reduction in their requisition-to-order costs by 48%; a reduction in their maverick spending by 36%.

The success of B2B applications depends on the real Value-Added Services (VAS) that support these applications. The VAS that can improve the profitability of the B2B application are: negotiation, coalition support tools and integration of production planning with the negotiation process (Renna and Argoneto, 2010). In particular, the negotiation process integrated with the production planning process may be considered to be a significant hindrance to the diffusion of B2B electronic commerce. The absence of appropriate tools to support the B2B platform does not allow the firms to gain significant benefits from the participation in these platforms (Calosso et al., 2004). The increasing importance of business to business electronic trading has driven interest in automated negotiation to soaring heights. The sequence of negotiation messages (proposals and counter-proposals) are modeled as the negotiation strategy of an actor (Jennings et al., 2001). In fact, it is important to distinguish between negotiation protocol and negotiation strategy. The protocol concerns the flow of messages in terms of who can say what, time of intervention, negotiation ending criteria, etc. The negotiation strategy is the sequence of the actions of the party in an effort to get the best outcome; for example, the utility function evaluation, how generate the counter-proposal and the threshold value to accept the proposals. The strategy is private information in order to gain an advantage to the negotiation opponents. The possibility of one opponent to incorporate additional knowledge of the opponents’ behavior can improve its performance. The effectiveness of providing knowledge about the domain of negotiation has been demonstrated in the trade-off strategy introduced in (Faratin et al., 2003). The information about the opponents’ behavior can be derived by the negotiation moves performed by that opponent during a negotiation.

Several authors proposed automated negotiation in different sectors without exploring learning possibility (Neubert et al., 2004; Hausen et al., 2006; Cheng et al., 2006; Renna, 2010). The scientific literature discussed the lack of learning opponents’ strategy in automated negotiation as a field to investigate. The automated negotiation process involves several disciplinary areas as: Multi Agent Systems (Ramchurn et al., 2007), game theory (Binmore and Vulkan, 1999), optimization (Arbib and Rossi, 2000), e-business (Huang et al., 2010) and decision support systems (Kersten and Lai, 2007).

The learning process can be supported by the reinforcement learning methodology. A Reinforcement-Learning (RL) agent learns by trial-and-error interaction with its dynamic environment (Kaelbling et al., 1996). At each time step, the agent perceives the complete state of the environment and takes an action, which causes the environment to transit into a new state. The agent receives a scalar reward signal that evaluates the quality of this transition. This feedback is less informative than in supervised learning, where the agent would be given the correct actions to take (Cherkassky and Mulier, 1998) (such information is, unfortunately, not always available). The RL feedback is, however, more informative than in unsupervised learning, where the agent would be left to discover the correct actions on its own, without any explicit feedback on its performance (Sejnowski and Hinton, 1999).