Multi-Agent Reinforcement Learning for Value Co-Creation of Collaborative Transportation Management (CTM)

Multi-Agent Reinforcement Learning for Value Co-Creation of Collaborative Transportation Management (CTM)

Liane Okdinawati, Togar M. Simatupang, Yos Sunitiyoso
DOI: 10.4018/IJISSCM.2017070105
(Individual Articles)
No Current Special Offers


Collaborative Transportation Management (CTM) is a collaboration model in transportation area. The use of CTM in today's business process is to create efficiency in transportation planning and execution processes. However, previous research paid little attention to demonstrate the ability for all agents in CTM to co-create value in services. The purpose of this paper is to increase the understanding of value co-creation in CTM area and learning processes in real systems based on value co-creation of CTM. Multiple case studies were used to analyze the value that was perceived by all agents in CTM in each collaboration stage and provided empirical evidence on the interactions among agents. Model-free reinforcement learning was used to predict how CTM could reduce transportation cost, increase visibility, and improve agility. The simulation results show that the input, feedback, and the experience of the agents are used to structure the collaboration processes and determine the strategies.
Article Preview


Collaborative Transportation Management (CTM) is an emerging model of collaboration in the transportation area (Tyan, Wang, & Du, 2003). Previous researchers developed CTM based on the main objective to increase the overall performance by using information sharing and resources sharing in their collaboration mechanisms. Several researchers such as Browning & White (2000), Sutherland (2003), Esper and Williams (2003) state the needs to incorporate CTM into logistics to avoid logistics bottlenecks, reduce inefficiency, reduce transportation costs, and provide mutual benefits for all collaborative parties.

CTM is developed among shipper, receiver, and carrier. In this research actors in CTM represented as agents. An agent is viewed as individual heterogeneity that represents agents’ decision rules that consists of beliefs, capabilities, choices, and commitments in the environment (Gilbert, 2008). Collaboration processes play an important role in the interactions among agents to understand the joint creation of value. Vargo and Lusch (2004) state that in a collaboration process, value is determined and created by each agent. According to Grönroos (2008), value co-creation only occurs when service providers extend their roles not only in providing services but also in influencing their services in accordance with the customers’ value.

Many researchers in CTM focus on the optimal solution for the agents’ collaborations that are carried out through mathematical modellings, computer simulations, and empirical researches (Okdinawati, Simatupang, & Sunitiyoso, 2015). However, the optimal solution in CTM does not answer how the interactions among agents’ collaborations leads to value co-creation and increase the agility and visibility if each agent’s collaboration has different objectives and different perspectives.

Multi-agent system is used in this research to model interaction among agents and learning process from their experience in synchronizing and coordinating collaboration processes, in order to share the benefit of CTM among them. Multi-agent system for CTM is formulated in vertical collaboration among shippers, receivers, and carriers to optimize the benefits for all agents through value co-creation. The information sharing mechanism is used as a foundation to make the effective decisions in this collaboration. Decision making is performed jointly by agents and is implemented in the process by one or more agents involved.

The benefit for CTM is associated with the value captured by all agents in collaboration processes. There are three benefits that are measured, namely, transportation cost, visibility, and agility. Transportation cost is a monetary measurement of operation expenses that must be spent by a carrier. Visibility is the available data that can be used to make decisions and determine strategies by all agents. Agility represents how fast all agents in CTM can respond and adapt to the changes in the environment.

The purpose of this paper is to increase understanding of value co-creation in CTM and analyze how value is co-created among agents in CTM. It provides knowledge by giving empirical evidence what kinds of activities and collaboration processes constitute value co-creation processes and how agents can learn in a collaboration process. The empirical findings present evidence on the interaction among agents, the relationship among agents, and the role of each agent in value co-creation processes.

Agent-based model in this research is adopted to model the effects of interactions among agents in CTM, where value is co-created, as well as models the roles and actions among agents in a collaboration stage in order to create value. Four scenarios are used in this research based on empirical research and based on an ideal condition. In order to model learning process of all agents in CTM, the model-free reinforcement learning method is employed.

The rest of the paper is organized as follows. First, the related literatures are briefly reviewed. Next, the problem situation of the empirical research is explained. Then the simulation process is presented. Afterwards, the simulation results and their discussion that illustrate this research are presented. The last, conclusions, and recommendations for future research conclude this paper

Complete Article List

Search this Journal:
Volume 17: 1 Issue (2024)
Volume 16: 1 Issue (2023)
Volume 15: 7 Issues (2022): 6 Released, 1 Forthcoming
Volume 14: 4 Issues (2021)
Volume 13: 4 Issues (2020)
Volume 12: 4 Issues (2019)
Volume 11: 4 Issues (2018)
Volume 10: 4 Issues (2017)
Volume 9: 4 Issues (2016)
Volume 8: 4 Issues (2015)
Volume 7: 4 Issues (2014)
Volume 6: 4 Issues (2013)
Volume 5: 4 Issues (2012)
Volume 4: 4 Issues (2011)
Volume 3: 4 Issues (2010)
Volume 2: 4 Issues (2009)
Volume 1: 4 Issues (2008)
View Complete Journal Contents Listing