Article Preview
Top1. Introduction
The World Wide Web (WWW) has established itself as the major repository for publicly accessible information. A template is a common structural layout shared by most of the webpages. The majority of the items shared by webpages, such as the menu and adverts, belong to templates. The structure of the pages displaying product information on a typical e-commerce site is identical. The structural resemblance between webpages is due to the use of the same templates. The Server-side scripts create these webpages by incorporating data from a database into an HTML page template. Although the structure and look of the webpages generated by the same server-side script are similar, the content is distinct. Figure 1 and Figure 2 illustrate two webpages that share a same template. In most cases, the informational content of the webpages generated by templates is formatted and organized. It can be retrieved and kept in databases to be analyzed in a variety of ways. However, because the information is contained within a template, retrieving this information is challenging. Wrappers are employed to address this issue. Wrappers generated manually are time consuming and prone to errors. As a result, wrappers are automatically generated. This is referred to as wrapper induction. Wrapper induction enables highly precise data extraction from template-generated webpages. Understanding the structure of structurally related and template-generated webpages is the first step in wrapper induction. A wrapper induction to function effectively requires the webpages to be clustered in such a way that all webpages generated by the same template are grouped together in the same cluster. For each type of template, simple data extraction rules are written and used to extract data from every webpage generated from that template.
The problem of clustering webpages based on templates is a component of the overall system for extracting information from webpages. After obtaining clusters, a wrapper is executed for each cluster to extract the information contents from webpages for further analysis. The proposed work addresses the problem of clustering template-generated webpages.
The problem of clustering of webpages created from templates has been widely studied by Kim & Shim (2011), Gottron (2008), Vieira et al. (2009), Grigalis & Cenys(2014) and Bagban & Kulkarni(2020). Many solutions with different techniques exist, but they are limited to clustering webpages of the same website and are not scalable at a web-scale level. This paper discusses the design and analysis of the solution to the problem of clustering of webpages belonging to different templates. The set of webpages created from different templates called heterogeneous webpages. The earlier solution is to build a Document Object Model (DOM) (Document Object Model (DOM), 2010) tree of webpages and use tree-edit distance and similarity measuring techniques (Gottron, 2008; Reis et al., 2004). However, this approach is computationally expensive with time complexity of O(n2), with n representing the number of nodes in a DOM tree.
The overall approach is described using webpages collected from a standard dataset referred by Crescenzi et al. (2001) showing baseball players’ information listing. All the webpages presenting players listing information share the same template as shown in below Figure 1 and Figure 2.
Figure 1. Webpage generated from a template
Figure 2.
Webpage generated from a template