Learning-based Relaxation of Completeness Requirements for Data Entry Forms

2.1 Data Entry Forms

Data entry forms are composed of fields of different types, such as textual, numerical, and categorical. Textual and numerical fields collect free text and numerical values, respectively (e.g., the name and the age of a private customer of an energy provider); categorical fields provide a list of options from which users have to choose (e.g., nationality). Form developers can mark form fields either as required or optional, depending on the importance of the information to be collected. This decision is made during the design phase of the form based on the application completeness requirements. Such requirements capture the input data that shall be collected for certain types of users; they are fulfilled by setting the required/optional property of the corresponding fields in a data entry form. In other words, the required fields (also called mandatory fields [46]) of a form collect input information considered as important to the stakeholders who plan to use the collected information; the absence of this information could affect the application usage. On the contrary, optional fields collect information that is nice to have but whose absence is acceptable. For example, an energy provider cannot open a customer account when the customer name is missing; hence, the corresponding input field in a data entry form should be marked as “required.” At the same time, an energy provider does not need to know the education level of a new private customer (though it could be useful for profiling), so the corresponding input field can be marked as “optional.”

Some required fields can be further classified conditionally required, i.e., they are required only if certain conditions hold. For example, the field “marriage date” is required only if the value of the categorical field “civil status” is set to “married.” Data entry forms that support “conditionally required fields” are generally called adaptive forms [6] or context-sensitive forms [3], since they exhibit adaptive behaviors based on the values filled by users. More specifically, these types of forms are programmed so a field can be set from “required” to “optional” during the form-filling session, based on the input data; a change of this property also toggles the visibility of the field itself in the form. Such adaptive behaviors make the data entry form easier to use [3], since users can focus on the right fields they need to fill in.

Before submitting a data entry form, the form usually conducts a client-side validation check [51]—using some scripting language or built-in features of the environment where the form is visualized, like HTML attributes—to ensure that all the required fields have been filled in.

In this work, we consider a simple representation of an input form, with basic input fields that can have only a unique value that can be selected or entered, such as a text box (e.g., <input type=“text”> or <textarea> in HTML), a drop-down menu (e.g, <select> with single selection), or a radio button (e.g., <input type=“radio” in HTML). This allows us to assume that a field can only have one completeness requirement; in other words, a field cannot be optional and required at the same time.

We do not support forms with more sophisticated controls or fields that can handle multiple selections (e.g., a checkbox group for multiple-choice answers or a drop-down menu with multiple selection), as often found in surveys and questionnaires. Note that in this case a field can be both optional and required at the same time, depending on the number of selected values in the group.² We plan to support this kind of complex controls as part of future work.

2.2 Motivating Example

Data entry forms are difficult to design [17] and subject to frequent changes [54]. These two aspects of data entry form design and development negatively impact the way developers deal with application completeness requirements in data entry forms.

For example, let us consider a data entry form in an energy provider information system, used for opening an account for business customers. For simplicity, we assume the form has only three required fields: “Company type” (categorical), “Field of activity” (categorical), and “Tax ID” (textual). Sometime after the deployment of the initial version of the system, the energy provider decides to support also the opening of customer accounts for non-profit organizations (NPOs). The developers update the form by adding (a) a new option “NPO” to the field “Company type” and (b) additional fields denoting information required for NPOs. After the deployment of the new form, a data entry operator of the energy provider (i.e., the end-user interfacing with the data entry form) notices a blocking situation when filling in the form for an NPO. Specifically, the form flags the field “Tax ID” as required; however, the company representative cannot provide one, since the company is exempted from paying taxes. The clerk is then obliged to fill in the required field with a meaningless value (e.g., “@”) to pass the validation check and be able to submit the form. Several weeks later, after noticing some issues in the generation of customers’ reports, the data quality division of the energy provider reviews the data collected through the data entry form, detecting the presence of meaningless values. A subsequent meeting with IT analysts and developers reveals that those values have been introduced because the data entry form design has not been updated to take into account the new business requirements (i.e., opening accounts for NPOs) and the corresponding completeness requirements (i.e., some NPOs in certain fields of activity do not have a tax ID). For example, the current (but obsolete) form design always flags “tax ID” as a required field; however, when the “Company type” field is set to “NPO” and the “Field of activity” field is either “charity” or “education,” the field “tax ID” should be optional.

These meaningless values filled during form filling negatively affect data quality [3], since they are considered as data entry errors and may lead to error propagation:

Meaningless value errors are difficult to identify, because such values can pass all validation checks of the data entry form. A business may establish the practice of using specific values (e.g., “@” and “-1”) when users do not need to fill some fields, as in the aforementioned example. However, even in this case the data quality team needs to carefully check the filled fields to ensure that all the data entry operators follow this convention, which is a time-consuming process.

Currently, there are some simple but rather impractical solutions to address the issue of filling meaningless values, including rule-based solution and dictionary-based solution:

However, the two solutions are not practical. Given the evolving nature of software [22, 52], the rule-based solution is not scalable and maintainable, especially when the number of fields (and their possible values, for categorical fields) increases. Moreover, as is the case for our industrial partner, it is difficult also for domain experts to formulate the completeness requirement of new fields, since they have to decide the exact impact of different field combinations on the new fields. Regarding the dictionary-based solution, the completeness requirement of a field usually depends on the values of other filled fields [3] (such as the aforementioned example of Tax ID) and cannot be detected only by looking at special/meaningless characters. This simple solution cannot help domain experts identify these useful conditions.

Therefore, we contend it is necessary to develop automated methods to learn such conditions directly from the data provided as input in past data entry sessions, so completeness requirements of form fields can be automatically relaxed during new data entry sessions. Moreover, the learned conditions could also help designers identify completeness requirements that should be relaxed.

2.3 Problem Definition

In this article, we deal with the problem of completeness requirement relaxation for data entry forms. The problem can be informally defined as deciding whether a required field in a form can be considered optional based on the values of the other fields and the values provided as input in previous data entry sessions for the same form. We formally define this problem as follows:

Let us assume we have a data entry form with n fields \(F=\lbrace f_1, f_2, \ldots , f_n\rbrace\). Taking into account the required/optional attribute of each field, the set of fields can be partitioned into two groups: required fields (denoted by R) and optional fields (denoted by \(\bar{R}\)), where \(\bar{R} \cup R=F\) and \(\bar{R} \cap R = \emptyset\). Let \(\mathit {VD}\) represent a value domain that excludes empty values. Each field \(f_i\) in F can take a value from a domain \(V_i\), where \(V_i = \mathit {VD}_i\) if the field is required and \(V_i = \mathit {VD}_i \cup \bot\) if the field is optional (\(\bot\) is a special element representing an empty value).

Let \(R^{c}\subseteq R\) be the set of conditionally required fields, which are required only when a certain condition \(\mathit {Cond}\) is satisfied. For a field \(f_k\in R^{c}\), we define the condition \(\mathit {Cond}_k\) as the conjunction of predicates over the value of some other fields; more formally, \(\mathit {Cond}_k=\bigwedge _{1 \le i \le n, i \ne k} h(f_i, v_i^c)\), where \(f_i \in F, v_i^c \in V_i\), and h is a predicate over the field \(f_i\) with respect to the value \(v_i^c\).

During form filling, at any time t the fields can be partitioned into two groups: fields that have been filled completely (denoted by \(C_{t}\)) and unfilled fields (denoted by \(\bar{C_{t}}\)); let G be the operation that extracts a field from a form during form filling \(G(F)= f\), such that \((f \in C_t)\vee (f\in \bar{C_{t}})\) and \(C_{t} \cap \bar{C_{t}}= \emptyset\). By taking into account also the required/optional attribute, we have: filled required fields \((C_{t}\cap R)\), filled optional fields \((C_{t}\cap \bar{R})\), unfilled required fields \((\bar{C_{t}} \cap R)\), and unfilled optional fields \((\bar{C_{t}} \cap \bar{R})\).

When a form is about to be submitted (e.g., to be stored in a database), we define an input instance of the form to be \(I^F=\lbrace \langle f_1, v_1 \rangle , \ldots , \langle f_n, v_n\rangle \rbrace\) with \(f_i \in F\) and \(v_i \in V_i\); we use the subscript \(t_j\) as in \(I^F_{t_j}\) to denote that the input instance \(I^F\) was submitted at time \(t_j\). We use the notation \(I^F(t)\) to represent the set of historical input instances of the form that have been submitted up to a certain time instant t; \(I^F(t)=\lbrace I^F_{t_{i}}, I^F_{t_{j}}, \ldots , I^F_{t_{k}}\rbrace\), where \(t_i \lt t_j \lt t_k \lt t\). Hereafter, we drop the superscript F when it is clear from the context.

The completeness requirement relaxation problem can be defined as follows: Given a partially filled form \(F=\lbrace f_1, f_2, \ldots , f_n\rbrace\) for which, at time t, we know \(\bar{C_{t}} \ne \emptyset\), \(C_{t}\), and \(R^c\), a set of historical input instances \(I^F(t)\), and a target field \(f_{p}\in (R^{c}\cap \bar{C_{t}})\) to fill, with \(p \in 1 \dots n\), we want to build a model M predicting whether, at time t, \(f_p\) should become optional based on \(C_{t}\) and \(I^F(t)\).

Application to the running example.

Fig. 1 is an example of a data entry form used to fill information needed to open an account for business customers with an energy provider The form F is composed of five fields, including \(f_1\):“Company Name,” \(f_2\):“Monthly revenue,” \(f_3\):“Company type,” \(f_4\):“Field of activity,” and \(f_5\):“Tax ID.” All the fields are initially required (i.e., \(R=\lbrace f_1,f_2,f_3,f_4, f_5\rbrace\)). Values filled in these fields are then stored in a database. An example of the database is shown on the right side of Fig. 1. These values are collected during the data entry session with an automatically recorded timestamp indicating the submission time. Each row in the database represents an input instance (e.g., \(I^F_{20180101194321}=\lbrace \langle {\it ``Company Name^{\prime \prime }}, {\it UCI} \rangle , \ldots , \langle {\it ``Tax ID^{\prime \prime }}, {\it T190}\rangle \rbrace\)), where the column name corresponds to the field name in the form. The mapping can be obtained from the existing software design documentation or software implementation [5]. Using the data collected from different users, we can build a model M to learn possible relationships of completeness requirement between different fields. Let us assume a scenario where during the creation of a customer account using F, the energy provider clerk has entered Wish, 20, NPO, and education for fields \(f_1\) to \(f_4\), respectively. The field \(f_5\) (“Tax ID”) is the next field to be filled. Our goal is to automatically decide if field \(f_5\) is required or not based on the values filled in fields \(f_1\) to \(f_4\).

Fig. 1.

View Figure

2.4 Towards Adaptive Forms: Challenges

Several tools for adaptive forms have been proposed [6, 19, 49]. These approaches use intermediate representations such as XML [6] and dynamic condition response graphs [49] to represent the completeness requirements rules and implement adaptive behaviors. Existing tools for adaptive forms usually assume that form designers already have, during the design phase, a complete and final set of completeness requirements, capturing the conditions for which a field should be required or optional.

However, this assumption is not valid in real-world applications. On the one hand, data entry forms are not easy to design [17]. Data entry forms need to reflect the data that need to be filled in an application domain. Due to time pressure and the complexity of the domain (e.g., the number of fields needed to be filled and their interrelation), it is difficult to identify all the completeness requirements when designing the data entry form [2, 12]. On the other hand, data entry forms are subject to change: A recent study [54] has shown that 49% of web applications will modify their data constraints in a future version. The frequent changes in data constraints may also make the existing completeness requirements obsolete.

Hence, the main challenge is how to create adaptive forms when the set of completeness requirements representing the adaptive behavior of a form is incomplete and evolving.

3.1 Bayesian Networks

Bayesian networks (BNs) are probabilistic graphical models (PGM) in which a set of random variables and their conditional dependencies are encoded as a directed acyclic graph: Nodes correspond to random variables and edges correspond to conditional probabilities.

The use of BNs for supervised learning [20] typically consists of two phases: structure learning and variable inference.

During structure learning, the graphical structure of the BN is automatically learned from a training set. First, the conditional probability between any two random variables is computed. Based on these probabilities, optimization-based search (e.g., hill climbing [21]) is applied to search the graphical structure. The search algorithm initializes a random structure and then iteratively adds or deletes its nodes and edges to generate new structures. For each new structure, the search algorithm calculates a fitness function (e.g., Bayesian information criterion, BIC [40]) based on the nodes’ conditional probabilities and on Bayes’ theorem [20]. Structure learning stops when it finds a graphical structure that minimizes the fitness function.

Fig. 2 shows an example of the BN structure learned based on the data submitted by the data entry form used in our example in Section 2.2. This BN contains three nodes corresponding to three fields in the data entry form: variable \(\mathit {Revenue}\) depends on variable \(\mathit {Company \ type}\); variable \(\mathit {Tax \ ID}\) depends on variables \(\mathit {Company \ type}\) and Revenue. For simplicity, we assume that the three variables are Boolean where a, b, and c denote that fields \(\mathit {Company \ type}\), \(\mathit {Revenue}\), and \(\mathit {Tax \ ID}\) are “required,” respectively, and \(\bar{a}\), \(\bar{b}\), and \(\bar{c}\) denote that these fields are optional.

Fig. 2.

View Figure

In the PGM, each node is associated with a probability function (in this case, encoded as a table), which represents the conditional probability between the node and its parent(s). For example, in Fig. 2 each variable has two values; the probability table for \(\mathit {Revenue}\) reflects the conditional probability \(P(\mathit {Revenue}\mid \mathit {Company \ type})\) between \(\mathit {Company \ type}\) and \(\mathit {Revenue}\) on these values.

Variable inference infers unobserved variables from the observed variables and the graphical structure of the BN using Bayes’ theorem [20]. For example, we can infer the probability of \(\mathit {Tax \ ID}\) to be required (i.e., \(\mathit {Tax \ ID} =c\)) when the completeness requirement of \(\mathit {Company \ type}\) is required (denoted by \(P(c \mid a)\)) as follows: \(\begin{equation*} \begin{aligned}P(c \mid a) &= \frac{P(a, c)}{P(a)} = \frac{P(a, b, c) + P(a, \overline{b}, c)}{P(a)} \\ &= \frac{P(c \mid a, b) P(b \mid a) P(a) + P(c \mid a, \overline{b}) P(\overline{b} \mid a) P(a)}{P(a)} \\ &= \frac{0.9*0.4*0.2 + 0.4*0.6*0.2}{0.2} =0.6. \end{aligned} \end{equation*}\)

BNs have been initially proposed for learning dependencies among discrete random variables. They are also robust when dealing with missing observed variables; more specifically, variable inference can be conducted when some conditionally independent observed variables are missing [20]. Recently, they have been applied in the context of automated form filling [5].

3.2 Synthetic Minority Oversampling Technique (SMOTE)

A frequently encountered problem for training machine learning models using real-world data is that the number of instances per class can be imbalanced [32, 48]. To address this problem, many imbalanced learning approaches have been proposed in the literature. One of them is SMOTE [9]; it uses an oversampling method to modify the class distribution in a dataset (i.e., the ratio between instances in different classes). It synthesizes new minority class instances to improve the learning ability of machine learning algorithms on the minority class. SMOTE conducts the instance synthesis by means of interpolation between near neighbors. Initially, each instance in the dataset is represented as a feature vector. SMOTE starts by randomly selecting a minority class instance i from the dataset. It determines the k nearest neighbors of i from the remaining instances in the minority class by calculating their distance (e.g., the Euler distance) based on their feature vectors. SMOTE synthesizes new instances using n instances randomly selected from the k neighbors. The selection is random to increase the diversity of the generated new instances. For each selected instance, SMOTE computes a “difference vector” that represents the difference of the feature vectors between the selected instance and instance i. SMOTE synthesizes new instances by adding an offset to the feature vector of instance i, where the offset is the product of the difference vector with a random number between 0 and 1. SMOTE stops generating new instances until a predefined condition is satisfied (e.g., the ratio of instances in the majority and minority classes is the same).

Fig. 3 illustrates the application of SMOTE to create new minority class instances. As shown in the table on the right, instances \(i_1,i_2\), and \(i_3\) belong to the minority class “Optional” of our target field. As a preliminary step, SMOTE computes the Euclidean distance between all the minority instances: \(d({i_1},{i_2})= \sqrt {\left(39-42\right)^2} = 3\), \(d({i_1},{i_3})= \sqrt {\left(39-25\right)^2} = 14\), and \(d({i_2},{i_3})= \sqrt {\left(42-25\right)^2} = 17\). SMOTE starts by randomly picking one instance from the minority class (e.g., \(i_2\)). Assuming that the value of k is equal to 1, SMOTE selects the nearest instance to \(i_2\), which in our example is the instance \(i_1\). To create a new instance \(i_8\), SMOTE computes the \(\mathit {Difference \ vector}\) based on the feature vectors \(\mathit {Monthly \ revenue}_{i_2}\) and \(\mathit {Monthly \ revenue}_{i_1}\) and multiplies it by a random value \(\lambda\) between 0 and 1. The value of the “Monthly revenue” column in the synthetically created instance \(i_8\) is equal to \(\mathit {Monthly \ revenue}_{i_2}\)+ \(\mathit {Difference \ vector}\). In our example, assuming that the value of \(\lambda\) is equal to 0.7, the new value of the “Monthly revenue” field for \(i_8\) is equal to \(42+((39 - 42) * 0.7)= 40\).

Fig. 3.

View Figure

4.1 Pre-processing

The first two phases of LACQUER include a preprocessing step to improve the quality of the data in historical input instances as well as the current input instance. As mentioned in Section 2.1, data entry forms can contain fields that are not applicable to certain users; this is the main cause of the presence of missing values and meaningless values in historical input instances. Missing values occur when users skip filling an (optional) field during form filling. A meaningless value is defined as any value filled into a form field that can be accepted during the validation check but does not conform with the semantics of the field. For example, given a data entry form with a textual field “Tax ID,” if a user fills “n/a” in this field, then the value can be accepted during the submission of the instance³; however, it should be deemed meaningless, since “n/a” does not represent an actual “Tax ID.”

For missing values, we replace them with a dummy value “Optional” in the corresponding field. As for the meaningless values, we first create a dictionary containing possible meaningless values based on domain knowledge. This dictionary is used to match possible meaningless values in historical input instances; we replace the matched values with “Optional.” The rationale for this strategy is that it is common practice, within an enterprise, to suggest data entry operators some specific keywords when a field is not applicable for them. For example, our industrial partner recommends users to fill such fields with special characters such as “@” and “$.” The overarching intuition behind replacing missing values and meaningless values with “Optional” is that, when data entry operators skip filling a field (resulting in a missing value in the form) or put a meaningless value, it usually means that this field is not applicable in the current context.

After detecting missing values and meaningless values, we preprocess other filled values. For textual fields, we replace all valid values with a dummy value “Required,” reflecting the fact that data entry operators deemed these fields to be applicable. After preprocessing, all values in textual fields are therefore either “Required” and “Optional” to help the model learn the completeness requirement based on this abstract presentation. Numerical fields can be important to decide the completeness requirement of other fields. For example, companies reaching a certain monthly revenue can have some specific required fields. For this reason, we apply data discretization to numerical fields to reduce the number of unique numeric values. Each numeric value is represented as an interval, which is determined using the widely used discretization method based on information gain analysis [7]. We do not preprocess categorical fields, since they have a finite number of candidate values. We keep the original values of categorical fields, since users who select the same category value may share common required information. At last, we delete all the fields that are consistently marked as “Required” or “Optional,” because such fields do not provide any discriminative knowledge to the model.

During the data entry session, similar preprocessing steps are applied. We skip values filled in fields that were removed in historical input instances. We replace values in textual fields with “Required” and “Optional,” as described above. We also map numerical values onto intervals and keep values in categorical fields.

The historical input instances are then divided in two parts that will be used separately for training (training input instances) and for the threshold determination (tuning input instances).

Application to the running example.

Fig. 5 shows an example of historical input instances collected from the data entry form presented in Fig. 1. During the preprocessing phase, LACQUER identifies meaningless values in different fields (e.g., “n/a” and “@”) and replaces them by the dummy value Optional. For the remaining “meaningful” values, LACQUER replaces values in the textual field “Company name” to the dummy value Required; values in the field “Monthly revenue” are discretized into intervals. In addition to historical input instances, LACQUER also preprocesses the input instance filled during the data entry session. For example, as shown in Fig. 1, a user fills values Wish, 20, NPO, and Education in fields “Company name,” “Monthly revenue,” “Company type,” and “Field of activity,” respectively. LACQUER will replace the value filled in the field “Company name” to “Required,” since it is a meaningful value. LACQUER also maps the value in the field “Monthly revenue” into the interval [20, 22).

Fig. 5.

View Figure

4.2 Model Building

The model building phase aims to learn the completeness requirement dependencies between different fields from training input instances related to a data entry form.

During the data entry session, we consider the filled fields as features to predict the completeness requirement of the target field (i.e., optional or required). However, in our previous work [5], we have shown that in an extreme scenario, users could follow any arbitrary order to fill the form, resulting in a large set of feature-target combinations. For example, given a data entry form with n fields, when we consider one of the fields as the target, we can get a total number of up to \(2^{n-1}-1\) feature (i.e., filled fields) combinations. Based on the assumption of identical features and targets [14] to train and test a machine learning model, a model needs to be trained on each feature-target combination, which would lead to training an impractical large number of models.

To deal with this problem, we select BNs as the machine learning models to capture the completeness requirement dependencies between filled fields and the target field, without training models on specific combinations of fields. As already discussed in our previous work [5], the reason is that BNs can infer the value of a target field using only information in the filled fields and the related PGM (see Section 3.1); BNs automatically deal with the missing conditionally independent variables (i.e., unfilled fields).

In this work, LACQUER learns the BN structure representing the completeness requirement dependencies from training input instances. Each field in the data entry form represents a node (random variable) in the BN structure; the edges between different nodes are the dependencies between different fields. To construct the optimal network structure, BN performs a search-based optimization based on the conditional probabilities of the fields and a fitness function. As in our previous work [5], we use hill climbing as the optimizer to learn the BN structure with a fitness function based on BIC [40].

Algorithm 1 illustrates the main steps of this phase. LACQUER takes as input a set of preprocessed historical input instances \(I^F(t)_{\mathit {train}}^\prime\) for training and learning the completeness requirement dependencies (e.g., the input instances in block of Fig. 6). Initially, for each field \(f_i\) in the list of fields extracted from \(I^F(t)_{\mathit {train}}^\prime\) (line 2), we create a temporary training set where we consider the field \(f_i\) as the target (line 4). Since we aim to predict whether the target field is required or optional during form filling, in the temporary training set, we keep the value “Optional” in the target field \(f_i\) and label other values as “Required” (block in Fig. 6). These two values are the classes according to which to predict \(f_i\).

Fig. 6.

View Figure

However, we may not train effective classification models directly on this temporary training set. This is caused by the imbalanced nature of input instances for different classes. Users commonly enter correct and meaningful values during form filling. They only fill meaningless values in certain cases. As a result, the number of input instances having meaningless values (i.e., in the “Optional” class) is usually smaller than the number of input instances in the “Required” class. This can make the learning process inaccurate [29], since machine learning models may consider the minority class as noise [43]. The trained models could also over-classify the majority class due to its increased prior probability [29]. For example, in block of Fig. 6, considering that the column “\(f_5\): Tax ID” is the current target, the number of instances in class “Required” is three, which is higher than the single instance in class “Optional.” If we train a model on such imbalanced dataset, then it might be difficult to learn the conditions (or dependencies) to relax this field as optional due to the small number of “Optional” instances.

To solve this problem, we apply SMOTE (line 5) on the temporary training set \(\mathit {train}_{f_{i}}\) to generate an oversampled training set \(\mathit {train}_{f_{i}}^{\mathit {oversample}}\) (as shown in block in Fig. 6). After oversampling, both classes have the same number of input instances. We train a BN model \(M_i\) based on the oversampled training set for the target field \(f_i\) (line 6). For example, block in Fig. 6 represents the model built for the target field “Tax ID.” Following this step, we can obtain a BN model for each field. We save all the BN models in the dictionary \(\mathcal {M}\) (line 7), where the key represents the name of the field and the value is the corresponding trained BN model. The output of Algorithm 1 is the dictionary \(\mathcal {M}\).

Application to the running example.

Given the preprocessed training input instances shown in block in Fig. 6, LACQUER creates a temporary training set for each target (e.g., the field “Tax ID”), where LACQUER replaces the meaningful and meaningless values of the target field to Required and Optional, respectively (in block ). The temporary training set is oversampled using SMOTE to create a balanced training set where the number of instances of both Required and Optional classes is the same (block of Fig. 6). This oversampled training set is used to train a BN model for the target field “Tax ID.” An example of the trained BN model is presented in block in Fig. 6. After the model building phase, LACQUER outputs a model for each target. For the example of training input instances related to Fig. 1, LACQUER returns five distinct models where each model captures the completeness requirement dependencies for a given target.

4.3 Form-filling Relaxation

The form-filling relaxation phase is an online phase that occurs during the data entry session. In this phase, LACQUER selects the model \(M_{p}\) \(\in \mathcal {M}\) corresponding to the target field \(f_p\). This model is then used to predict the completeness requirement of the target \(f_p\) based on the filled fields \(C_{t}\). The main steps are shown in Algorithm 2.

The inputs of the algorithm are the dictionary of trained models \(\mathcal {M}\), the set \(C_{t}\) representing the filled fields during the entry session and their values, the target field \(f_p\), and the endorsing threshold \(\theta _p\) for \(f_p\). The algorithm starts by applying the preprocessing techniques outlined in Section 4.1 to the set of the filled fields in \(C_{t}\) to obtain a new set of preprocessed filled fields \(C_{t}^{\prime }\) (line 1). LACQUER then selects the model \(M_p\) from \(\mathcal {M}\) (line 2), since this model is trained for the target field \(f_p\) based on the oversampled data with a balanced number of instances for each class of \(f_p\). With the selected model, LACQUER predicts the completeness requirement for \(f_p\) (line 3) and gets the top-ranked completeness requirement based on the prediction probability (line 4).

Application to the running example.

Fig. 7 shows the process of predicting the completeness requirement of the field “Tax ID” based on the input values in Fig. 1. LACQUER first selects the model related to the current target for prediction (block in Fig. 7). Let us assume that, based on the BN variable inference, LACQUER predicts that field “Tax ID” has a probability of 0.80 to be Optional. Since the top predicted value is Optional, LACQUER activates the endorser module (block in Fig. 7) to decide whether the level of confidence is acceptable. For example, let us assume the automatically decided threshold value for field “Tax ID” is 0.70 (i.e., \(\theta _{\mathit {tax ID}}\)=0.70). Since the probability value of the “Optional” class (0.80) is higher than this threshold, the Boolean flag \(\mathit {checkOP}_\mathit {TaxID}\) remains true. LACQUER decides to set the field “Tax ID” to Optional.

Fig. 7.

View Figure

4.4 Endorser Threshold Determination

We automatically determine the value of the threshold in the endorser module for each target. This step occurs offline and assumes that the models in \(\mathcal {M}\) built during the model building phase are available. The threshold \(\theta _i\) for the target field i is determined with the set of preprocessed tuning input instances. The basic idea is that for each historical input instance in this subset, we consider all fields except field i to be filled and use the model trained for field i to predict its completeness requirement with different values of \(\theta _i\). We determine the value of \(\theta _i\) based on the value that achieves the highest prediction accuracy on tuning input instances.

The main steps are shown in Algorithm 3. The algorithm takes as input the set of preprocessed historical input instances for tuning \(I^F(t)_{\mathit {tune}}^\prime\) and the trained models \(\mathcal {M}\). For each field \(f_i\) in the list of fields extracted from \(I^F(t)_{\mathit {tune}}^\prime\) (line 2), we generate a temporary dataset \(I^F(t)_{\mathit {tune}_i}^\prime\) where the value of field \(f_i\) is transformed into “Optional” and “Required” using the method presented in Fig. 6(B) (line 5). We select the model corresponding to \(f_i\) from \(\mathcal {M}\) (line 6) and use the selected model to predict the completeness requirement of field \(f_i\) based on the values of other fields in \(I^F(t)_{\mathit {tune}_i}^\prime\) (line 8). While predicting, we try different thresholds, varying from 0 to 1 with a step equal to 0.05. For each threshold value, we compare the predicted completeness requirement with the actual completeness requirement of field \(f_i\) in each input instance of \(I^F(t)_{\mathit {tune}_i}^\prime\) to calculate the prediction accuracy (line 9). LACQUER selects the value of \(\theta _i\) that achieves the highest prediction accuracy value in \(I^F(t)_{\mathit {tune}_i}^\prime\) as the threshold for \(f_i\) (line 11). The algorithm ends by returning a dictionary containing the thresholds of all fields.

5.1 Dataset and Settings

Dataset Preparation - Application Example.

Fig. 8 illustrates an example of application of our dataset preparation method. The table on the left-hand side of the picture represents the information submitted during the data entry session through the data entry form introduced in our motivating example in Section 2.3. We split this dataset into a training set (80% of instances), a tuning set (10% of instances), and a testing set (10% of instances); let us assume the last row in the table is an instance in the testing set. The testing set is then processed to simulate the two form-filling scenarios. The sequential filling scenario uses the filling order following the tabindex value of the form fields. Assuming the tabindex order for the example is \(f_1\rightarrow f_2 \rightarrow f_3 \rightarrow f_4 \rightarrow f_5\), we can generate two test instances S1 and S2 (shown in the top right box of Fig. 8) to predict the completeness requirement of \(f_2\) and \(f_5\), respectively. As for the partial random filling scenario, this scenario takes into account the controls or grouping of fields specified by the designer. For example, let us assume that “\(f_3\) : company type” and “\(f_4\): field of activity” belong to the same group of fields named “Business activities”: this means that \(f_3\) and \(f_4\) should be filled sequentially. A possible filling order, randomly generated taking into account this constraint, is then \(f_1 \rightarrow (f_3 \rightarrow f_4) \rightarrow f_2 \rightarrow f_5\). The bottom right box in the figure shows the corresponding generated test instances PR1 and PR2.

Fig. 8.

View Figure

5.2 Effectiveness (RQ1)

To answer RQ1, we assessed the accuracy of LACQUER in predicting the correct completeness requirement for each target field in the dataset. To the best of our knowledge, there are no implementations of techniques for automatically relaxing completeness requirements; therefore, we selected as baselines two rule-based algorithms that can be used to solve the form-filling completeness requirements relaxation problem: association rule mining(ARM) [33] and repeated incremental pruning to produce error reduction(Ripper); these rule-based algorithms can provide form-filling relaxation suggestions under different filling orders. ARM mines association rules having the format “if antecedent then consequent” with a minimal level of support and confidence, where the antecedent includes the values of certain fields and the consequent shows the completeness requirement of a target field for a given antecedent. ARM matches the filled fields with the antecedents of mined association rules and suggests the consequent of the matched rules. Ripper is a propositional rule-based classification algorithm [11]; it creates a rule set by progressively adding rules to an empty set until all the positive instances are covered [31]. Ripper includes also a pruning phase to remove rules leading to bad classification performance. Ripper has been used in a variety of classification tasks in software engineering [23, 48]. Similar to ARM, Ripper suggests the consequent of the matched rule to users.

Methodology. We used Precision (\(\mathit {Prec}\)), Recall (\(\mathit {Rec}\)), Negative Predictive Value (\(\mathit {NPV}\)), and Specificity (\(\mathit {Spec}\)) to assess the accuracy of different algorithms. These metrics can be computed from a confusion matrix that classifies the prediction results into true positive (TP), false positive (FP), true negative (TN), and false negative (FN). In our context, TP means that a field is correctly predicted as required, FP means that a field is misclassified as required, TN means that a field is correctly predicted as optional, and FN means that a field is misclassified as optional. Based on the confusion matrix, we have \(\mathit {Prec}=\frac{\mathit {TP}}{\mathit {TP}+\mathit {FP}}\), \(\mathit {Rec}=\frac{\mathit {TP}}{\mathit {TP}+\mathit {FN}}\), \(\mathit {NPV}=\frac{\mathit {TN}}{\mathit {TN}+\mathit {FN}}\), and \(\mathit {Spec}=\frac{\mathit {TN}}{\mathit {TN}+\mathit {FP}}\). Precision is the ratio of correctly predicted required fields over all the fields predicted as required. Recall is the ratio of correctly predicted required fields over the number of actual required fields. NPV represents the ratio of correctly predicted optional fields over all the fields predicted as optional. Finally, specificity represents the ratio of correctly predicted optional fields over the number of actual optional fields.

We chose these metrics because they can evaluate the ability of an algorithm in predicting both required fields (using precision and recall) and optional fields (using NPV and specificity). A high value of precision and recall means that an algorithm can correctly predict most of required fields (i.e., the positive class); hence, we can avoid business loss caused by missing information. A high value of NPV and specificity means that an algorithm can correctly predict most of the optional fields (i.e., the negative class); users will have fewer unnecessary constraints during form filling. In other words, we can avoid users filling meaningless values that may affect the data quality.

In our application scenario, we aim to successfully relax a set of obsolete required fields to “optional,” while keeping the real required fields. Therefore, LACQUER needs to get high precision and recall values, which can preserve most of real required fields to avoid business loss. Meanwhile, the NPV value should be high, which means LACQUER can correctly avoid users filling meaningless values by relaxing the completeness requirements. Concerning the specificity, a relatively low value is still useful. For instances, a specificity value of 50% means LACQUER can reduce by half the data quality issues caused by meaningless values.

In the case of ARM, we set the minimum acceptable support and confidence to 5 and 0.3, respectively, as done in previous work [5, 33] in which it was applied in the context of form filling.

5.3 Performance (RQ2)

To answer RQ2, we measured the time needed to perform the training and predict the completeness requirement of target fields. The training time evaluates the ability of LACQUER to efficiently update its models when new input instances are added daily to the set of historical input instances. The prediction time evaluates the ability of LACQUER to timely suggest the completeness requirement during the data entry phase.

5.4 Impact of SMOTE and Endorser (RQ3)

LACQUER is based on two main modules: (1) SMOTE oversampling module, which tries to solve the class imbalance problem by synthetically creating new minor class instances in the training set (Section 4.2), and (2) the endorsing module, which implements a heuristic that aims to keep only the optional predicted instances with a certain level of confidence. To answer this RQ, we assessed the impact of these two modules on the effectiveness of LACQUER.

5.5 Threats to Validity

To increase the generalizability of our results, LACQUER should be further evaluated on different datasets from different domains. To partially mitigate this threat, we evaluated LACQUER on two datasets with different data quality: the PEIS dataset, which is proprietary and of high quality, and the NCBI dataset, which is public and was obtained from an environment with looser data quality controls.

The size of the pool of training sets is a common threat to all AI-based approaches. We do not expect this problem to be a strong limitation of LACQUER, since it targets mainly enterprise software systems that can have thousands of entries per day.

Since LACQUER needs to be run online during the data entry session, it is important to ensure seamless interaction with users. In our experiments (Section 5.3), LACQUER was deployed locally. The response time of its prediction complies with human-computer interaction standards. However, the prediction time depends on the deployment method (e.g., local deployment or cloud-based). This is not necessarily a problem, since different engineering methods can help reduce prediction time such as parallel computing and a cache for storing previous predictions.

5.6 Data Availability

The implementation of LACQUER, the NCBI dataset, and the scripts used for the evaluation are available at https://figshare.com/articles/software/LACQUER-replication-package/21731603 ; LACQUER is distributed under the MIT license. The PEIS dataset cannot be distributed due to an NDA.

6.1 Adaptive Forms

The approach proposed in this article is mainly related to approaches that implement adaptive forms for producing context-sensitive form-based interfaces. These approaches progressively add (remove) fields to (from) the forms, depending on the values that the user enters. They use form specification languages [19] or form definition languages [6] to allow form designers to describe the dynamically changing behavior of form fields. Such a behavior is then implemented through dedicated graphical user interface programming languages (such as Tcl/Tk) [50] or through server-side validation [6]. The dynamic behavior of a form has also been modeled using a declarative, business process-like notation (DCR—Dynamic Condition Response graph [49]), where nodes in the graph represent fields and edges show the dynamic relations among fields (e.g., guarded transitions); the process declarative description is then executed by a process execution engine that displays the form. However, all these works assume that designers already have a complete and final set of completeness requirements describing the adaptive behavior of the form during the design phase, which can be expressed through (adaptive) form specification/definition languages or tools. In contrast, LACQUER can automatically learn the different completeness requirements from the historical input instances filled by users without requiring any knowledge from the form designers.

Although some approaches [1, 16] try to automatically generate data entry forms based on the schema of the database tables linked to a form (e.g., using column name and primary keys), they can only generate some “static” rules for fields. For example, if a column is “not null” in the schema, then they can set the corresponding field in the form as (always) required. In contrast, LACQUER aims to learn conditions from the data so completeness requirements of form fields can be automatically and dynamically relaxed during new data entry sessions.

6.2 Comparing LACQUER with LAFF

The overall architecture (including the use of the endorser module) of LACQUER has been inspired by LAFF, a recent approach for automated form filling of data entry forms [5]. In this subsection, we explain the similarities and differences between the two approaches.

6.3 Using Bayesian Networks in Software Engineering Problems

Besides LAFF, BNs have been applied to different software engineering problems spanning over a wide range of software development phases, such as project management (e.g., to estimate the overall contribution that each new software feature to be implemented would bring to the company [34]), requirement engineering (e.g., to predict the requirement complexity to assess the effort needed to develop and test a requirement [44]), implementation (for code auto-completion [39]), quality assurance (e.g., for defect prediction [13, 28]), and software maintenance [41].

The main reason to use BN in software engineering (SE) problems is the ability of BN to address the challenges of dealing with “large volume datasets” and “incomplete data entries.” First, software systems usually generate large amounts of data [41]. For instance, to improve software maintenance, companies need to analyze large amounts of software execution data (e.g., traces and logs) to identify unexpected behaviors such as performance degradation. To address this challenge, Rey Juárez et al. [41] used BN to build an analysis model on the data, since BN can deal with large datasets and high-dimensional data while keeping the model size small and the training time low. Second, incomplete data is a common problem in SE [15, 38]. For example, some metrics in defect prediction datasets might be missing for some software modules. To solve this challenge, Okutan and Yıldız [38] and Del Águila and Del Sagrado [15] used BN to train prediction models, because of its ability to perform inference with incomplete data entries. These two challenges confirm our choice of using BN to solve the relaxing completeness problem. Specifically, these two challenges are aligned with the challenges of form filling. During data entry sessions, a form is usually partially filled and LACQUER needs to provide decisions on incomplete data. Besides, in our context, we need to deal with large datasets, since we mainly target enterprise software systems that can collect a huge number of entries every day.

7.1 Usefulness

The main goal of LACQUER is to prevent the entering of meaningless values by relaxing the data entry form completeness requirements. To assess the capability of LACQUER, we evaluated it with two real-world datasets, including a public dataset from the biomedical domain and a proprietary dataset from the banking domain. These two datasets are related to existing data entry forms.

Experiment results show that LACQUER outperforms baselines in determining completeness requirements with a specificity score of at least 0.20 and an NPV score higher than 0.72. In the context of completeness requirement relaxation, these results mean that LACQUER can correctly (i.e., NPV \(\ge\) 0.72) prevent the filling of at least 20% meaningless values. In addition, LACQUER can correctly determine (with precision above 0.76) when a field should be required with a recall value of at least 0.97. This recall value means that LACQUER can almost determine all the required fields. The high precision value shows that LACQUER rarely incorrectly predicts optional fields as required. In other words, LACQUER will not add much extra burden to users by adding more restrictions during the form-filling process.

As discussed in Section 5.2, LACQUER can determine more optional fields (i.e., a higher specificity) in the PEIS dataset than in the NCBI dataset due to the higher data quality of the former. Since we target data entry functionalities in enterprise software, we expect to find similar conditions in other contexts in which data entry operators follow corporate guidelines for selecting appropriate values that should be filled when a field is not applicable. In such contexts, LACQUER is expected to provide results that are similar to those achieved on the PEIS dataset.

7.2 Practical Implications

This subsection discusses the practical implications of LACQUER for different stakeholders: software developers, end-users, and researchers.

7.3 Combining LACQUER with LAFF

Despite the differences explained in Section 6, LACQUER and LAFF are complementary in practice. Both approaches can be combined as an AI-based assistant for form filling to help users fill forms and ensure better data quality.

Fig. 9 shows a possible scenario that uses both approaches together during a form-filling session. In this example, we assume that the user follows the sequential filling order. First, after filling in the company name field, LACQUER can already check whether the “monthly income” field is required or not. Since “monthly income” is a numerical field, LAFF cannot perform a prediction (LAFF only supports categorial fields). In this example, LACQUER determines that the field is required, hence the user should fill it out. The “Company type” and “Field of activity” fields are both categorical. For these two fields, based on the filled fields, first LACQUER determines the completeness requirement for each field. Once the user clicks on a field, LAFF is enabled to provide a ranked list of possible values that can be used for this field. If the decision of LACQUER on a field is optional, then LAFF can still be activated to provide suggestions as long as the user wants to fill in the field. Finally, let us assume that the “Tax ID” field (a numerical one) is optional by design. In this case, both LAFF and LACQUER are not enabled, since there is no need for LACQUER to relax a completeness requirement and the field is numerical and thus not compatible with LAFF.

Fig. 9.

View Figure