Erratum to: BicPAMS: software for biological data analysis with pattern-based biclustering

Upon publication of this article [1], it was brought to our attention that the online version of Table 8 contained an error. The correct Table 8 is shown below: Table 8 has been corrected in the original article. Reference 1. Henriques R, et al. BicPAMS: software for biological data analysis with pattern-based biclustering. permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver

Pattern-based biclustering emerged in recent years as an attempt to address these limitations [13]. Patterns coherently observed on a subset of rows, columns or nodes reveal homogeneous subspaces. In this context, patternbased biclustering algorithms rely on widely-researched principles for efficiently mining distinct patterns (including frequent itemsets, association rules or sequential patterns) in large databases as the means to identify these subspaces in real-valued matrices or weighted graphs.
The major benefits of pattern-based approaches for biclustering are: 1) scalable searches with optimality guarantees [11]; 2) possibility to discover biclusters with parameterizable coherency strength and coherency assumption (including constant, additive, plaid and orderpreserving plaid assumptions) [11,12,14]; 3) flexible structures of biclusters (arbitrary positioning of biclusters) and searches (non-fixed number of biclusters) [15,16]; 4) robustness to noise and missing values [11] by introducing the possibility to assign multiple symbols or ranges of values to a single data element; 5) easy extension for labeled data analysis using discriminative patterns [11]; 6) applicability to sparse matrices and network data [2,17]; 7) well-defined statistical tests to assess/enforce the statistical significance of biclusters [18], and 8) easy incorporation of constraints to guide the search [11].
Furthermore, results on biological data show their unique ability to retrieve non-trivial yet meaningful biclusters with high biological significance [2,11,14].
To integrate these dispersed contributions, BicPAMS (Biclustering based on PAttern Mining Software) is proposed to discover biclusters with customizable structure, coherency and quality, yet powerful default behavior. Bic-PAMS makes available earlier pattern-based biclustering algorithms (including BicPAM [11], BiModule [16] and DeBi [15]), well suited for expression data analysis. Furthermore, BicPAMS implements recent contributions that guarantee the applicability of biclustering towards network data (BicNET [17]), the discovery of orderpreserving and plaid models (BicSPAM [12] and BiP [14]) and the incorporation of domain knowledge [19].
This work is organized as follows. The remaining part of this section provides the background on patternbased biclustering. "Implementation" section describes the behavior of BicPAMS, covering the allowed inputs, parameters and visualization options. "Results" section provides empirical evidence of BicPAMS' role to unravel non-trivial and relevant putative modules from biological data. Finally, the major implications are highlighted.

Definition 1 Given a real-valued matrix (or network)
A with a set of rows (or nodes) X={x 1 , .., x n }, a set of columns Y ={y 1 , .., y m } and elements a ij relating row x i and column y j (or relating nodes x i and x j ): the biclustering task aims to identify a set of biclusters B={B 1 , .., B p }, where each bicluster B k =(I k , J k ) is defined by a subset of rows I k ⊂ X and columns J k ⊂ Y (or two subsets of nodes) satisfying specific criteria of homogeneity and statistical significance.
The placed homogeneity criteria determine the structure, coherency and quality of a biclustering solution, while the statistical significance criteria guarantees that the probability of a bicluster to occur deviates from expectations. The structure of a biclustering solution is defined by the number, size, shape and positioning of biclusters. A flexible structure has a non-fixed of arbitrarily positioned biclusters. The coherency of a bicluster is determined by the form of correlation among its data elements (coherency assumption) and by the allowed deviations per element against the perfect correlation (coherency strength). The quality of a bicluster is defined by the type and degree of tolerated noise. Figure 1 shows biclusters with different coherency assumptions for an illustrative symbolic dataset.

Definition 2 Given a matrix A, let the elements in a bicluster a ij ∈ B have coherency across rows (patterns on rows)
given by a ij = k j + γ i + η ij , where k j is the value expected for column y j , γ i is the adjustment for row x i , and η ij is the noise factor (determining the quality of the bicluster). Coherency across columns is identically defined over the transposed matrix, A T . LetĀ be the amplitude of values in A. Given A, coherency strength is a real value δ ∈ [0,Ā], such that a ij = k j +γ i +η ij and η ij ∈ [−δ/2, δ/2].

Definition 3
The properties of a ij elements define the coherency assumption: constant when γ =0 and additive otherwise. Multiplicative assumption is observed when a ij is better described by k j γ i + η ij . Symmetries can be accommodated on rows, a ij c i where c i ∈ {1,-1}. Order-preserving assumption is observed when the values along the subset of columns induce the same linear ordering per row. A plaid assumption considers the cumulative effects associated with elementar contributions from multiple biclusters on areas where their columns and rows overlap. Definition 4 Given a bicluster B = (I, J), the bicluster pattern ϕ B is the set of expected values (k j ) in the absence of noise (η ij = 0) and adjustments (γ i = 0) according to a fixed ordering of columns: {k j | y j ∈ J}; while its support, |I|, is the number of rows satisfying the pattern.
Consider the bicluster (I 2 , J 2 ) = ({x 1 , x 2 }, {y 1 , y 2 , y 4 , y 5 }) in N + 0 from Fig. 1 with an additive coherency assumption across rows. This bicluster can be described by a ij = k j +γ i with the pattern ϕ = {k 1 = 1, k 2 = 0, k 4 = 1, k 5 = 0}, supported by two rows with additive adjustments γ 1 = 5 and γ 2 = 1. Pattern-based Biclustering. The recently exploited synergies between biclustering and pattern mining paved the rise of a new class of algorithms, generally referred as pattern-based biclustering algorithms [13]. Pattern-based biclustering algorithms are natively prepared to efficiently find exhaustive solutions of biclusters and offer the unique possibility to affect their structure, coherency and quality [13]. This behavior justifies the increasing attention paid in recent years to this class of biclustering algorithms by the bioinformatics community for biological data exploration [11,12,[14][15][16][17]20].
Let L be a set of items. In the scope of pattern mining research [21], a pattern is a frequent composition of items P, either an itemset (P ⊆ L), association rule (P : P 1 → P 2 where P 1 ⊆ L∧P 2 ⊆ L) or sequence (P = P 1 · · · P n where P i ⊆ L). Given a set of observations D={P 1 , .., P n }, let a full-pattern be a pair (P, P ), where P is a pattern and P is the set of observations in D containing P. Let a closed pattern to be a pattern without supersets with the same support (∀ P ⊃P | P | < | P |).
Given a real-valued matrix A, pattern-based biclustering relies on mappings from A into D and on pattern mining methods able to discover all closed full-patterns, which are used to derive all maximal biclusters satisfying certain coherency (e.g. η ij < ) and structure criteria (e.g.
. A maximal bicluster with regards to a specified homogeneity criteria is a bicluster that cannot be extended with additional rows or columns while still satisfying the target criteria. See [22] for a detailed formal view on pattern-based biclustering.
In this context, a pattern-based biclustering solution is optimal with regards to certain coherency, quality and structure criteria. The optimality of pattern-based biclustering algorithms is linked with their exhaustive and unrestricted behavior, contrasting with peer greedy and stochastic biclustering algorithms.
The major potentialities of pattern-based biclustering against alternative biclustering approaches include the possibility to: perform efficient searches with guarantees of optimality [12]; discover biclusters with parameterizable coherency assumption and strength [11,12]; guarantee robustness to noise, missing values and discretization problems through the possibility of assigning or imputing multiple values or symbols to a single data element [11]; discover structures with a non-fixed number and positioning of biclusters possibly characterized by plaid effects [14,16]; annotate biclusters with a measure of their statistical significance [18]; extend their applicability towards network data and sparse data matrices [2,17]; and incorporate domain knowledge from user expectations, knowledge repositories and literature in the form of constraints to guarantee a focus on biologically relevant and non-trivial biclusters [22]. Related work. Following Madeira and Oliveira's taxonomy [1], biclustering algorithms can be categorized according to their homogeneity criteria (determined by the underlying merit function) and type of search (defined by whether the merit function is applied within a greedy [7,23], exhaustive [10,11] or stochastic [9] algorithmic setting). Hundreds of algorithms were proposed in the last decade to discover biclusters satisfying specific forms of homogeneity, as shown by recent surveys on biclustering algorithms for biological data analysis [3][4][5][6]. As a result, some of the algorithms with most visibility have been made publicly available recurring to different software, such as BicAT 1 [24], biclust 2 [25], Expander 3 [10] or BicOverlapper 4 [26]. However, the available biclustering algorithms (regardless of whether they are provided or not with adequate interfaces) assume very specific forms of homogeneity and therefore do not support the enumerated benefits of pattern-based biclustering approaches. Table 1 synthesizes the inherent properties of the stateof-the-art pattern-based biclustering algorithms and how they tackle the problems of peer biclustering algorithms. Despite their inherent benefits, they are not yet accessible through adequate graphical or application programming interfaces (GUI/API), and their contributions remain Table 1 Recent breakthroughs on pattern-based biclustering: algorithms and tackled limitations   Contribution Biological output Behavior Tackled limitations

Constant Models
BicPAM [11] Putative functional modules robust to noise, such as co-expressed genes with a regulatory pattern given by possibly different expression levels across a subset of conditions.
Algorithms consistently combining preprocessing, pattern mining (itemsets and association rules) and postprocessing procedures to guarantee the flexibility and robustness of the outputs.

Multiplicative and Additive Models
BicPAM [11] Modules with shifting and scaling factors to deal with the distinct responsiveness of biological entities and handle biases introduced by the applied measurement.
Iterative discovery of pattern differences (shifts) and least common divisors (scales), together with pruning strategies, to learn additive and multiplicative models.
Precise modeling of shifting and scaling factors across rows; Flexible structure and parameterizable quality.

Order Preserving Model
BicSPAM [12] Coherent variation of gene expression or molecular concentrations across samples or within a temporal progression (such as stages of a disease or drug response).
Biclustering is parameterized with enhanced sequential pattern miners (by ordering column indexes per row according to the observed values) to flexibly discover noise-tolerant orderings.
Surpasses efficiency and robustness issues of exhaustive peers; Flexible structures with guarantees of optimality, addressing the problems of greedy peers.

Symmetric
Bic(S)PAM [11,12] Modules associated with biological processes simultaneously capturing activation and repression mechanisms within transcriptomic, proteomic or metabolic data.
Combinatorial sign-adjustments (together with pruning principles) to model symmetries and integrate them with scales, shifts and orderings.
Discovery of non-constant biclusters with symmetries; Parameterizable properties.

Network Modules
BicNET [17] Coherent modules in homo/heterogeneous biological networks with weighted/labeled interactions. Modules able to capture non-trivial forms of behavior and accommodate less-studied biological entities.
Extension of previous contributions towards biological networks. For this end, new data structures and searches are proposed to effectively and efficiently deal with the inherent sparsity of network data.
Discovery of non-dense modules; Robustness to noisy and missing interactions; Scalable for large networks.

Plaid Model
BiP [14] Overlapping regulatory influence in expression data (cumulative effects that multiple biological processes have on a gene at a particular time) and network data (cumulative effects in interactions belonging to multiple modules).
Extended searches to recover excluded areas (due to cumulative contributions on regions where biclusters overlap) and to remove noisy areas. New composition functions and relaxations to deal with noise and non-linear cumulative effects.
Addresses the exact additive plaid assumption with relaxations; No need for all the data elements to follow a plaid assumption; Models non-constant biclusters.

Constraints
BiC2PAM [19] Biological modules in accordance with user expectations (e.g. non-trivial homogeneity, satisfying a given pattern or preferred regulatory behavior (such as repression)) or with consistent functional terms.
Extended searches to benefit from background knowledge, including: constraints with succinct, anti-monotone and convertible properties, and incorporation of terms from knowledge repositories.
Focus on regions of interest; Efficiency gains; Removal of uninformative values.
dispersed, being the possibility to consistently integrate them still uncertain.

Implementation
BicPAMS (Biclustering based on PAttern Mining Software) is the first tool consistently combining state-ofthe-art pattern-based biclustering algorithms and making them available within usable interfaces (GUI and API). Figures 2 and 3 provide snapshots of the graphical interface of BicPAMS (where parameters P1 to P20 can be used to determine the desirable properties of the output). First, BicPAMS is described according to the possibilities to parameterize the coherency, structure and quality of its outputs, and the principles to guarantee the efficiency of the underlying searches. We also visit additional contributions of BicPAMS associated with the exploration of potentialities inherent to the integration of pattern-based biclustering algorithms. Second, we cover implementation details associated with the behavior of BicPAMS and the provided interfaces.

Pattern-based biclustering with BicPAMS
Coherency of biclusters. As highlighted in Table 1

Structure of biclusters.
BicPAMS relies on the iterative application of dedicated pattern mining searches to guarantee that biclustering can be performed in the presence of a meaningful stopping criteria [P12], such as the Efficiency. BicPAMS also relies on enhanced pattern mining searches able to explore efficiency gains from the biclustering task, inputted constraints and desirable structures [P17,P18]. BicPAMS supports frequent itemset mining and association rule mining (including Apriori-based, vertical or dedicated frequent patterngrowth searches [21]), as well as sequential pattern mining (including state-of-the-art and dedicated searches [27]). New searches based on annotated pattern-based trees (F2G search [28]) and diffsets are implemented within BicPAMS to surpass the problems associated with bitsetbased searches, as well as searches able to seize efficiency gains from item-indexable properties (IndexSpan [29]). These searches are integrated with heuristics, guaranteeing an effective pruning of the search space in the presence of constraints such as minimum number of columns. BicPAMS also makes available data structures to deal with sparse data [17] that guarantee a heightened timeand-memory efficiency in the presence of network data.
Finally, the application programming interface (API) of BicPAMS can be used to explore additional efficiency gains from non-optimal searches (mining approximate patterns) and the application of pattern mining within distributed/partitioned data settings.
Synergies. BicPAMS provides the unprecedented possibility to consistently integrate the previously described options, thus combining the contributions of BicPAM [11], BicNET [17], BicSPAM [12], BiP [14], DeBi [15] and BiModule [16]. Furthermore, BicPAMS can incorporate background knowledge according to the contributions made available in BiC2PAM [19], such as the possibility to remove uninformative elements. The API further Table 2 BicPAMS: input data, major parameters, and output models The accepted file formats include attribute-relation files (.ARFF) and standard matrix files (such as .TXT).
The first line of standard matrix files should specify the column identifiers, while the first entry of each line should specify the row identifier. Tabular data can be either delimited by tabs, spaces or commas.

P2 Network
BicPAMS accepts any input file format (such as .TXT or .SIG) assuming that: the first line specifies the column identifiers, and each other line specifies an interaction/entry within the network. An entry specifies the nodes and the association strength. Entries can be either delimited by tabs, spaces or commas. In addition to the file, the column index identifying the first node, second node and association strength needs to be inputted. Illustrating, for a network with header "idProteinA, nameProteinA,idProteinB,nameProteinB,weight", the user should fix (node1,node2,score) indexes as (0,2,4) or (1,3,4). Finally, the user can specify whether each entry is directional from the first node towards the second node or bidirectional. Bidirectional entries increase the density of the network.

Desirable
Biclustering Models

P3 Coherency Assumption
The coherency assumption defines the correlation of values within a bicluster. In constant models, an observed pattern (possibly containing different items) is preserved across rows (or columns). In additive or multiplicative models, shifting or scaling factors are allowed per row (or column) in order to allow meaningful variations of the original pattern. In order-preserving models, the values per row induce the same ordering across columns. A plaid model considers the cumulative effect of the contributions from multiple biclusters on areas where their rows and columns overlap. Previous models can further accommodate symmetric factors.

P4 Coherency Strength
The number of items determines the allowed deviations from expected values. Illustrating, a gene expression matrix parameterized with 5 items will have 2 levels of activation ({1,2}), 2 levels of repression ({-1,-2}) and 1 level of unchanged expression ({0}). By going beyond the differential values, BicPAMS enables the discovery of non-trivial yet coherent and meaningful correlations. To maintain consistency, additive (multiplicative) models should be used with an uneven (even) number of items. When considering order-preserving models, the number of items should be increased to balance the degree of co-occurrences versus precedences.

P5 Quality
This field specifies the maximum number of allowed noisy/missing elements (determining the minimum overlapping threshold for merging procedures). The tolerance of biclusters to noise can be additionally addressed using noise handlers (see mapping options) and alternative postprocessing procedures.

P15 Pattern Representation
Closed patterns (default option) enable the discovery of maximal biclusters (biclusters that cannot be extended without the need of removing rows and columns). Maximal patterns gives a preference towards flattened biclusters, possibly neglecting both vertical and smaller biclusters. Finally, the use of simple/all frequent patterns leads to biclustering solutions with a high number of biclusters (possibly contained by another bicluster), which can be useful to guide postprocessing steps. As the user specifies one of these three options, the available pattern miners are dynamically updated.

P16 Orientation
Coherency can be either observed across rows (default) or columns (searches are applied on the transposed matrix). When the number of columns highly exceeds the number of rows (or vice-versa when searches are applied on the transposed matrix), pattern miners with vertical data formats such as Eclat should be preferred.

Output
Upon successfully running BicPAMS, a textual and graphical display of the outputs is provided. The user can select the level of details associated with the outputted biclustering solution (statistics only, list of rows and columns per bicluster, disclosure of values per bicluster).  Table 2) P6 Normalization Depending on the properties of the input data, the user can either normalize data per Row, Column or for the Overall data elements or ignore normalization by selecting the None option. Both outliers and missing values are handled separately.

P7 Discretization
Real-valued data needs to be discretized to apply pattern-based biclustering (see noise handling to understand how BicPAMs guarantees robustness to discretization drawbacks). The user can select the cut-off points of a Gaussian distribution (default) or fixed ranges of values (equal sized intervals after excluding outliers). Note that fixed ranges can lead to an imbalanced distribution of items. The user can bypass this option for symbolic data by selecting the None option.

P8 Noise Handler
Multi-item assignments can be considered to handle deviations on the expected values within a bicluster caused by noise or discretization issues. By selecting this option, 2 items are assigned to elements with a value near a boundary of discretization (value in range c ∈ ). In this context, a data element becomes associated with a varying number of items, thus increasing the size of data for analysis.

P9 Symmetries
This option is dynamically selected if the input data is composed by positive and negative values (as it naturally affects the properties of the outputted biclusters). When using symmetric ranges, additive (multiplicative) models should be parameterized with an odd (even) number of items to guarantee consistent shifts (scales).

Missings Handler
The user can specify what happens in the presence of missing values. Since BicPAMS is natively prepared to analyze sparse data, the Remove option (default) simply signals the algorithms to exclude missings from the searches. Alternatively, the Replace option uses WEKA's imputation methods to fill missings (the error of imputations can be minimized by simultaneously activating a noise handler). We suggest the use of Remove option for network data and other meaningfully sparse datasets since BicPAMS is able to discover biclusters with missing interactions (see Quality parameter).

P11 Remove Uninformative Elements
This option supports the possibility to remove uninformative data elements. Zero Entries can be selected to remove the {0}-items, while the Differential option is used to focus on items with high absolute value (e.g. {-3,-2,2,3} when |L|=6). Uninformative elements may correspond to: 1) weak interactions in networks, 2) unchanged expression, 3) healthy evaluations from clinical data, among others.

Mining
Options (includes P3, P15 and P16 from Table 2) P12 Stopping Criteria The search algorithm runs until any of the available stopping criteria is met. The available options are: 1) minimum number of biclusters before merging (default), 2) minimum covered area by the discovered biclusters (as a percentage of the elements of the input data matrix or network), and 3) minimum support threshold (minimum number of rows per bicluster specified as a fraction of overall rows). The value associated with the selected option should be additionally specified. We suggest the definition of a high number of biclusters (>50) as the default option, in order to guarantee an adequate exploration of the input dataset.

P13 Minimum Columns
The minimum number of columns per bicluster can be optionally inputted to promote efficiency and align the outputs according to user expectations. A good principle to fix this value is to use the square root of the number of columns (interactions per nodes) of the input matrix (network).

P14 Iterations
BicPAMS default behavior relies on two iterations. For data with large coherent regions that may prevent the discovery of smaller (yet relevant) regions, the number of iterations can be increased to guarantee their discovery. On every new iteration, 25% of the most selected data elements (from the biclusters discovered from the previous iteration) are removed to guarantee a focus on new regions. 3 iterations already guarantee an adequate space exploration for hard data settings.

P17 Pattern Miner
The available pattern mining algorithms are dynamically provided based on the selected coherency assumption and pattern representation. Sequential pattern miners (SPM) are provided for order-preserving models: PrefixSpan and IndexSpan (an optimized algorithm able to explore gains in efficiency from the item-indexable properties) are made available for simple pattern representations, while BIDE+ is provided for closed pattern representations. Frequent itemset miners (FIM) are selected for the remaining coherency assumptions. AprioriTID, F2G (pattern-growth method for data with a large extent of coherent areas) and Eclat (vertical method for data with a high number of columns) are made available for simple pattern representations. CharmDiffsets, AprioriTID and CharmTID are made available for closed pattern representations, while CharmMFI with diffsets is provided for maximal pattern representations.

P18 Scalability
This option specifies whether data partitioning principles are applied or not to guarantee the scalability of the searches (only suggested for data with >100 Mb).  [29]): heuristic (default option) for an efficient quasi-exhaustive merging; and combinatorial and multi-support FIM alternatives for an exhaustive yet more costly postprocessing step.

P20 Filtering
Filtering is essential to guarantee compact solutions (applied after merging). A biclustered is filtered if it has not enough Dissimilar Elements, Dissimilar Rows or Dissimilar Columns against a larger bicluster. Considering a filtering option with 20% of dissimilar elements. In this context, biclusters sharing more than 80% of their elements against a larger bicluster are removed.
supports the specification of constraints and the integrative biclustering analysis of experimental data with annotations derived from knowledge repositories. In this context, although BicPAMS offers an environment with a substantial number of parameters, it makes available default and dynamic parameterizations that are suitable for the majority of data contexts (see Table 4).
Furthermore, BicPAMS explores efficiency gains from particular combinations of parameters. This is, for instance, the case when BicPAMS is applied with multiple coherency or quality criteria at a time. In this context, the search benefits from new heuristics (based on the principle that biclusters with stricter coherency or quality are contained in biclusters with more flexible coherency or quality) and the joint application of pre-and postprocessing procedures.

On how to use BicPAMS
Input and output. BicPAMS supports the loading of input data according to a wide-variety of tabular and network data formats (see Tables 2 and 3). Upon running BicPAMS, when the stopping criteria is achieved, a success message is displayed, enabling the visualization of the output. Both graphical and textual presentations (heatmaps and signal signatures) of the found biclusters are provided. Biclusters can be filtered, sorted and exported to be stored in knowledge bases or visualized on alternative software. Figure 4 provides an illustrative application of Bic-PAMS for an inputted dataset (either in network or matrix format), showing the outputted biclusters for varying coherency assumptions. For this analysis, we assumed |L| = 4, fixed discretization ranges and the assignment of multi-items for an adequate tolerance to noise.

Graphical interface (GUI).
The desktop interface can be used to soundly parameterize pattern-based biclustering algorithms, as well as to visualize their output. Figures 2  and 3 provide illustrative snapshots. Soundness is guaranteed by: performing automatic form checks, disabling inconsistent fields when specific parameters are selected, and adequately displaying possible causes of errors (such as timeout alerts for heavy requests or data format inconsistencies). Console, API and source-code. Alternatively to the previous interfaces, BicPAMS makes available a console to facilitate its invocation within language-independent scripts, as well as a Java API, the respective source code and the accompanying documentation. The API is essential to: extend the behavior of pattern-based biclustering

P15 Pattern representation
Closed Closed pattern representations enable the discovery of maximal biclusters (biclusters that cannot be extended without removing rows or columns).

Patterns on rows
In accordance with Def.2. Considering expression data where rows correspond to genes, a bicluster with coherency across rows is defined by a group of genes with the same pattern along a subset of conditions. When rows correspond to conditions, a less-trivial bicluster is given by a group genes with preserved expression spanning a subset of conditions.

Row
Normalization of values per biological entity or sample.

P7 Discretization
Gaussian Cut-off points of a learned Gaussian curve to minimize imbalanced distributions of items.

P8 Noise handler
None By default multi-item assignments are deactivated for an easy interpretation of results. Nevertheless, we suggest the selection of multi-item assignments to guarantee a heightened robustness to discretization drawbacks and noise. Guarantees the removal of small and highly coherent regions in the dataset (after the 1st iteration) to enable the discovery of less-trivial biclusters. This number can be increased to promote a more even distribution of biclusters across the regions of the inputted data.

P17 Pattern miner
Dynamic From empirical evidence, CharmDiff is suggested for closed patterns, CharmMFI for maximal patterns, and F2G for simple patterns. When order-preserving coherency is inputted, IndexSpan is suggested by default.

P18 Scalability
Dynamic Option activated in the presence of very large datasets (>20 million elements under a constant assumption and >1 million elements for the remaining coherency assumptions).

Closing P19 Merging Heuristic
Guarantees an efficient yet quasi-exact postprocessing.

P20 Filtering 40% dissimilar elements
Guarantees an adequate level of dissimilarity. Biclusters sharing more than 60% of their elements with a larger bicluster are removed.    MSigDB Oncogenic Signatures 9 ± 1, 5 The Molecular Signatures database (MSigDB) tests the enrichment of genes with potential to cause cancer. Interestingly, the majority of the discovered biclusters have a single delineated oncogene (signature with considerably higher enrichment than peer signatures). A few illustrative signatures include: VEGFA UP with V1 DN (8.2E-8) corresponding to genes down-regulated by treatment with angionic factor VEGFA; RPS14 DN with V1 UP (4.3E-11) corresponding to genes up-regulated in CD34+ hematopoietic progenitor cells after knockdown of RPS14; or CAMP UP with V1 UP (3.4E-9) associated with genes up-regulated in primary thyrocyte cultures in response to cAMP signaling. This knowledge further discriminates the putative role of each bicluster. Gene Ontology GO Biological processes 298 ± 90 All biclusters show a high number of functionally coherent terms associated with cellular biological processes. An analysis of the enrichment for some biclusters is provided in Table 6. Complementary analyzes are provided in Table 9. Similarly to cellular components, the knowledge of the enriched molecular functions can be used to enlarge the GO-based analysis of biclusters. Each bicluster was found to be associated with a compact set of molecular functions consistently related with the molecular mechanisms underlying immunological responses to chemotherapy. Considering two randomly selected biclusters: the first bicluster showed enriched terms (with corrected p-values below 1E-6) associated with antigen binding and the binding of amide, protein complexes and small proteins (inc. chemokine receptor); while the second bicluster showed enriched terms (<1E-6) associated with protein kinase binding and regulation, structure-specific DNA binding, and ATPase activity.

Regulation
algorithms for other tasks (such as classification and indexation), and adapt the current behavior to guarantee an optimum ability to handle biological data with specific regularities. Detailed scenarios showing advanced possibilities made available in the API of BicPAMS are provided in the software's webpage. . BicPAMS supports the parameterization of two post-processing procedures: maximum degree of noisy or missing elements per bicluster (using merging procedures [29]) [P5/P19] and dissimilarity criteria (using filtering procedures [11]) [P20]. The API further provides the possibility to specify a desirable minimum homogeneity threshold to extend or reduce the target biclusters according to a parameterizable merit function [11]. Tables 2 and 3 provide an in-depth description of each of these parameters, showing their default values and how they can be modified according to the properties of the input data and desirable outputs. For an exhaustive exploration of biological data without apriori knowledge of the desirable outputs, BicPAMS can be iteratively applied with varying coherency assumptions, coherency strength (|L| ∈ {3, 4, 5}) and quality ({60%, 80%, 100%}). Table 4 discusses the default and data-driven parameterizations provided by BicPAMS, showing their adequacy for exploratory yet robust biological data analysis.

Scalability/limits.
Although biclustering is inherently a computationally complex task, BicPAMS is natively prepared to analyze large-scale matrices/networks (>1 Gb) and, under strict optimality criteria, data with more than one million of entries (∼200 Mb). BicPAMS provides the possibility to select data partitioning procedures. Assuming coherency across rows (patterns on rows), partioning procedures should be applied when (|X| > 20000 ∧ |Y | > λ × 1000) is satisfied for the constant assumption (λ = 1) or remaining coherency assumptions (λ = 0.1). In this context, BicPAMS is able to efficiently analyze expression data with more than 20000 genes (magnitude of human genome) in hundreds of conditions, as well as over sparse biological networks with over 20000 nodes.
Testing cases. Synthetic data (resembling biological data), gene expression data and biological networks with >100 Mb are provided with BicPAMS for testing purposes. In BicPAMS webpage, we provide study cases using both synthetic and real data with varying properties to illustrate the multifaceted potentialities of the software.

Results
Additional file 1 provides extensive experiments that extend the already available assessments of pattern-based biclustering algorithms [2,11,12,[14][15][16] towards: new synthetic and real data, and new performance views (including metrics of completeness, precision and accuracy). In these experiments, the performance of 15 distinct biclustering algorithms was for the first time compared for data contexts with varying size, regularities, and amount and type of noise. The gathered results confirm the enumerated advantages of BicPAMS, including its unique ability to efficiently find exhaustive and flexible solutions with superior robustness to noise.
Follows a brief analysis of some of the results gathered from applying BicPAMS to discover regulatory modules in expression and network data. Additional file 1 extends these analyzes (concerning both the functional enrichment and transcriptional regulation of the discovered modules) and demonstrates the relevance and completeness of pattern-based biclustering outputs against the outputs produced by alternative state-ofthe-art biclustering algorithms. The biological relevance of the biclusters was given by the assessment of the over-represented functional terms using an hypergeometric test after Bonferroni correction. We considered a term to be highly enriched if it has a corrected p-value below 0.01.
Case studies on expression data analysis. Three gene expression datasets were used: dlbcl dataset (660 genes, 180 conditions) gathering human responses to chemotherapy [30], hughes dataset (6300 genes, 300 conditions) to analyze nucleosome occupancy [31], and gasch dataset (6152 genes, 176 conditions) with Yeast responses to environmental stimuli [32]. The goal is to discover coherent expression patterns corresponding to known and putative transcriptional modules associated with the experimental goal (such as elicited immune responses in dlbcl and stress responses in gasch). Table 5 provides a functional enrichment analysis of pathways, cell lines, transcription factors (TFs) and gene ontology (GO) terms associated with 182 pattern-based biclusters found in the dlbcl dataset. BicPAMS was applied with a constant coherency assumption, multi-item assignments, 80% quality, 50% dissimilarity and 3 iterations. The number of genes per bicluster varies between 89 and 166. The enrichment was computed using the Enrichr web tool [33] against terms from the following databases: KEEG, WIKI, Reactome and BioCarta, human PPIs, Gene Ontology, NCI-60 and cancer cell line Encyclopedia, Human Gene Atlas and MSigDB. Each database annotates groups of genes with dedicated terms. Table 5 shows that all the biclusters found are associated with dissimilar sets of coherent terms. The analysis of enriched pathways, TFs, cancer cell lines, target cells, oncogenic signatures and GO terms confirm that the discovered biclusters are associated with meaningful and well-defined putative cellular responses to chemotherapy. Similar analyzes were conducted for the hughes and gasch data, revealing an identical average number of enriched pathways per bicluster and a significantly higher average number of enriched TFs and GO terms per bicluster. Table 6 lists a compact subset of biological processes with significantly enriched terms for pattern-based biclusters found in dlbcl, hughes and gasch datasets. The analysis of the enriched transcription factors (TFs) of these modules confirms their role in regulating cellular responses to chemotherapy (human) [34] and stress conditions (yeast) [35]. Table 7 further shows the properties of an illustrative subset of pattern-based biclusters with high biological significance as verified by the number of highly enriched terms after Bonferroni correction. These biclusters could not be identified by peer biclustering methods due to the presence of noise-tolerant patterns with multiple expression levels (B1, B2 and B5) and non-constant coherency assumptions (B3, B6, B8). Additional file 1: Tables S6-S9 further stress the relevance of discovering biclusters with plaid, ordering-preserving and symmetric assumptions. Figure 5 plots four pattern-based biclusters from gasch data with distinct coherent responses of genes to heat shock at different points in time. These biclusters rely on constant, multiplicative, additive and symmetric assumptions, each associated with noise-tolerant patterns with five expression levels (|L| = 5). Understandably, alternative state-of-the-art biclustering algorithms are not able to discover identical biclusters due to the restrictive   5 Pattern-based biclusters retrieved from gasch data following a constant assumption with symmetries a, multiplicative assumption with symmetries (b), and additive assumption c and d  Y6 order-preserving 176×12 Transport of organic acids (incidence on aminoacid transmembrane transport). Y7 order-preserving 235×9 Oxidation-reduction process and metabolism of aminoacids. Y8 order-pres. (few high a ij ) 146×8 Transport of molecules (highest enrichment found for drug transmembrane).
STRING (human) H1 dense (high noise-tolerance) 811×28 Multiple metabolic processes with incidence on transcription activity. H2 constant (few high a ij ) 693×14 Regulation of intracellular signal transduction (over twenty highly enriched terms). H3 constant (few high a ij ) 645×10 Regulation of molecular functions with incidence on catalytic activity. H4 order-preserving 720×24 Establishment of protein localization (protein targeting to ER and membrane). H5 order-preserving 733×29 Protein phosphorylation and its subterms.
Case studies on network data analysis. Four biological networks were extracted from DryGIN [36] and STRING v10 [37] databases ( Table 8). The goal is to discover putative functional modules given by non-trivial yet coherently interconnected subsets of biological entities. Table 9 illustrates some of all highly enriched biclusters discovered by BicPAMS over the biological networks in Table 8, gathering modules with varying: tolerance to noise (0-15% noisy interactions per bicluster), amount of missing values (0-20% missing interactions per bicluster), coherency assumptions (dense/differential, constant and order-preserving) and coherency strength (D 1 -D 4 biclusters with L = {−2, −1, 1, 2}, Y 1 −Y 4 and H 1 −H 2 with L = {1, 2, 3}, Y 5 and H 3 with L = {1, 2, 3, 4}). The biclusters were discovered using multi-item assignments to guarantee their robustness to noise. The results show that all biclusters have highly enriched terms, and the enriched terms per bicluster were also found to be taxonomically related (see Additional file 1). These results further suggest that the found modules are characterized by cohesive putative biological functions. Table 10 characterizes some the enriched pattern-based biclusters, reinforcing the role of BicPAMS to find modules with varying shape, coherency and quality; non-trivial yet biologically meaningful as shown by the number of enriched terms after correction.

Conclusions
BicPAMS consistently integrates the state-of-the-art contributions from pattern-based biclustering within graphical, scripting and application programming interfaces for the analysis of biological data. BicPAMS is essential for the user-assisted unsupervised exploration of biological data as it overcomes the commonly placed restrictions by peer biclustering algorithms and provides the unprecedented possibility to parameterize the properties of the biclustering solutions. Unprecedentedly, BicPAMS offers the possibility to customize the coherency (including coherency assumption, orientation and strength), quality (including tolerance to noise and missing values), structure and statistical significance (including minimum number of rows and/or columns) of the outputted biclusters. BicPAMS is applicable to dense or sparse, symbolic or real-valued data, and optionally able to incorporate domain knowledge. In order to guarantee the usability of this parametrically rich environment, default parameterizations and simple guidelines (according to the properties of input data and desired output) are provided. BicPAMS further supports multiple data formats and representations of the output, verifying the soundness of requests. Empirical evidence shows that BicPAMS is able to efficiently and effectively discover non-trivial yet coherent biclusters that are robust to noise and biologically significant.