Graph-Based Discriminative Learning for Location Recognition

Abstract

Recognizing the location of a query image by matching it to an image database is an important problem in computer vision, and one for which the representation of the database is a key issue. We explore new ways for exploiting the structure of an image database by representing it as a graph, and show how the rich information embedded in such a graph can improve bag-of-words-based location recognition methods. In particular, starting from a graph based on visual connectivity, we propose a method for selecting a set of overlapping subgraphs and learning a local distance function for each subgraph using discriminative techniques. For a query image, each database image is ranked according to these local distance functions in order to place the image in the right part of the graph. In addition, we propose a probabilistic method for increasing the diversity of these ranked database images, again based on the structure of the image graph. We demonstrate that our methods improve performance over standard bag-of-words methods on several existing location recognition datasets.

Appendix: Derivation for the iterative usage of the update factor

Here we provide the full derivation of how our diversity reranking method updates the probability scores for each image each time we select a new image for the shortlist. Let $n$ denote the number of images that have already been selected for the ranked shortlist $RL = \{i_1, i_2, ..., i_n\}$. For any image $u$ in the as-yet-unselected set of images $\mathcal {I} \setminus RL$, we wish to compute

$$\begin{aligned} P_u^{(n)} = \Pr (X_u = 1 | X_{i_1} = 0, ..., X_{i_n} = 0), \end{aligned}$$

which is the conditional probability of image $u$ matching the query image given that all the $n$ images in the current ranking list do not match the query. To simplify the analysis, we separately consider the cases $n=1$, $n=2$, and the general case $n=k$ (note that in Section 5 we denote $P_u^{(1)}$ as $P'_u$).

• $n = 1$: From Eq. (2), Section 5 derives the update factor in Eq. (3) used to compute $P_i^{(1)}$:

  $$\begin{aligned} P_u^{(1)} = P_u \frac{1 - \frac{P_{u,i_1}}{P_u}P_{i_1}}{1-P_{i_1}} \end{aligned}$$

  where $P_u$ is the original probability estimate from our GBP approach.

• $n = 2$: From the definition of conditional probability, we have that

  $$\begin{aligned} P_u^{(2)}&= \Pr (X_u = 1 | X_{i_2} = 0, X_{i_1} = 0) \nonumber \\&= \frac{\Pr (X_u = 1, X_{i_2} = 0, X_{i_1} = 0)}{\Pr (X_{i_2} = 0, X_{i_1} = 0)} \nonumber \\&= \frac{\Pr (X_u = 1, X_{i_2} = 0 | X_{i_1} = 0)\Pr (X_{i_1} = 0)}{\Pr (X_{i_2} = 0, X_{i_1} = 0)} \nonumber \\ \end{aligned}$$

  (7)

  Recall from Section 5 that we make two simplifying assumptions:

  • 1. Independence of $(X_{i_1} = 0)$ and $(X_{i_2} = 0)$ for distant $i_1$ and $i_2$:

    $$\begin{aligned} \Pr (X_{i_2} = 0, X_{i_1} = 0) = \Pr (X_{i_2} = 0) \Pr (X_{i_1} = 0) \end{aligned}$$

  • 2. Independence of $(X_u = 1, X_{i_2} = 1)$ and $(X_{i_1} = 0)$:

    $$\begin{aligned} \Pr (X_u = 1, X_{i_2} = 1 | X_{i_1} = 0) = \Pr (X_u = 1, X_{i_2} = 1) \end{aligned}$$

  Given these assumptions, we can simplify Eq. (7) as follows:

  $$\begin{aligned}&P_u^{(2)} = \Pr (X_u = 1 | X_{i_2} = 0, X_{i_1} = 0) \nonumber \\&= \frac{\Pr (X_u = 1, X_{i_2} = 0 | X_{i_1} = 0) \Pr (X_{i_1} = 0)}{\Pr (X_{i_2} = 0) \Pr (X_{i_1} = 0)} \nonumber \\&= \frac{\Pr (X_u = 1, X_{i_2} = 0 | X_{i_1} = 0)}{\Pr (X_{i_2} = 0)} \nonumber \\&= \frac{\Pr (X_u = 1 | X_{i_1} = 0) - \Pr (X_u = 1, X_{i_2} = 1 | X_{i_1} \!=\! 0)}{1-\Pr (X_{i_2}\!=\!1)} \nonumber \\&= \frac{\Pr (X_u = 1 | X_{i_1} = 0) - \Pr (X_u = 1, X_{i_2} = 1)}{1-\Pr (X_{i_2}=1)} \nonumber \\&= \frac{P_u^{(1)} - \Pr (X_u = 1 | X_{i_2} = 1) \Pr (X_{i_2} = 1)}{1-\Pr (X_{i_2}=1)} \nonumber \\&= \frac{P_u^{(1)} - P_{u,i_2} P_{i_2}}{1-P_{i_2}} = P_u^{(1)} \left( \frac{1 - \frac{P_{u,i_2}}{P_u^{(1)}} P_{i_2}}{1-P_{i_2}} \right) \end{aligned}$$

  (8)

  where $P_u^{(1)}$ is the updated conditional probability of $X_{u} = 1$ using the evidence $X_{i_1} = 0$. Note that the update factor at the end of Eq. (8) is of the same form of Eq. (3).

• $n = k$: More generally, we can similarly derive that

  $$\begin{aligned} P_u^{(k)}&= P_u^{(k-1)} \left( \frac{1 - \frac{P_{u,i_k}}{P_u^{(k-1)}} P_{i_k}}{1-P_{i_k}} \right) \end{aligned}$$

  (9)

  Hence, through Eq. (9), we can update each as-yet-unchosen image $u$ conditional probability $P_u^{(k)}$ in a iterative fashion each time we add a new image to the ranking list $RL$.