The automatic generation of metaphorical expressions helps us write imaginative texts such as poems or novels. This paper proposes a new metaphor generation task, evaluation metrics, and a method to solve the task. Our task is formalized as a problem of finding metaphorical paraphrases for a literal Japanese phrase consisting of a subject, an object, and a verb. We use four evaluation metrics: synonymousness, metaphoricity, novelty, and comprehensibility. Our proposed method generates metaphorical expressions by using three automatically computable scores—similarity, figurativeness, and rarity—corresponding to one of the evaluation metrics. By crowdsourcing, we show how these scores are related to those given by humans in terms of the evaluation metrics and how they are useful in finding human’s preferred expressions in pairwise comparisons.