This study constructed a multi-layered structure of adjective pairs expressing the tactile properties of materials. We analyzed the causalities between adjectives by using the DEMATEL method involving twenty-nine adjective pairs and forty-six materials. Based on the evaluated causalities, we constructed a multi-layered structure, which contained three types of perceptual layers expressing tactile textures, i.e.: psychophysical, affective, and preferential layers. The psychophysical layer consisted of adjectives related to the physical properties of materials, which were rough/smooth, uneven/flat, hard/soft, cold/warm, sticky/slippery, and wet/dry. The affective layer expresses affective aspects such as comfortable/uncomfortable and friendly/unfriendly. The preferential layer, which is the highest layer of the structure, included adjectives related to individual preferences: rich/poor, good/bad, like/dislike, and happy/sad. The constructed structure supports us to understand affective and preferential perceptions of materials through psychophysical perceptions.