Due to the informal and freely constructed sentence structures, it is a difficult classification task to detect the sentiment polarity of tweets, especially for multi-class cases. Extracting features with more valuable information from tweets is crucial for sentiment analysis. In this paper, to address this problem, a hybrid feature space combining bag-of-words and word embedding, named as Discriminative Sentiment Chunk (DSC) vector, is proposed. Then a semi-supervised method is proposed based on Autoencoder technique to learn discriminative sentiment chunk vectors, which convert a high dimensional bag-of-words vector into a continuous vector space with lower dimension without losing the chunk order. Our experimental results show that using discriminative sentiment chunks gains better accuracies and F1 scores on different twitter datasets and outperforms some popular bag-of-words oriented methods and a few deep network approaches