This paper discusses the effectiveness of neural networks for modeling dynamic systems as follows. The neural network consists of the input, state and output layers. The state layer comprises unit delay feedback connections in order to enable the expression of dynamic systems. In the first approach, the neural network is used to implement the external description model for a class of dynamic systems. It is shown that the neural network gives better estimation results and compares favorably with a linear regression model. In the second approach, as internal description models, linear and bilinear models are obtained by using the previously estimated results. That is, these models can be calculated by using the first and second approximations of sigmoid functions at threshold, according to Taylor's formula. Further, reduced-order realizations for each model are executed by using internally balanced realization methods, by which the singular values of reachability and observability matrices are given equally. Finally, some numerical examples are presented to illustrate the proposed methods.