The confusion about Miles and Flora is in some extent caused by Mrs. Grose's unclear statements but definitely altered through the governess perception.
The governess receives no clear information about Miles' dismissal nor does Mrs. Grose give any details about the possible reasons for Miles' naughtiness. The veiled communication of the women characterizes the novel as much as the narrowed perception of the governess and her assumptions. Both women do not talk straight about what happened at Bly, especially about the mysterious case of Miss Jessel and Peter Quint. In this, also the children seem to follow a kind of conspiracy in not casting a light at this case; Miles avoids the topic consistently. .
An argument supportive to the deceptive nature of the governess is her invented communications with the ghosts, in particular on one encounter with the ghost of Miss Jessel. When asked by Mrs. Grose the governess says that the ghost was telling her about the hell's torments she suffered and that she wanted to take Flora with her. When asked by Mrs. Grose about the exact words, the governess replies " It came to that."" , meaning that it expressed just that. .
It can be argued of course that the governess is highly sensitive and might be able to read the horror in Miss Jessel's face. But it could also be argued that the governess is delusional, that she invents these instances because of her inmost fear that she might end as Miss Jessel - in misery with an illegitimate child and unrequited love.
A person not present but highly communicative is the uncle who at the beginning of the story instructs the governess not to contact him at all about the children but rely on herself. He is a young and attractive man who uses his power over the governess - and other women - to get them to agree to his demands. This exertion of his influence on the governess has several effects; the governess assumes that the master has faith in her judgements and that leaves all commitment to her decisions.