Nathaniel Hawthorne did a splendid job with symbolism throughout The Scarlet Letter. Everything in the story was a shrewdly placed symbol of something deeper. This was even true of the characters. I believe this is true of Roger Chillingworth. It is hard to think of Chillingworth as a character at all. He had little depth; we knew little of him besides his self-consuming desire to find and torment Hester's associate in sin.
As soon as Chillingworth is introduced to the audience, it is apparent that he is a frightening, sinister character. The book describes him as, "clad, in a strange disarray of civilized and savage costume,"" giving you the image that Chillingworth is not entirely sane. Hester also presents an instant fear of him (" at the first instant of perceiving that thing visage, and the slight deformity of the figure, she pressed the infant to her bosom with so convulsive a force that the poor babe uttered a cry of pain."") which shows that she knows that the man will want vengeance in one way or another. She knew of this apparent imbalance of mind even before it became apparent to us.
Of course, Chillingworth's intentions became obvious later. (" Never, sayest thou?- rejoined [Chillingworth], with a smile of dark and self-relying intelligence. Never know him! Believe me, Hester, there are few things hidden from the man who devotes himself earnestly and unreservedly to the solution of a mystery.-). After this point in the story, Chillingworth is mostly characterized by his methodical and maniacal torture of Dimmesdale.
In the The Scarlet Letter, the audience sees more and more subtle changes as Chillingworth becomes depleted with hate. His physical distortion is mentioned in connection to a physical representation of the rancor that fills him. The dimension of his silent rage transformed him from a little mentioned man of knowledge to a creature who is hateful and immoral.