There are various possibilities. A common one is something you may have done or do that makes him uneasy. For example, you may have startled him if you dropped things or carry large objects or maybe sneezed or coughed. Maybe you move fast or talk in a certain way. Maybe he's protective of the children and growls when you approach. It would be interesting to see in what context the growling occurs to identify what triggers him (e.g. walking close too him, coming close to your son or daughter). However, if you mention this happens always, then it likely it started with something you did in the past (or keep on doing) and then spread to your mere presence, hence always evoking the growling.