This may seem like a no brainer, but this is a very common problem for new photographers.
There are some simple techniques to follow that will help you steady your camera and take better photographs.
Most digital cameras now have really nice LCD screens and they are tempting to use, but you have to hold the camera away from your body to see the screen.
This will cause more movement and especially in low light situations even the slightest movements will create blurry images in your photographs.
This is because the shutter is open for a longer period of time in low light situations.
The only way to produce a photograph without any blurring is to use a tripod.
If you don't have a tripod available then the best way to avoid camera shake is to hold the camera with two hands.
You will probably be tempted to use one hand but two hands will give you more stability.
There is really no single right way to hold a camera.
It depends on the make of the camera and the individual user.
There are some techniques that do work well.

• Grip the right side of the camera with the right hand.
  Most cameras now have hand grips on them.
  Your right thumb should grip the back of the camera.
  Hold tightly.
  
• Your left hand should hold most of the weight of the camera.
  It should grip around the lens of the camera.
  
• Tuck your elbows into your sides and hold the camera close to your body while looking through the viewfinder.
  
• You should stand with your feet shoulder width apart or lean against something or kneel.
  This will give you added stability and prevent you from moving.
  
• Be conscience of your breathing.
  It is helpful to take a breath hold it and shoot.
  This also helps steady the camera.
  

You could also increase your shutter speed or buy special stabilizing lenses to decrease blurring, but holding the camera close to your body creates the same results.
Everyone will have there own techniques they feel comfortable with and prefer to use.